Есть запрос:
SELECT DISTINCT
nlevel(org.path),
org.path,
subpath(org.path,0,-1),
org.id,
name,
org.children_counter,
e.employee_id AS manager_employee_id,
e.fio AS manager_fio,
pos.position_id,
pos.pos_name,
to_char(pos.effective_start_date,'DD.MM.YYYY') AS effective_start_date,
coalesce(to_char(pos.effective_end_date,'DD.MM.YYYY'), '') AS effective_end_date,
pos.grade,
pos.employed,
man_pos.pos_name AS manager_pos_name,
spvisor.fio,
spvisor_pos.pos_name
FROM org_struct AS org
LEFT JOIN positions AS pos ON org.id = pos.org_id
LEFT JOIN employees AS e ON org.manager_position_id = e.position_id AND e.activity = true AND NOT e.internal_by_worker
LEFT JOIN positions AS man_pos ON org.manager_position_id = man_pos.position_id
LEFT JOIN employees AS spvisor ON pos.position_id = spvisor.position_id AND spvisor.activity = true AND NOT spvisor.internal_by_worker AND NOT pos.managers_equality
LEFT JOIN positions AS spvisor_pos ON spvisor_pos.position_id = spvisor.position_id
WHERE path <@ '0' AND ( (nlevel(org.path) = nlevel('0')) OR (nlevel(org.path) - 1 = nlevel('0')) )
ORDER BY nlevel(org.path), org.id;
Созданные индексы, которые на крохи ускоряют работу запроса:
CREATE INDEX IF NOT EXISTS positions_cover ON positions (position_id, pos_name);
CREATE INDEX IF NOT EXISTS positions_cover_1 ON positions_20200731t090000 (position_id, pos_name);
CREATE INDEX IF NOT EXISTS org_manager_position_id ON org_struct USING hash (manager_position_id);
CREATE INDEX IF NOT EXISTS org_manager_position_id_1 ON org_struct_20200731t090000 USING hash (manager_position_id);
CREATE INDEX IF NOT EXISTS employees_position_id ON employees USING hash (position_id);
CREATE INDEX IF NOT EXISTS employees_position_id_1 ON employees_20200731t090000 USING hash (position_id);
В запросе в условии WHERE path <@ '0' AND ( (nlevel(org.path) = nlevel('0')) OR (nlevel(org.path) - 1 = nlevel('0')) ) ORDER BY nlevel(org.path), org.id;
вместо '0'
может стоять любой необходимый path
и вместо nlevel(org.path)-1
может быть nlevel(org.path)-4
например.
Используется наследование таблиц, поэтому создаются индексы для дочерних таблиц.
Результат explain analyse
:
Unique (cost=9151760368251.44..9485933360431.67 rows=7426066492894 width=445) (actual time=2934.328..2934.364 rows=32 loops=1)
-> Sort (cost=9151760368251.44..9170325534483.67 rows=7426066492894 width=445) (actual time=2934.327..2934.329 rows=32 loops=1)
" Sort Key: (nlevel(org.path)), org.id, org.path, (subpath(org.path, 0, '-1'::integer)), org.name, org.children_counter, e.employee_id, e.fio, pos.position_id, pos.pos_name, (to_char(pos.effective_start_date, 'DD.MM.YYYY'::text)), (COALESCE(to_char(pos.effective_end_date, 'DD.MM.YYYY'::text), ''::text)), pos.grade, pos.employed, man_pos.pos_name, spvisor.fio, spvisor_pos.pos_name"
Sort Method: quicksort Memory: 41kB
-> Nested Loop Left Join (cost=141953.05..76517052727.23 rows=7426066492894 width=445) (actual time=2926.233..2934.131 rows=32 loops=1)
Join Filter: (NOT pos.managers_equality)
Rows Removed by Join Filter: 16
-> Hash Right Join (cost=141953.05..6355371.34 rows=466140267 width=291) (actual time=2926.199..2933.608 rows=32 loops=1)
Hash Cond: (pos.org_id = org.id)
-> Append (cost=0.00..2015.56 rows=37038 width=81) (actual time=0.047..11.617 rows=37037 loops=1)
-> Seq Scan on positions pos (cost=0.00..0.00 rows=1 width=1061) (actual time=0.007..0.007 rows=0 loops=1)
-> Seq Scan on positions_20200731t090000 pos_1 (cost=0.00..1830.37 rows=37037 width=81) (actual time=0.038..8.183 rows=37037 loops=1)
-> Hash (cost=36745.28..36745.28 rows=2517102 width=214) (actual time=2895.757..2895.757 rows=4 loops=1)
Buckets: 65536 Batches: 64 Memory Usage: 512kB
-> Hash Right Join (cost=2206.01..36745.28 rows=2517102 width=214) (actual time=2882.075..2894.603 rows=4 loops=1)
Hash Cond: (e.position_id = org.manager_position_id)
-> Append (cost=0.00..1325.01 rows=34410 width=36) (actual time=0.040..9.456 rows=34393 loops=1)
-> Seq Scan on employees e (cost=0.00..0.00 rows=1 width=524) (actual time=0.008..0.008 rows=0 loops=1)
Filter: (activity AND (NOT internal_by_worker))
-> Seq Scan on employees_20200731t090000 e_1 (cost=0.00..1152.96 rows=34409 width=36) (actual time=0.030..6.676 rows=34393 loops=1)
Filter: (activity AND (NOT internal_by_worker))
Rows Removed by Filter: 3303
-> Hash (cost=2023.13..2023.13 rows=14630 width=186) (actual time=2881.988..2881.988 rows=4 loops=1)
Buckets: 16384 Batches: 1 Memory Usage: 129kB
-> Nested Loop Left Join (cost=0.00..2023.13 rows=14630 width=186) (actual time=2878.217..2881.974 rows=4 loops=1)
-> Append (cost=0.00..1698.60 rows=79 width=142) (actual time=2878.159..2881.870 rows=4 loops=1)
-> Seq Scan on org_struct org (cost=0.00..0.00 rows=1 width=560) (actual time=0.012..0.012 rows=0 loops=1)
Filter: ((path <@ '0'::ltree) AND ((nlevel(path) = 1) OR ((nlevel(path) - 1) = 1)))
-> Seq Scan on org_struct_20200731t090000 org_1 (cost=0.00..1698.20 rows=78 width=137) (actual time=2878.145..2881.853 rows=4 loops=1)
Filter: ((path <@ '0'::ltree) AND ((nlevel(path) = 1) OR ((nlevel(path) - 1) = 1)))
Rows Removed by Filter: 9164
-> Append (cost=0.00..4.09 rows=2 width=47) (actual time=0.021..0.022 rows=0 loops=4)
-> Seq Scan on positions man_pos (cost=0.00..0.00 rows=1 width=520) (actual time=0.003..0.003 rows=0 loops=4)
Filter: (org.manager_position_id = position_id)
-> Index Only Scan using positions_cover_1 on positions_20200731t090000 man_pos_1 (cost=0.41..4.08 rows=1 width=47) (actual time=0.013..0.013 rows=0 loops=4)
Index Cond: (position_id = org.manager_position_id)
Heap Fetches: 0
-> Nested Loop Left Join (cost=0.00..1.13 rows=370 width=75) (actual time=0.013..0.013 rows=1 loops=32)
-> Append (cost=0.00..0.13 rows=2 width=32) (actual time=0.007..0.008 rows=1 loops=32)
-> Seq Scan on employees spvisor (cost=0.00..0.00 rows=1 width=520) (actual time=0.000..0.000 rows=0 loops=32)
Filter: (activity AND (NOT internal_by_worker) AND (pos.position_id = position_id))
-> Index Scan using employees_position_id_1 on employees_20200731t090000 spvisor_1 (cost=0.00..0.12 rows=1 width=32) (actual time=0.005..0.005 rows=1 loops=32)
Index Cond: (position_id = pos.position_id)
Filter: (activity AND (NOT internal_by_worker))
-> Append (cost=0.00..0.48 rows=2 width=47) (actual time=0.006..0.006 rows=1 loops=26)
-> Seq Scan on positions spvisor_pos (cost=0.00..0.00 rows=1 width=520) (actual time=0.000..0.000 rows=0 loops=26)
Filter: (position_id = spvisor.position_id)
-> Index Only Scan using positions_cover_1 on positions_20200731t090000 spvisor_pos_1 (cost=0.41..0.47 rows=1 width=47) (actual time=0.004..0.004 rows=1 loops=26)
Index Cond: (position_id = spvisor.position_id)
Heap Fetches: 1
Planning Time: 2.121 ms
JIT:
Functions: 62
" Options: Inlining true, Optimization true, Expressions true, Deforming true"
" Timing: Generation 28.331 ms, Inlining 32.580 ms, Optimization 1839.802 ms, Emission 1003.417 ms, Total 2904.130 ms"
Execution Time: 2963.485 ms
Запрос работает почти 3 секунды, добавление новых индексов не помогает. Пытался переписать запрос со слиянием таблиц из подзапроса. Как еще можно переписать запрос или добавить необходимый индекс? Главное условие, чтобы не было повторений и отсортировано по nlevel(org.path) - (уровень в дереве подчиненных организаций), org.id. После добавления индексов
CREATE INDEX IF NOT EXISTS org_struct_path_gist_idx_1 ON org_struct_20200731t090000 USING GIST (path);
CREATE INDEX IF NOT EXISTS org_struct_path_nlevel_1 ON org_struct_20200731t090000 (nlevel(path));
план запроса следующий:
Unique (cost=9151760368251.44..9485933360431.67 rows=7426066492894 width=445) (actual time=2524.638..2524.669 rows=32 loops=1)
" Buffers: shared hit=3850, temp read=26 written=481"
-> Sort (cost=9151760368251.44..9170325534483.67 rows=7426066492894 width=445) (actual time=2524.636..2524.639 rows=32 loops=1)
" Sort Key: (nlevel(org.path)), org.id, org.path, (subpath(org.path, 0, '-1'::integer)), org.name, org.children_counter, e.employee_id, e.fio, pos.position_id, pos.pos_name, (to_char(pos.effective_start_date, 'DD.MM.YYYY'::text)), (COALESCE(to_char(pos.effective_end_date, 'DD.MM.YYYY'::text), ''::text)), pos.grade, pos.employed, man_pos.pos_name, spvisor.fio, spvisor_pos.pos_name"
Sort Method: quicksort Memory: 41kB
" Buffers: shared hit=3850, temp read=26 written=481"
-> Nested Loop Left Join (cost=141953.05..76517052727.23 rows=7426066492894 width=445) (actual time=2518.494..2524.434 rows=32 loops=1)
Join Filter: (NOT pos.managers_equality)
Rows Removed by Join Filter: 16
" Buffers: shared hit=3850, temp read=26 written=481"
-> Hash Right Join (cost=141953.05..6355371.34 rows=466140267 width=291) (actual time=2518.462..2523.973 rows=32 loops=1)
Hash Cond: (pos.org_id = org.id)
" Buffers: shared hit=3715, temp read=26 written=481"
-> Append (cost=0.00..2015.56 rows=37038 width=81) (actual time=0.039..12.641 rows=37037 loops=1)
Buffers: shared hit=1460
-> Seq Scan on positions pos (cost=0.00..0.00 rows=1 width=1061) (actual time=0.007..0.007 rows=0 loops=1)
-> Seq Scan on positions_20200731t090000 pos_1 (cost=0.00..1830.37 rows=37037 width=81) (actual time=0.032..8.757 rows=37037 loops=1)
Buffers: shared hit=1460
-> Hash (cost=36745.28..36745.28 rows=2517102 width=214) (actual time=2485.630..2485.630 rows=4 loops=1)
Buckets: 65536 Batches: 64 Memory Usage: 512kB
Buffers: shared hit=2255
-> Hash Right Join (cost=2206.01..36745.28 rows=2517102 width=214) (actual time=2471.818..2485.328 rows=4 loops=1)
Hash Cond: (e.position_id = org.manager_position_id)
Buffers: shared hit=2255
-> Append (cost=0.00..1325.01 rows=34410 width=36) (actual time=0.029..9.942 rows=34393 loops=1)
Buffers: shared hit=776
-> Seq Scan on employees e (cost=0.00..0.00 rows=1 width=524) (actual time=0.007..0.007 rows=0 loops=1)
Filter: (activity AND (NOT internal_by_worker))
-> Seq Scan on employees_20200731t090000 e_1 (cost=0.00..1152.96 rows=34409 width=36) (actual time=0.020..6.781 rows=34393 loops=1)
Filter: (activity AND (NOT internal_by_worker))
Rows Removed by Filter: 3303
Buffers: shared hit=776
-> Hash (cost=2023.13..2023.13 rows=14630 width=186) (actual time=2471.739..2471.740 rows=4 loops=1)
Buckets: 16384 Batches: 1 Memory Usage: 129kB
Buffers: shared hit=1479
-> Nested Loop Left Join (cost=0.00..2023.13 rows=14630 width=186) (actual time=2468.608..2471.730 rows=4 loops=1)
Buffers: shared hit=1479
-> Append (cost=0.00..1698.60 rows=79 width=142) (actual time=2468.561..2471.660 rows=4 loops=1)
Buffers: shared hit=1469
-> Seq Scan on org_struct org (cost=0.00..0.00 rows=1 width=560) (actual time=0.013..0.013 rows=0 loops=1)
Filter: ((path <@ '0'::ltree) AND ((nlevel(path) = 1) OR ((nlevel(path) - 1) = 1)))
-> Seq Scan on org_struct_20200731t090000 org_1 (cost=0.00..1698.20 rows=78 width=137) (actual time=2468.544..2471.642 rows=4 loops=1)
Filter: ((path <@ '0'::ltree) AND ((nlevel(path) = 1) OR ((nlevel(path) - 1) = 1)))
Rows Removed by Filter: 9164
Buffers: shared hit=1469
-> Append (cost=0.00..4.09 rows=2 width=47) (actual time=0.013..0.013 rows=0 loops=4)
Buffers: shared hit=10
-> Seq Scan on positions man_pos (cost=0.00..0.00 rows=1 width=520) (actual time=0.002..0.002 rows=0 loops=4)
Filter: (org.manager_position_id = position_id)
-> Index Only Scan using positions_cover_1 on positions_20200731t090000 man_pos_1 (cost=0.41..4.08 rows=1 width=47) (actual time=0.007..0.007 rows=0 loops=4)
Index Cond: (position_id = org.manager_position_id)
Heap Fetches: 0
Buffers: shared hit=10
-> Nested Loop Left Join (cost=0.00..1.13 rows=370 width=75) (actual time=0.011..0.012 rows=1 loops=32)
Buffers: shared hit=135
-> Append (cost=0.00..0.13 rows=2 width=32) (actual time=0.006..0.006 rows=1 loops=32)
Buffers: shared hit=55
-> Seq Scan on employees spvisor (cost=0.00..0.00 rows=1 width=520) (actual time=0.000..0.000 rows=0 loops=32)
Filter: (activity AND (NOT internal_by_worker) AND (pos.position_id = position_id))
-> Index Scan using employees_position_id_1 on employees_20200731t090000 spvisor_1 (cost=0.00..0.12 rows=1 width=32) (actual time=0.003..0.004 rows=1 loops=32)
Index Cond: (position_id = pos.position_id)
Filter: (activity AND (NOT internal_by_worker))
Buffers: shared hit=55
-> Append (cost=0.00..0.48 rows=2 width=47) (actual time=0.005..0.006 rows=1 loops=26)
Buffers: shared hit=80
-> Seq Scan on positions spvisor_pos (cost=0.00..0.00 rows=1 width=520) (actual time=0.000..0.000 rows=0 loops=26)
Filter: (position_id = spvisor.position_id)
-> Index Only Scan using positions_cover_1 on positions_20200731t090000 spvisor_pos_1 (cost=0.41..0.47 rows=1 width=47) (actual time=0.004..0.004 rows=1 loops=26)
Index Cond: (position_id = spvisor.position_id)
Heap Fetches: 1
Buffers: shared hit=80
Planning Time: 2.479 ms
JIT:
Functions: 62
" Options: Inlining true, Optimization true, Expressions true, Deforming true"
" Timing: Generation 27.421 ms, Inlining 28.490 ms, Optimization 1605.063 ms, Emission 833.284 ms, Total 2494.258 ms"
Execution Time: 2552.608 ms
Пример 'path': '0.123.2345', где 0, 123, 2345 id организаций, nlevel(0.123.2345) = 3
стандартная функция ltree в postgres