Bug 2184046 - Slow search when using filter with a virtual attribute (eg: nsRole ).
Summary: Slow search when using filter with a virtual attribute (eg: nsRole ).
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Directory Server
Classification: Red Hat
Component: 389-ds-base
Version: 11.6
Hardware: x86_64
OS: Linux
high
urgent
Target Milestone: DS12.3
: dirsrv-12.3
Assignee: thierry bordaz
QA Contact: LDAP QA Team
Evgenia Martynyuk
URL:
Whiteboard: sync-to-jira
Depends On:
Blocks: 2265530 2265536
TreeView+ depends on / blocked
 
Reported: 2023-04-03 13:43 UTC by Têko Mihinto
Modified: 2024-02-22 15:38 UTC (History)
9 users (show)

Fixed In Version: redhat-ds-12-9030020230711000312-1674d57
Doc Type: Enhancement
Doc Text:
Feature: A filter component may contain 'nsrole' attribute that can not be indexed. WIth this feature the component is rewritten into another component that can be indexed. This RFE only applies to managed and filtered roles but not to nested roles. Reason: the virtual attribute 'nsrole' is supported in filter component. As a virtual attribute it can not be indexed and the component is unindexed. If the others components of the filter are not indexed then the search will be unindexed and result in very long operation. Result: 'nsroles' attribute used in filter component are rewritten and then can be indexed. Resulting with better response time. Only for managed and filtered roles.
Clone Of:
: 2265530 2265536 (view as bug list)
Environment:
Last Closed: 2023-11-21 15:13:16 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github 389ds 389-ds-base issues 5722 0 None open RFE When a filter contains 'nsrole', improve response time by rewriting the filter. 2023-04-07 15:12:46 UTC
Red Hat Issue Tracker IDMDS-2920 0 None None None 2023-04-04 14:17:28 UTC
Red Hat Issue Tracker IDMDS-3522 0 None None None 2023-08-09 09:39:34 UTC
Red Hat Product Errata RHEA-2023:7429 0 None None None 2023-11-21 15:13:34 UTC

Description Têko Mihinto 2023-04-03 13:43:32 UTC
Description of problem:

This customer has an LDAP DB of 1.2 million of entries.
There are applications which are running queries using the nsRole virtual attribute.
For instance:
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName "

The initial query is taking too long to complete and applications are hitting their timeouts.
Subsequent queries are faster.

1st run:
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName "
RESULT err=0 tag=101 nentries=11 wtime=0.000070735 optime=179.484774853 etime=179.484840590 notes=A details="Fully Unindexed Filter"

Following queries
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName"
RESULT err=0 tag=101 nentries=11 wtime=0.000060507 optime=0.558663030 etime=0.558718327 notes=A details="Fully Unindexed Filter


Version-Release number of selected component (if applicable):

$ cat <SOS_REPORT>/etc/redhat-release
Red Hat Enterprise Linux release 8.6 (Ootpa)
$
$ grep ^389-ds <SOS_REPORT>/installed-rpms
389-ds-base-1.4.3.29-3.module+el8dsrv+14615+a86efbbf.x86_64 Mon Sep  5 10:44:21 2022
389-ds-base-libs-1.4.3.29-3.module+el8dsrv+14615+a86efbbf.x86_64 Mon Sep  5 10:44:19 2022
$

How reproducible:
Always.

Steps to Reproduce:
1. Create a DB with a million of entries
2. Add the nsRoleDN attribute to 20K entries
3. Run searches using the nsRole attribute

Actual results:
Slow searches

Expected results:
Faster results so applications won't timeout.
After increasing the Normalized DN cache, subsequent queries are faster.
Now the concern is about how to improve the first search.

Additional info:
* Upstream ticket:
    https://github.com/389ds/389-ds-base/issues/5695
* Indexing and virtual attributes:
    https://bugzilla.redhat.com/show_bug.cgi?id=1773643

Comment 21 thierry bordaz 2023-05-09 13:25:54 UTC
Fix is pushed upstream -> POST

Comment 28 Viktor Ashirov 2023-08-17 09:43:02 UTC
Automated test passed:
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.9.17, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
389-ds-base: 2.3.5-1.module+el9dsrv+19320+04706864
nss: 3.90.0-3.el9_2
nspr: 4.35.0-3.el9_2
openldap: 2.6.3-1.el9
cyrus-sasl: not installed
FIPS: disabled
rootdir: /root/ds/dirsrvtests, configfile: pytest.ini
collected 1 item

dirsrvtests/tests/suites/roles/basic_test.py::test_managed_and_filtered_role_rewrite PASSED                                                                                                                  [100%]

==================================================================================== 1 passed, 12 warnings in 72.19s (0:01:12) =====================================================================================

Marking as VERIFIED.

Comment 30 errata-xmlrpc 2023-11-21 15:13:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (redhat-ds:12 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:7429


Note You need to log in before you can comment on or make changes to this bug.