Skip to content

Commit a76587b

Browse files
docs update (#247)
Add region-based enrichment caching doc using ZO_ENRICHMENT_TABLE_GET_REGION Document the query_recommendations stream, its usage and structure Correction on the organizations.md file Add match_all_hash UDF usage to Sensitive Data Redaction guide
1 parent 20ee715 commit a76587b

14 files changed

+152
-60
lines changed

docs/environment-variables.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,6 +333,8 @@ OpenObserve is configured using the following environment variables.
333333
| ZO_NATS_DELIVER_POLICY | all | Starting point in the stream for message delivery. Allowed values are all, last, new. |
334334
| ZO_NATS_SUB_CAPACITY | 65535 | Maximum subscription capacity. |
335335
| ZO_NATS_QUEUE_MAX_SIZE | 2048 | Maximum queue size in megabytes. |
336+
| ZO_NATS_KV_WATCH_MODULES | 2048 | Defines which internal modules use the NATS Key-Value Watcher instead of the default NATS Queue for event synchronization. Add one or more module prefixes separated by commas, such as /nodes/ or /user_sessions/. When left empty, all modules use the default NATS Queue mechanism. |
337+
| ZO_NATS_EVENT_STORAGE | memory | Controls how NATS JetStream stores event data. Use memory for high-speed, in-memory event storage or file for durable, disk-based storage that persists across restarts. <br> Performance Benchmark Results: <br> • File Storage: 10,965 ops/sec (10.71 MB/s throughput, ~911 µs mean latency) <br>• Memory Storage: 16,957 ops/sec (16.56 MB/s throughput, ~589 µs mean latency) <br> Memory storage offers ~55 percent higher throughput and lower latency, while file storage ensures durability. |
336338

337339

338340
## S3 and Object Storage
145 KB
Loading
144 KB
Loading

docs/images/match-all-hash.png

230 KB
Loading
192 KB
Loading
195 KB
Loading
183 KB
Loading
145 KB
Loading

docs/user-guide/enrichment-tables/enrichment-table-upload-recovery.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,4 +59,25 @@ When no local disk cache is available:
5959
- The querier fetches the latest enrichment data from the metadata database, such as PostgreSQL, and the remote storage system, such as S3. It then provides the data to the restarting node.
6060

6161

62+
## Region-based caching in multi-region super clusters
63+
In a multi-region super cluster deployment, enrichment tables are typically queried from all regions when a node starts up and rebuilds its cache. While this ensures data completeness, it can slow startup or cause failures if one or more regions are unavailable.
64+
65+
To address this, OpenObserve Enterprise supports primary region–based caching, controlled by the environment variable `ZO_ENRICHMENT_TABLE_GET_REGION`.
66+
67+
### Requirements
68+
69+
- Available only in Enterprise Edition.
70+
- Requires Super Cluster to be enabled.
71+
- The `ZO_ENRICHMENT_TABLE_GET_REGION` variable must specify a valid region name.
72+
73+
### How it works
74+
When a node starts, OpenObserve calls internal methods such as `get_enrichment_table_data()` and `cache_enrichment_tables()` to retrieve enrichment table data. <br>
75+
The boolean parameter `apply_primary_region_if_specified` controls whether to use only the primary region for these fetch operations.
76+
77+
In a multi-region super cluster deployment, when `apply_primary_region_if_specified = true`, OpenObserve checks the value of `ZO_ENRICHMENT_TABLE_GET_REGION`.
78+
79+
- If `ZO_ENRICHMENT_TABLE_GET_REGION` specifies a primary region, the node queries only that region to fetch enrichment table data during cache initialization.
80+
- If `ZO_ENRICHMENT_TABLE_GET_REGION` is not set, or the region name is empty, OpenObserve continues to query all regions as before.
81+
82+
6283

docs/user-guide/identity-and-access-management/organizations.md

Lines changed: 36 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -11,48 +11,48 @@ Organizations provide logical boundaries for separating data, users, and access
1111
1212
![Organizations in OpenObserve](../../images/organization-in-openobserve.png)
1313

14-
## Organization Types
14+
## Organization types
1515

1616
OpenObserve supports two types of organizations:
1717

1818
- **Default organization:** Automatically created for each user upon account creation. Typically named **default** and owned by the user. The UI labels it as type **default**.
1919
- **Custom organization:** Any organization other than the **default**. These are created manually using the UI or ingestion (if enabled). Displayed in the UI as type **custom**.
2020

21-
!!! Info "What Is **_meta** Organization?"
22-
**_meta Organization** is considered as a **custom** organization. It is a system-level organization that exists in both single-node and multi-node (HA) deployments.
23-
24-
- The **_meta** organization provides visibility into the health and status of the OpenObserve instance, including node metrics, resource usage, and configuration across all organizations.
25-
- Use the **IAM > Roles > Permission** in the **_meta** organization to manage users across all organizations and control who can list, create, update, or delete organizations.
26-
27-
## Access
28-
29-
In OpenObserve, access to organization-level operations, such as listing, creating, updating, or deleting organizations, depends on the deployment mode.
30-
31-
### Open-Source Mode
32-
Any authenticated user can create new organizations using the Add Organization button in the UI.
33-
### Enterprise Mode with RBAC Enabled
34-
- Access to organization management is strictly controlled through RBAC, which must be configured in the _meta organization.
35-
- The **root** user always has unrestricted access to all organizations, including **_meta**.
36-
- Only roles defined in **_meta** can include permissions for managing organizations.
37-
- The **organization** module is available in the role editor only within the **_meta** organization.
38-
39-
!!! Info "How to Grant Organization Management Access?"
40-
To delegate organization management to users in enterprise mode:
41-
42-
1. Switch to the **_meta** organization.
43-
2. Go to **IAM > Roles**.
44-
3. Create a new role or edit an existing one.
45-
4. In the **Permissions** tab, locate the Organizations module.
46-
5. Select the required operations:
47-
48-
- **List**: View existing organizations
49-
- **Create**: Add new organizations
50-
- **Update**: Modify organization details
51-
- **Delete**: Remove organizations
52-
6. Click **Save**. <br>
53-
![Grant Organization Management Access in OpenObserve](../../images/organization-role-permission.png)
54-
55-
Once this role is assigned to a user within the **_meta** organization, they will have access to manage organizations across the system.
21+
### _meta organization
22+
**_meta Organization** is considered as a **custom** organization. It is a system-level organization that exists in both single-node and multi-node (HA) deployments.
23+
24+
- The **_meta** organization provides visibility into the health and status of the OpenObserve instance, including node metrics, resource usage, and configuration across all organizations.
25+
- Use the **IAM > Roles > Permission** in the **_meta** organization to manage users across all organizations and control who can list, create, update, or delete organizations.
26+
27+
!!! note "Who can access"
28+
## Who can access
29+
In OpenObserve, access to organization-level operations, such as listing, creating, updating, or deleting organizations, depends on the deployment mode.
30+
31+
### Access in the open-source mode
32+
Any authenticated user can create new organizations using the **Add Organization** button in the UI.
33+
### Access in the enterprise mode with RBAC enabled
34+
- Access to organization management is strictly controlled through RBAC, which must be configured in the _meta organization.
35+
- The **root** user always has unrestricted access to all organizations, including **_meta**.
36+
- Only roles defined in **_meta** can include permissions for managing organizations.
37+
- The **organization** module is available in the role editor only within the **_meta** organization.
38+
39+
## How to grant organization management access?
40+
To delegate organization management to users in enterprise mode:
41+
42+
1. Switch to the **_meta** organization.
43+
2. Go to **IAM > Roles**.
44+
3. Create a new role or edit an existing one.
45+
4. In the **Permissions** tab, locate the Organizations module.
46+
5. Select the required operations:
47+
48+
- **Create**: Add new organizations
49+
- **Update**: Modify organization details
50+
!!! note "Note"
51+
By default, OpenObserve displays the list of organizations a user belongs to. You do not need to explicitly grant permission to view or retrieve organization details.
52+
6. Click **Save**. <br>
53+
![Grant Organization Management Access in OpenObserve](../../images/organization-role-permission.png)
54+
55+
Once this role is assigned to a user within the **_meta** organization, they will have access to manage organizations across the system.
5656

5757

5858
## Create an Organization

0 commit comments

Comments
 (0)