New docs: Query recommendations and Histogram caching. Update docs: Env variables page, Schema settings... (#251)

* update docs (#250)

* update docs (#252)

Built in regex pattern
Legend and gridline config in dashboards
Env var for shared memtable
Partial data error in dashboards
Explore and analyze feature
Incorporate changes after aggregation cache and streaming aggregation are removed from UI

Author: DebashisBorgohainO2

docs/environment-variables.md

@@ -15,6 +15,11 @@ OpenObserve is configured using the following environment variables.
 | ZO_LOCAL_MODE | true | If local mode is set to true, OpenObserve becomes single node deployment.If it is set to false, it indicates cluster mode deployment which supports multiple nodes with different roles. For local mode one needs to configure SQLite DB, for cluster mode one needs to configure PostgreSQL (recommended) or MySQL. |
 | ZO_LOCAL_MODE_STORAGE | disk | Applicable only for local mode. By default, local disk is used as storage. OpenObserve supports both disk and S3 in local mode. |
 | ZO_NODE_ROLE | all | Node role assignment. Possible values are ingester, querier, router, compactor, alertmanager, and all. A single node can have multiple roles by specifying them as a comma-separated list. For example, compactor, alertmanager. |
+| ZO_NODE_ROLE_GROUP | "" | Each query-processing node can be assigned to a specific group using ZO_NODE_ROLE_GROUP. <br>
+**interactive**: Handles queries triggered directly by users through the UI. <br>
+**background**: Handles automated or scheduled queries, such as alerts and reports. <br>
+**empty string** (default): Handles all query types. <br>
+In high-load environments, alerts or reports might run large, resource-intensive queries. By assigning dedicated groups, administrators can prevent such queries from blocking or slowing down real-time user searches. |
 | ZO_NODE_HEARTBEAT_TTL | 30 | Time-to-live (TTL) for node heartbeats in seconds. |
 | ZO_INSTANCE_NAME | - | In the cluster mode, each node has a instance name. Default is instance hostname. |
 | ZO_CLUSTER_COORDINATOR | nats | Defines how nodes in the cluster discover each other. |
@@ -82,7 +87,19 @@ OpenObserve is configured using the following environment variables.
 | ZO_FILE_MOVE_FIELDS_LIMIT | 2000 | Field count threshold per WAL file. If exceeded, merging is skipped on the ingester. |
 | ZO_MEM_TABLE_MAX_SIZE | 0 | Total size limit of all memtables. Multiple memtables exist for different organizations and stream types. Each memtable cannot exceed ZO_MAX_FILE_SIZE_IN_MEMORY, and the combined size cannot exceed this limit. If exceeded, the system returns a MemoryTableOverflowError to prevent out-of-memory conditions. Default is 50 percent of total memory. |
 | ZO_MEM_PERSIST_INTERVAL | 5 | Interval in seconds at which immutable memtables are persisted from memory to disk. Default is 5 seconds. |
-
+| ZO_FEATURE_SHARED_MEMTABLE_ENABLED | false | When set to true, it turns on the shared memtable feature and several organizations can use the same in-memory table instead of each organization creating its own. This helps reduce memory use when many organizations send data at the same time. It also works with older non-shared write-ahead log (WAL) files. |
+| ZO_MEM_TABLE_BUCKET_NUM | 1 | This setting controls how many in-memory tables OpenObserve creates, and works differently depending on whether shared memtable is enabled or disabled. <br> 
+**When ZO_FEATURE_SHARED_MEMTABLE_ENABLED is true (shared memtable enabled)**:
+<br>
+OpenObserve creates the specified number of shared in-memory tables that all organizations use together. <br> **If the number is higher**: OpenObserve creates more shared tables. Each table holds data from fewer organizations. This can make data writing faster because each table handles less data. However, it also uses more memory. <br>
+**If the number is lower**: OpenObserve creates fewer shared tables. Each table holds data from more organizations. This saves memory but can make data writing slightly slower when many organizations send data at the same time.
+<br>
+**When ZO_FEATURE_SHARED_MEMTABLE_ENABLED is false (shared memtable disabled)**:
+<br>
+Each organization creates its own set of in-memory tables based on the ZO_MEM_TABLE_BUCKET_NUM value.
+<br>
+For example, if ZO_MEM_TABLE_BUCKET_NUM is set to 4, each organization will create 4 separate in-memory tables.
+This is particularly useful when you have only one organization, as creating multiple in-memory tables for that single organization can improve ingestion performance.|
 
 ## Indexing
 | Environment Variable | Default Value | Description |
@@ -573,9 +590,9 @@ OpenObserve is configured using the following environment variables.
 | Environment Variable             | Default Value | Description                                             |
 | -------------------------------- | --------- | ------------------------------------------------------- |
 | ZO_QUICK_MODE_ENABLED            | false | Indicates if quick mode is enabled.                     |
-| ZO_QUICK_MODE_NUM_FIELDS         | 500   | The number of fields to consider for quick mode.        |
 | ZO_QUICK_MODE_STRATEGY           |  | Possible values are `first`, `last`, `both`.               |
-| ZO_QUICK_MODE_FORCE_ENABLED           | true | |
+| ZO_QUICK_MODE_FORCE_ENABLED           | true | Enables automatic activation of Quick Mode from the backend. When set to true, OpenObserve applies Quick Mode automatically if the number of fields in a stream exceeds the limit defined by `ZO_QUICK_MODE_NUM_FIELDS`, even when the Quick Mode toggle in the UI is turned off.|
+| ZO_QUICK_MODE_NUM_FIELDS         | 500   | This defines the number of fields beyond which the quick mode will be force enabled.        |
 
 ## Miscellaneous 
 | Environment Variable | Default Value | Description |