Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions modules/otel-exporters-aws-cloudwatch-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-aws-cloudwatch-exporter_{context}"]
= AWS CloudWatch Logs Exporter

[role="_abstract"]
The AWS CloudWatch Logs Exporter sends logs data to the Amazon CloudWatch Logs service and signs requests by using the AWS SDK for Go and the default credential provider chain.

:FeatureName: The AWS CloudWatch Logs Exporter
include::snippets/technology-preview.adoc[]

.OpenTelemetry Collector custom resource with the enabled AWS CloudWatch Logs Exporter
[source,yaml]
----
# ...
config:
exporters:
awscloudwatchlogs:
log_group_name: "<group_name_of_amazon_cloudwatch_logs>" # <1>
log_stream_name: "<log_stream_of_amazon_cloudwatch_logs>" # <2>
region: <aws_region_of_log_stream> # <3>
endpoint: <protocol><service_endpoint_of_amazon_cloudwatch_logs> # <4>
log_retention: <supported_value_in_days> # <5>
# ...
----
<1> Required. If the log group does not exist yet, it is automatically created.
<2> Required. If the log stream does not exist yet, it is automatically created.
<3> Optional. If the AWS region is not already set in the default credential chain, you must specify it.
<4> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
<5> Optional. With this parameter, you can set the log retention policy for new Amazon CloudWatch log groups. If this parameter is omitted or set to `0`, the logs never expire by default. Supported values for retention in days are `1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `2192`, `2557`, `2922`, `3288`, or `3653`.

[role="_additional-resources"]
.Additional resources
* link:https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html[What is Amazon CloudWatch Logs?] (Amazon CloudWatch Logs User Guide)
* link:https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials[Specifying Credentials] (AWS SDK for Go Developer Guide)
// * link:https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/configure-auth.html[Configure Authentication] (AWS SDK for Go v2 Developer Guide)
* link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference)
72 changes: 72 additions & 0 deletions modules/otel-exporters-aws-emf-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-aws-emf-exporter_{context}"]
= AWS EMF Exporter

[role="_abstract"]
The AWS EMF Exporter converts the following OpenTelemetry metrics datapoints to the AWS CloudWatch Embedded Metric Format (EMF):

- `Int64DataPoints`
- `DoubleDataPoints`
- `SummaryDataPoints`

The EMF metrics are then sent directly to the Amazon CloudWatch Logs service by using the `PutLogEvents` API.

One of the benefits of using this exporter is the possibility to view logs and metrics in the Amazon CloudWatch console at link:https://console.aws.amazon.com/cloudwatch/[].

:FeatureName: The AWS EMF Exporter
include::snippets/technology-preview.adoc[]

.OpenTelemetry Collector custom resource with the enabled AWS EMF Exporter
[source,yaml]
----
# ...
config:
exporters:
awsemf:
log_group_name: "<group_name_of_amazon_cloudwatch_logs>" # <1>
log_stream_name: "<log_stream_of_amazon_cloudwatch_logs>" # <2>
resource_to_telemetry_conversion: # <3>
enabled: true
region: <region> # <4>
endpoint: <protocol><endpoint> # <5>
log_retention: <supported_value_in_days> # <6>
namespace: <custom_namespace> # <7>
# ...
----
<1> You can use the `log_group_name` parameter to customize the log group name or set the default `+/metrics/default+` value or the following placeholders:
+
The `+/aws/metrics/{ClusterName}+` placeholder is for searching for the `ClusterName` or `+aws.ecs.cluster.name+` resource attribute in the metrics data and replacing it with the actual cluster name.
+
The `+{NodeName}+` placeholder is for searching for the `NodeName` or `+k8s.node.name+` resource attribute.
+
The `+{TaskId}+` placeholder is for searching for the `TaskId` or `+aws.ecs.task.id+` resource attribute.
+
If no resource attribute is found in the resource attribute map, the placeholder is replaced by the `undefined` value.
<2> You can use the `log_stream_name` parameter to customize the log stream name or set the default `+otel-stream+` value or the following placeholders:
+
The `+{ClusterName}+` placeholder is for searching for the `+ClusterName+` or `+aws.ecs.cluster.name+` resource attribute.
+
The `+{ContainerInstanceId}+` placeholder is for searching for the `+ContainerInstanceId+` or `+aws.ecs.container.instance.id+` resource attribute. This resource attribute is valid only for the AWS ECS EC2 launch type.
+
The `+{NodeName}+` placeholder is for searching for the `+NodeName+` or `+k8s.node.name+` resource attribute.
+
The `+{TaskDefinitionFamily}+` placeholder is for searching for the `+TaskDefinitionFamily+` or `+aws.ecs.task.family+` resource attribute.
+
The `+{TaskId}+` placeholder is for searching for the `TaskId` or `+aws.ecs.task.id+` resource attribute in the metrics data and replacing it with the actual task ID.
+
If no resource attribute is found in the resource attribute map, the placeholder is replaced by the `undefined` value.
<3> Optional. Converts resource attributes to telemetry attributes such as metric labels. Disabled by default.
<4> The AWS region of the log stream. If a region is not already set in the default credential provider chain, you must specify the region.
<5> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
<6> Optional. With this parameter, you can set the log retention policy for new Amazon CloudWatch log groups. If this parameter is omitted or set to `0`, the logs never expire by default. Supported values for retention in days are `1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `2192`, `2557`, `2922`, `3288`, or `3653`.
<7> Optional. A custom namespace for the Amazon CloudWatch metrics.

[role="_additional-resources"]
.Additional resources
* link:https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html[Amazon CloudWatch User Guide: Specification: Embedded metric format]
* link:https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html[Amazon CloudWatch Logs API Reference: PutLogEvents]
* link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[AWS General Reference: Amazon CloudWatch Logs endpoints and quotas]
45 changes: 45 additions & 0 deletions modules/otel-exporters-aws-xray-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-aws-xray-exporter_{context}"]
= AWS X-Ray Exporter

[role="_abstract"]
The AWS X-Ray Exporter converts OpenTelemetry spans to AWS X-Ray Segment Documents and then sends them directly to the AWS X-Ray service. The AWS X-Ray Exporter uses the `PutTraceSegments` API and signs requests by using the AWS SDK for Go and the default credential provider chain.

:FeatureName: The AWS X-Ray Exporter
include::snippets/technology-preview.adoc[]

.OpenTelemetry Collector custom resource with the enabled AWS X-Ray Exporter
[source,yaml]
----
# ...
config:
exporters:
awsxray:
region: "<region>" # <1>
endpoint: <protocol><endpoint> # <2>
resource_arn: "<aws_resource_arn>" # <3>
role_arn: "<iam_role>" # <4>
indexed_attributes: [ "<indexed_attr_0>", "<indexed_attr_1>" ] # <5>
aws_log_groups: ["<group1>", "<group2>"] # <6>
request_timeout_seconds: 120 # <7>
# ...
----
<1> The destination region for the X-Ray segments sent to the AWS X-Ray service. For example, `eu-west-1`.
<2> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
<3> The Amazon Resource Name (ARN) of the AWS resource that is running the Collector.
<4> The AWS Identity and Access Management (IAM) role for uploading the X-Ray segments to a different account.
<5> The list of attribute names to be converted to X-Ray annotations.
<6> The list of log group names for Amazon CloudWatch Logs.
<7> Time duration in seconds before timing out a request. If omitted, the default value is `30`.

[role="_additional-resources"]
.Additional resources
* link:https://docs.aws.amazon.com/xray/latest/devguide/aws-xray.html[AWS X-Ray Developer Guide: What is AWS X-Ray?]
* link:https://docs.aws.amazon.com/sdk-for-go/api/index.html[AWS Documentation: AWS SDK for Go API Reference]
* link:https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials[AWS SDK for Go Developer Guide: Specifying Credentials]
// * link:https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/configure-auth.html[AWS SDK for Go v2 Developer Guide: Configure Authentication]
* link:https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html[AWS Identity and Access Management User Guide: IAM roles]
34 changes: 34 additions & 0 deletions modules/otel-exporters-debug-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-debug-exporter_{context}"]
= Debug Exporter

[role="_abstract"]
The Debug Exporter prints traces and metrics to the standard output.

.OpenTelemetry Collector custom resource with the enabled Debug Exporter
[source,yaml]
----
# ...
config:
exporters:
debug:
verbosity: detailed # <1>
sampling_initial: 5 # <2>
sampling_thereafter: 200 # <3>
use_internal_logger: true # <4>
service:
pipelines:
traces:
exporters: [debug]
metrics:
exporters: [debug]
# ...
----
<1> Verbosity of the debug export: `detailed`, `normal`, or `basic`. When set to `detailed`, pipeline data are verbosely logged. Defaults to `normal`.
<2> Initial number of messages logged per second. The default value is `2` messages per second.
<3> Sampling rate after the initial number of messages, the value in `sampling_initial`, has been logged. Disabled by default with the default `1` value. Sampling is enabled with values greater than `1`. For more information, see the page for the link:https://pkg.go.dev/go.uber.org/zap/zapcore?utm_source=godoc#NewSamplerWithOptions[sampler function in the `zapcore` package] on the Go Project's website.
<4> When set to `true`, enables output from the Collector's internal logger for the exporter.
41 changes: 41 additions & 0 deletions modules/otel-exporters-file-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-file-exporter_{context}"]
= File Exporter

[role="_abstract"]
The File Exporter writes telemetry data to files in persistent storage and supports file operations such as rotation, compression, and writing to multiple files. With this exporter, you can also use a resource attribute to control file naming. The only required setting is `path`, which specifies the destination path for telemetry files in the persistent-volume file system.

:FeatureName: The File Exporter
include::snippets/technology-preview.adoc[]

.OpenTelemetry Collector custom resource with the enabled File Exporter
[source,yaml]
----
# ...
config: |
exporters:
file:
path: /data/metrics.json # <1>
rotation: # <2>
max_megabytes: 10 # <3>
max_days: 3 # <4>
max_backups: 3 # <5>
localtime: true # <6>
format: proto # <7>
compression: zstd # <8>
flush_interval: 5 # <9>
# ...
----
<1> The file-system path where the data is to be written. There is no default.
<2> File rotation is an optional feature of this exporter. By default, telemetry data is exported to a single file. Add the `rotation` setting to enable file rotation.
<3> The `max_megabytes` setting is the maximum size a file is allowed to reach until it is rotated. The default is `100`.
<4> The `max_days` setting is for how many days a file is to be retained, counting from the timestamp in the file name. There is no default.
<5> The `max_backups` setting is for retaining several older files. The default is `100`.
<6> The `localtime` setting specifies the local-time format for the timestamp, which is appended to the file name in front of any extension, when the file is rotated. The default is the Coordinated Universal Time (UTC).
<7> The format for encoding the telemetry data before writing it to a file. The default format is `json`. The `proto` format is also supported.
<8> File compression is optional and not set by default. This setting defines the compression algorithm for the data that is exported to a file. Currently, only the `zstd` compression algorithm is supported. There is no default.
<9> The time interval between flushes. A value without a unit is set in nanoseconds. This setting is ignored when file rotation is enabled through the `rotation` settings.
44 changes: 44 additions & 0 deletions modules/otel-exporters-kafka-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-kafka-exporter_{context}"]
= Kafka Exporter

[role="_abstract"]
The Kafka Exporter exports logs, metrics, and traces to Kafka. This exporter uses a synchronous producer that blocks and does not batch messages. You must use it with batch and queued retry processors for higher throughput and resiliency.

.OpenTelemetry Collector custom resource with the enabled Kafka Exporter
[source,yaml]
----
# ...
config:
exporters:
kafka:
brokers: ["localhost:9092"] # <1>
protocol_version: 2.0.0 # <2>
topic: otlp_spans # <3>
auth:
plain_text: # <4>
username: example
password: example
tls: # <5>
ca_file: ca.pem
cert_file: cert.pem
key_file: key.pem
insecure: false # <6>
server_name_override: kafka.example.corp # <7>
service:
pipelines:
traces:
exporters: [kafka]
# ...
----
<1> The list of Kafka brokers. The default is `+localhost:9092+`.
<2> The Kafka protocol version. For example, `+2.0.0+`. This is a required field.
<3> The name of the Kafka topic to read from. The following are the defaults: `+otlp_spans+` for traces, `+otlp_metrics+` for metrics, `+otlp_logs+` for logs.
<4> The plain text authentication configuration. If omitted, plain text authentication is disabled.
<5> The client-side TLS configuration. Defines paths to the TLS certificates. If omitted, TLS authentication is disabled.
<6> Disables verifying the server's certificate chain and hostname. The default is `+false+`.
<7> ServerName indicates the name of the server requested by the client to support virtual hosting.
45 changes: 45 additions & 0 deletions modules/otel-exporters-load-balancing-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-load-balancing-exporter_{context}"]
= Load Balancing Exporter

[role="_abstract"]
The Load Balancing Exporter consistently exports spans, metrics, and logs according to the `routing_key` configuration.

:FeatureName: The Load Balancing Exporter
include::snippets/technology-preview.adoc[leveloffset=+1]

.OpenTelemetry Collector custom resource with the enabled Load Balancing Exporter
[source,yaml]
----
# ...
config:
exporters:
loadbalancing:
routing_key: "service" # <1>
protocol:
otlp: # <2>
timeout: 1s
resolver: # <3>
static: # <4>
hostnames:
- backend-1:4317
- backend-2:4317
dns: # <5>
hostname: otelcol-headless.observability.svc.cluster.local
k8s: # <6>
service: lb-svc.kube-public
ports:
- 15317
- 16317
# ...
----
<1> The `routing_key: service` exports spans for the same service name to the same Collector instance to provide accurate aggregation. The `routing_key: traceID` exports spans based on their `traceID`. The implicit default is `traceID` based routing.
<2> The OTLP is the only supported load balancing protocol. All options of the OTLP exporter are supported.
<3> You can configure only one resolver.
<4> The static resolver distributes the load across the listed endpoints.
<5> You can use the DNS resolver only with a Kubernetes headless service.
<6> The Kubernetes resolver is recommended.
44 changes: 44 additions & 0 deletions modules/otel-exporters-otlp-exporter.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-collector/otel-collector-exporters.adoc

:_mod-docs-content-type: REFERENCE
[id="otel-exporters-otlp-exporter_{context}"]
= OTLP Exporter

[role="_abstract"]
The OTLP gRPC Exporter exports traces and metrics by using the OpenTelemetry protocol (OTLP).

.OpenTelemetry Collector custom resource with the enabled OTLP Exporter
[source,yaml]
----
# ...
config:
exporters:
otlp:
endpoint: tempo-ingester:4317 # <1>
tls: # <2>
ca_file: ca.pem
cert_file: cert.pem
key_file: key.pem
insecure: false # <3>
insecure_skip_verify: false # # <4>
reload_interval: 1h # <5>
server_name_override: <name> # <6>
headers: # <7>
X-Scope-OrgID: "dev"
service:
pipelines:
traces:
exporters: [otlp]
metrics:
exporters: [otlp]
# ...
----
<1> The OTLP gRPC endpoint. If the `+https://+` scheme is used, then client transport security is enabled and overrides the `insecure` setting in the `tls`.
<2> The client-side TLS configuration. Defines paths to TLS certificates.
<3> Disables client transport security when set to `true`. The default value is `false` by default.
<4> Skips verifying the certificate when set to `true`. The default value is `false`.
<5> Specifies the time interval at which the certificate is reloaded. If the value is not set, the certificate is never reloaded. The `reload_interval` accepts a string containing valid units of time such as `ns`, `us` or `µs`, `ms`, `s`, `m`, `h`.
<6> Overrides the virtual hostname of authority such as the authority header field in requests. You can use this for testing.
<7> Headers are sent for every request performed during an established connection.
Loading