td-agent throwing error - 2023-05-22 14:08:44 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="'columnify' utility must be in PATH for -h compression"

[DivyaYash]

2023-05-22 14:08:44 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="'columnify' utility must be in PATH for -h compression"

which columnify

/root/go/bin/columnify

columnify -h

Usage of columnify: columnify [-flags] [input files]
-output string
path to output file; default: stdout
-parquetCompressionCodec string
parquet compression codec, default: SNAPPY (default "SNAPPY")
-parquetPageSize int
parquet file page size, default: 8kB (default 8192)
-parquetRowGroupSize int
parquet file row group size, default: 128MB (default 134217728)
-recordType string
record data format type, [avro|csv|jsonl|ltsv|msgpack|tsv] (default "jsonl")
-schemaFile string
path to schema file
-schemaType string
schema type, [avro|bigquery]

[okkez]

Install Columnify on the PATH where the Fluentd process can be found. Alternatively, add the path where you installed Columnify to your Fluentd PATH.

[shubhdeep22]

@DivyaYash @okkez how did you resolved this issue ? How I can add package path to fluentd path

[okkez]

@shubhdeep22 Read the documents carefully, please.

• https://docs.treasuredata.com/display/public/PD/td-agent+Reference
• https://docs.fluentd.org/