DSXchange

Hi everyone,
I received a disk usage alert from my service tier server this morning. Upon further review of the disk I am seeing that the IBM\InformationServer\shared-open-source\kafka\logs directory is 12GB. It does not appear that any of the logs are bloated per se, just a steady accumulation of log files. Does anyone know how to address this? The logs go back for a couple years, I assume there must be a log retention setting that can be configured?

Two ways come to my mind:

1) change the retention period for events on the topic
$InstallPath/shared-open-source/kafka$ update-config.sh --zookeeper serverName:port --entity-type topics --alter --add-config retention-ms=500 --entity-name InfosphereEvents

2) delete the topic. It will be automatically recreated

Thanks for the reply! I have been reading a bit about altering the log retention settings. I see you're calling the update-config script and passing in new retention attributes from there. Is this different than editing the server.properties file itself in $InstallPath\shared-open-source\kafka\conf ?

I'm on a windows server unfortunately. I'm also not entirely sure I understand what a topic is in this context.

It should be ok to add the new parameter to the configuration manually. My preference is to use the commands provided to do all the maintenance tasks

A Topic is just a stream of records, In the Kafka or streaming world is called topic. For Information Server the default name of the topic is InfosphereEvents

Official word thus far from IBM is to delete the log files. I did that back to January of this year for the kafka-server log files only. I am awaiting a response from them on proper configuration of server log file retention as it was indicated that those logs are not affected by the config file.

Official word back from IBM was that log retention configuration for kafka-server.log files cannot be done via any log files. These need to be manually cleaned or scripted to maintain a limited retention. I created a scheduled task that runs the following command: Replace 'D:\IBM\InformationServer\shared-open-source\kafka\logs' with your IBM InformationServer directory for your logs.