Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I am trying to do integration test for app that using kafka, kafka-streams and cassandra. But when I am trying to setUp test class, i've got 2 errors: ERROR [main] BrokerMetadataCheckpoint: Failed to read meta.properties file under dir ERROR [main] KafkaServer: Fail to read meta.properties under log directory

I am using spring-boot-starter 2.1.2, spring-boot-starter-test 2.1.2, spring-kafka 2.2.0, spring-kafka-test 2.2.0, apache.kafka-streams 2.1.0

trying to change logs.dir and logs.dirs params. use @EnableKafka @EnableKafkaStreams

@RunWith(SpringRunner.class)
@SpringBootTest
@EmbeddedKafka(partitions = 3, controlledShutdown = false, count = 1, topics = {"zc.deviceposition"}, brokerProperties = {"listeners=PLAINTEXT://localhost:9092", "port=9092", "log.dir=/home/name/logs"})
@EmbeddedCassandra(timeout = 60000)
@CassandraDataSet(value = {"bootstrap_test.cql"}, keyspace = "statistics")
@ActiveProfiles("test")
@DirtiesContext
public class CassandraTripsAggregatorProcessorSupplierIntegrationTest {
  @Test
  public void someTest() {System.out.println("hello world");}
I excpect to run up context with embedded kafka, but for now I receive an error that meta.properties is no exist
                I would suggest to strip down the problem a bit, as it does not seem to have anything to do with Cassandra or Kafka Streams. The core problem is that the Kafka server requires a meta.properties file, but Spring's EmbeddedKafkaBroker does not generate such a file for test purposes (at least as far as I could find out).
– Marek Walasek
                Apr 12, 2019 at 14:30
                creating in docker with predefined environment, but I think its the same like install or uninstall whole system. So it not a solution programmatic
– faceoff
                May 25, 2020 at 15:12
I checked the kafka source code to see what is really happening:
The error message is thrown in this class.
During startup of the kafka server the meta.properties file is read and created if it does not exist (see here).
So I think there is nothing that can be done about it since it is not actually an error, but probably the log level in the kafka code is not correct.
Since you're using an own log.dir in your broker properties, one solution is to create a meta.properties file under your log directory with some minimum config: 
version=0
broker.id=0
I would also recommend to place that file and directory somewhere in your project's test directory (i.e. not necessarily under /home/) and ensure that it is properly emptied after each test run.
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.