Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Now the following topic Reroute works perfectly for 1 table type:

transforms=Reroute
transforms.Reroute.type=io.debezium.transforms.ByLogicalTableRouter
transforms.Reroute.topic.regex=(.*)raw_app_logs(.*)
transforms.Reroute.topic.replacement=$1raw_app_logs
This takes raw_app_logs_20200720, raw_app_logs_20200721 etc and writes it into the raw_app_logs topic perfectly well.
How do I add the other 2 entries so that they are matched and sent to their topic names?
EG: raw_app_logs_20200720 goes to the topic raw_app_logs while raw_users_logs_20200720 goes to a topic named raw_users_logs and city_logs_20200720 goes to a topic named city_logs in the same config file.
Use regex groups and then just print the first group:
transforms=Reroute
transforms.Reroute.type=io.debezium.transforms.ByLogicalTableRouter
transforms.Reroute.topic.regex=(.*)([^0-9]).*
transforms.Reroute.topic.replacement=$1
                Hi @rezizter,  I am getting the following error when trying to push data from 2 different tables. The first table changes are flowing fine.  Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema being registered is incompatible with an earlier schema for subject  error code: 409
– Vikash
                Sep 2, 2021 at 8:05
                Hi @Vikash. Are your schemas different? in our case our schemas are the same (we write daily tables and i needed to stream each daily table which has the same schema)
– rezizter
                Sep 3, 2021 at 9:08
                Hi @rezizter, Thanks for your reply, Yes our schemas are different. Basically, we want the changes from two different tables into the same topic. Both the tables are having different sets of data but logically both are related. Like employee entity and address entity and account details entity etc.
– Vikash
                Sep 7, 2021 at 7:51
You can use a different transforms alias for each table :
transforms=app, users, city
transforms.app.type=io.debezium.transforms.ByLogicalTableRouter
transforms.app.topic.regex=(.*)raw_app_logs(.*)
transforms.app.topic.replacement=$1raw_app_logs
transforms.users.type=io.debezium.transforms.ByLogicalTableRouter
transforms.users.topic.regex=(.*)raw_users_logs(.*)
transforms.users.topic.replacement=$1raw_users_logs
transforms.city.type=io.debezium.transforms.ByLogicalTableRouter
transforms.city.topic.regex=(.*)city_logs(.*)
transforms.city.topic.replacement=$1city_logs
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.