Apache Kafka is a distributed messaging system that is widely used for real-time data processing and stream processing. In this tutorial, we'll introduce you to the basics of Apache Kafka and show you

Apache Kafka Tutorial: An Introduction to Distributed Messaging Systems

Table Of Contents

Introduction to Apache Kafka Kafka Architecture Overview Setting up Kafka: Installation and Configuration Creating Topics in Kafka Producers and Consumers in Kafka Kafka Connect: Integrating External Systems Kafka Streams: Processing Real-Time Data Kafka Security: Authentication and Authorization Kafka Monitoring and Metrics Best Practices for Kafka Deployment and Operations Kafka Case Studies and Real-World Use Cases Advanced Kafka Concepts: Transactions, Exactly-Once Semantics, and more

Apache Kafka Tutorial: An Introduction to Distributed Messaging Systems

15 0 0 0 18

+10 pts

Apache Kafka Tutorial: An Introduction to Distributed Messaging Systems

Creating Topics in Kafka

Kafka is a distributed streaming platform that allows you to publish and subscribe to streams of records. A record is a key-value pair that can contain any data. A stream is a sequence of records that are produced and consumed by applications.

One of the core concepts in Kafka is a topic. A topic is a logical name for a stream of records. You can create topics in Kafka using the command-line tool kafka-topics.sh or the Admin API.

To create a topic using the command-line tool, you need to specify the name of the topic, the number of partitions, and the replication factor. For example, to create a topic named "test" with 3 partitions and 2 replicas, you can run:

kafka-topics.sh --create --topic test --partitions 3 --replication-factor 2 --bootstrap-server localhost:9092

To create a topic using the Admin API, you need to use the AdminClient class and call the createTopics method with a collection of NewTopic objects. For example, to create the same topic as above in Java, you can write:

AdminClient admin = AdminClient.create(props);
NewTopic newTopic = new NewTopic("test", 3, (short) 2);
admin.createTopics(Collections.singleton(newTopic));

Conclusion

Creating topics in Kafka is easy and flexible. You can use either the command-line tool or the Admin API to create topics with different configurations. Topics are essential for producing and consuming data streams in Kafka.

FAQs

Q: How do I list all the topics in Kafka?

A: You can use the command-line tool kafka-topics.sh with the --list option or call the listTopics method of the AdminClient class.

Q: How do I delete a topic in Kafka?

A: You can use the command-line tool kafka-topics.sh with the --delete option or call the deleteTopics method of the AdminClient class. Note that deleting a topic may not be immediate and may depend on some broker configurations.

Q: How do I change the configuration of a topic in Kafka?

A: You can use the command-line tool kafka-configs.sh with the --alter option or call the alterConfigs method of the AdminClient class. You can change various parameters such as retention time, compression type, segment size,etc.

Previous Next