Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CLI Tools you need to know for your everyday Ka...

CLI Tools you need to know for your everyday Kafka | Current 2024

In Hugo Guerrero's presentation at Current, he delved into the essential **CLI tools** for managing **Apache Kafka** environments, perfect for users who prefer the terminal over graphical interfaces. He covered everything from setting up clusters to managing **Kafka Connect connectors** and users. The session introduced key scripts from the **Apache Kafka** distribution and useful tools like **kcat** for Kafka, **kcctl** for Kafka Connect, and even resources for **Kubernetes** users like **kubectl** and **strimzi-kafka-cli**. This presentation equips developers with practical commands and tools to boost productivity in Kafka environments.

Hugo Guerrero

September 26, 2024
Tweet

More Decks by Hugo Guerrero

Other Decks in Programming

Transcript

  1. You Need To Know For Your Everyday Kafka Command Line

    Tools Hugo Guerrero | @hguerreroo Sr. Principal Developer Advocate, Red Hat 1
  2. Hugo Guerrero Sr. Principal Developer Advocate @Red Hat Likes: Travel

    + Food + History Mexican @ Massachusetts, USA AsyncAPI Ambassador @hguerreroo Articles: https://developers.redhat.com/author/hugo-guerrero {“Hello”:”World!”}
  3. COMMAND LINE TOOLS YOU NEED TO KNOW 5 @hguerrero Why

    Use Docker CLI? • Simplifies the management of containerized applications and services. • Ideal for developers building, testing, and deploying applications in containers. • Supports complex workflows with volumes, networks, and multi-container setups. • Perfect for CI/CD, DevOps pipelines, and local development environments. Key Features: • Build and run containers: Build images from Dockerfiles and run containers from images. • Manage images: Pull, push, and list Docker images from local and remote repositories. • Container operations: Start, stop, remove, and inspect running containers. • Networking: Create and manage container networks. • Volumes: Persist data by managing Docker volumes across containers. docker Container Management Tool
  4. COMMAND LINE TOOLS YOU NEED TO KNOW 6 @hguerrero Why

    use kcat? • Lightweight and powerful Kafka tool. • Great for developers, testers, and operators. • Simple interface for testing Kafka APIs. • Enables rapid debugging and local development. Key Features: • Produce messages: Send messages to Kafka topics. • Consume messages: Read messages from Kafka topics. • Query metadata: Inspect Kafka topics, brokers, and partitions. • Low-level debug: Helpful for troubleshooting Kafka clusters. kcat (formerly known as kafkacat) The Versatile Kafka CLI Tool
  5. COMMAND LINE TOOLS YOU NEED TO KNOW 7 @hguerrero Why

    use kaf? • Comprehensive CLI for Kafka cluster management. • Combines Kafka topic, broker, and consumer group management in a single tool. • Fast and efficient – no need for multiple tools or interfaces. • Perfect for development, testing, and production environments. kaf Kafka CLI inspired by kubectl & docker Key Features: • Manage topics: Create, list, describe, and delete topics. • Produce and consume messages: Send and read messages to/from Kafka topics. • Monitor consumer groups: Check consumer group lags, offsets, and more. • View broker metadata: Inspect brokers, partitions, and replication details.
  6. COMMAND LINE TOOLS YOU NEED TO KNOW 8 @hguerrero Why

    use kcli? • A simple, easy-to-use Kafka management tool. • Fast and efficient for basic Kafka operations. • Ideal for developers and operators who need quick access to Kafka functionality. • Minimal setup, making it perfect for local development and testing. kcli Kafka Command-Line Interface Key Features: • Manage topics: Create, list, delete, and describe topics. • Inspect partitions: View partition details, including leader and replicas. • Monitor consumer offsets: Track consumer offsets and lags. • Query broker metadata: Get broker information and cluster status.
  7. COMMAND LINE TOOLS YOU NEED TO KNOW 9 @hguerrero Why

    use kafkactl? • A versatile tool for managing Kafka clusters via the REST Proxy. • Simplifies production and consumption of messages. • Ideal for users looking to perform CRUD operations and monitor Kafka resources efficiently. • Perfect for automation, CI/CD, and scripting use cases. kafkactl Kafka Command-Line Management Tool Key Features: • Manage topics: Create, delete, describe, and list Kafka topics. • Produce and consume messages: Send and read messages from Kafka topics. • Monitor consumer groups: List, describe, and check consumer group offsets. • Query Kafka resources: Inspect brokers, partitions, and configurations.
  8. COMMAND LINE TOOLS YOU NEED TO KNOW 10 @hguerrero Why

    use kcctl? • Makes Kafka Connect cluster management simple and efficient. • No need for REST API calls – everything is done via CLI. • Streamlines development and operational workflows. • Ideal for managing multiple connectors and environments. 🧸 kcctl Command-Line Tool for Kafka Connect Key Features: • Register connectors: Easily register new Kafka Connect connectors. • Manage connectors: Pause, resume, or delete connectors with ease. • Monitor status: Check the status of connectors and tasks. • Update configurations: Update connector configurations from the terminal.
  9. COMMAND LINE TOOLS YOU NEED TO KNOW 11 @hguerrero Why

    use kubectl? • The most comprehensive CLI for interacting with Kubernetes clusters. • Allows for full control over Kubernetes resources. • Supports automation and integration with CI/CD pipelines. • Ideal for developers, system administrators, and DevOps engineers managing production or development clusters. kubectl Kubernetes Command-Line Tool Key Features: • Manage resources: Create, update, and delete Kubernetes resources (e.g., pods, deployments, services). • Inspect cluster state: Get detailed information on pods, nodes, and workloads. • Port-forwarding: Forward local ports to services running inside the cluster. • Logs and events: View logs and monitor events for troubleshooting. • Apply configurations: Use declarative configurations with kubectl apply for Kubernetes resources.
  10. COMMAND LINE TOOLS YOU NEED TO KNOW 12 @hguerrero Why

    Use Strimzi CLI? • Designed for managing Kafka on Kubernetes with Strimzi’s custom resource definitions (CRDs). • Simplifies the complexity of operating Kafka clusters in a cloud-native environment. • Ideal for automating Kafka operations in Kubernetes. • Perfect for DevOps teams managing Kafka in Kubernetes for scaling, resilience, and ease of deployment. Strimzi CLI Kafka Management on Kubernetes Key Features: • Manage Kafka clusters: Create, update, and delete Kafka clusters. • Topic operations: Create, list, and delete Kafka topics. • User management: Manage Kafka users and their permissions. • Cluster status: Check the health and status of Kafka clusters. • Configurable context: Easily switch between multiple Kubernetes clusters or namespaces.