Azure Event Grid and Azure Event Hubs. Let’s discuss what they are, how they can help us, and when can we use them.
Azure Event Grid and Azure Event Hubs are services that help us build applications that respect the event-based architecture. Event-based architecture gives us the ability to build applications that communicate with different systems by sharing information through events.
AZURE EVENT GRID
Event Grid can distribute events from different sources like Azure Blob Storage or Azure Media Services to different handlers like Azure Function (Event Grid trigger function) or Webhook.
- Events → What happened
The Event is a lightweight notification of a condition or a state change, which means that it doesn’t contain the entire object that was changed. The sender of the event is known as publisher and the receiver is known as subscriber.
- Event Sources → Where the event took place
The Event source (publisher) is responsible for sending events to the Event Grid. Each event source is related to one or more event types (for example, Azure Storage is the event source for blob created events).
- Topics → The endpoint where publishers send events
The Event Topic categorizes the events into groups. The Event source sends events to the topics, using a public endpoint.
There are 2 types of topics:
- System topics are built-in topics provided by Azure services. We don’t see system topics in our Azure subscription because the publisher owns the topics, but we can subscribe to them.
- Custom topics are application and third-party topics. We can see custom topics in our subscription if we create or we are assigned access to them.
- Event subscriptions → The endpoint or built-in mechanism to route events, sometimes to multiple handlers
Event subscription defines which events on a topic an event handler wants to receive. Subscription can also filter events based on their type or subject, so we can ensure that an event handler receives only relevant events.
- Event handlers → The app or service reacting to the event
Event handler (subscriber) is any component that can receive events from Event Grid.
Now we know what Event Grid is and how it can help us. Let’s see when we can use it.
For example, we have a blob storage and we want to execute an azure function each time a file is uploaded.
We can do this easily with Azure function:
Azure function (Event Grid Trigger)
- Create an Event Grid Trigger function.
- Go to the storage account → events and add an event subscription.
- At the endpoint type, we need to select Azure Function and at the endpoint field to select the event grid trigger function that we created earlier.
- That’s it. Once we upload a file to the storage, the event trigger function will be executed.
On the other hand, if we want to publish events to an Event Grid Topic, in order to accomplish this we can use three different approaches:
- Event Grid SDK
- Sending an HTTP POST with an authentication header
- Durable Functions
AZURE EVENT HUBS
Event Hubs are an intermediary for the publish-subscribe communication pattern. Unlike Event Grid, it is a service for processing huge amounts of events (millions of events per second) with low latency. We should consider the Event Hubs as the starting point in an event processing pipeline. Furthermore, we can use the Event Hubs as the event source of the Event Grid service.
As we can see in the image above, an Event Hub contains multiple partitions. Let’s explain what these partitions are, and why an Event Hub contains them. Event Hub receives data and it divides it into partitions. Partitions are buffers into which the data is saved. Because of these buffers, an event isn’t missed just because a subscriber is busy or even offline. The subscriber can always use this buffer to get the events. By default, events stay in the buffer for 24 hours before they automatically expire.
These buffers are called partitions because the data is divided amongst them. Every event hub has at least two partitions, and each partition has a separate set of subscribers.
One big advantage of Event Hubs is that it can provide a Kafka endpoint that can be used by our existing Kafka with a small configuration change. The big difference between Kafka and Event Hubs is that Event Hubs is a cloud service. There is no need to manage servers or networks.
Another advantage is that it can be used for logging and telemetry. Moreover, it can be integrated with the serverless real-time analytics, Stream Analytics and the business analytics service, Power BI.
Based on the advantages above, we can also understand Event Hubs usage.
Differences between Event Grid and Event Hubs
The noticeable difference between them is that Event Hubs are accepting only endpoints for the ingestion of data and they don’t provide a mechanism for sending data back to publishers. On the other hand, Event Grid sends HTTP requests to notify events that happen in publishers.
Event Grid can trigger an Azure Function. In the case of Event Hubs, the Azure Function needs to pull and process an event.
Another difference is durability. Event Grid is a distribution system, not a queueing mechanism. If an event is pushed in, it gets pushed out immediately and if it doesn’t get handled, it’s gone forever. Unless we send the undelivered events to a storage account. This process is known as dead-lettering. By default this option is disabled, so if we need to enable it, we have to specify a storage account at the creation of the Event Grid.
In Event Hubs, publishers and subscribers read and write from durable storage. The data can be kept in the Event Hubs for up to seven days and then replayed. This gives us the ability to resume from a certain point or to restart from an older point in time and reprocess events when we need it.
Moreover, Event Grid doesn’t guarantee the order of the events. Contrarily, Event Hubs use partitions as we discussed earlier, and these partitions are ordered sequences, so it can maintain the order of the events in the same partition.
Azure Event Hubs is a more suitable solution when we need a service that can receive and process millions of events per second and provide low-latency event processing. It can handle data from concurrent sources and route it to a variety of stream-processing infrastructure and analytics services, as I have already mentioned. Azure Event Hubs are used more for telemetry scenarios.
On the other hand, Azure Event Grid is ideal for reactive scenarios, like when an item has been shipped or an item has been added or updated on storage. We have to take into account also its native integrations with Functions, Logic Apps and Webhooks. Moreover, Event Grid is cheaper than Event Hubs and more suitable when we don’t have to deal with big data.