Custom data on Webhooks
So, you’ve configured your VOD encoding, started it, and then… then it’s time to wait. The encoder is doing its thing and until it’s done, there isn’t much you can do with that content.
When you initially prototype, that’s not an issue. There’s a great time to get a coffee 
. If you want to keep an eye on the progress whilst you sip your favourite brew, you can just look in the dashboard. Or, if you’ve used one of numerous public SDK examples, you’ll likely be using the APIs to poll its status.
From polling to webhooks
However one day your project will grow up, will go to production, and that way ain’t gonna fly anymore, chief! Particularly if you have dozens of encodings running in parallel, and/or you’re working in some form of serverless fashion (say, with an AWS Lambda, or Google Cloud function or Azure Function). You just can’t keep your function running to just poll the status, it’s not efficient.
Enter webhooks, or programmatic notifications (or callbacks, or whatever you fancy calling them). They allow you to start the encoding and then leave and just “forget” about it. When the encoding finishes (or fails, which obviously never happens…), the Bitmovin all-knowing platform will then send a notification to a URL of your choice to tell you (or rather the system listening on the other end of that line) that the job is done.
You can easily set webhooks up in the dashboard for example, or through the APIs (with separate endpoints for notifications of completion or error). And just like that, notifications will automatically get fired for every encoding in that account!
The loss of contextual information
But using webhooks now gives you a new problem (of sorts). Typically when an encoding completes, you’ll want to perform further steps downstream in your workflow with the encoded content. And that part of the process is maybe going to require some information that was defined in the encoding itself.
Say for example that your downstream component in the workflow needs to know where the encoding’s output manifests were written to, in order to perform some publishing task and update a CMS with those manifests URLs.
When you were gently polling your encoding after configuring it and starting it, it wasn’t an issue, since you were still working within that overall context. But with a webhook, the context is gone. The payload that you get on the receiving end of the webhook will be - by design - sparse, only containing the encoding ID and its name. It will look something like this:
{
"id": "f28b9255-6937-4772-b9c2-4edd52cd07dc",
"eventId": "3b5d14ac-4e04-4d78-a8ee-2822e462ac25",
"orgId": "bc1dcb27-6fc8-4a93-bc0b-66b25f62710f",
"userId": "3dd56220-4bae-4619-91ce-012dc83fe5da",
"resourceType": "ENCODING",
"type": "WEBHOOK",
"eventType": "ENCODING_FINISHED",
"state": "FIRED",
"triggeredAt": "2022-07-26T09:45:16.520280385Z",
"encoding": {
"id": "d69c812b-6b27-45c3-8b97-176af0c08037",
"type": "VOD",
"name": "My beautiful encoding's name"
},
"webhookId": "c98965b8-0bc3-4c1a-ac5e-e66b410699f0"
}
It’s really not the end of the world, since armed with that encoding ID, you can retrieve any information about the configuration of that encoding through the APIs again (effectively making GET calls where previously you used POST calls to configure it). But that by itself requires having the relevant Bitmovin access credentials (API key and Org ID), and you might prefer not to have multiple components in your workflow in need of those (separation of concerns and all that…)
Naturally, there are plenty of possible solutions to this, such as calculating all the parameters in advance (instead of doing it in the encoding configuration script), and recording them in a DB, alongside the encoding ID, and then doing a lookup on receiving the notification. Or (please don’t do that… 
) hard-coding some of that decisioning logic in both the encoding and post-encoding components, etc etc.
A sprinkle of custom data
However in the remainder of this post, I’ll suggest another method that might just be more expedient, in particular for the simpler use cases. And there are 2 features that make it possible it:
- With the Bitmovin API, you can create webhooks that are specific to an encoding, instead of being global catch-all ones that get triggered for all encodings. They can all use the same URL (and therefore point to the same listener).
- A webhook (like many other Bitmovin resources) can be decorated with custom data, which can be any set of key-value pairs.
And the cherry on the cake? That custom data (which can therefore be encoding-specific) will be passed into the webhook notification payload.
That’s it, it’s that simple, and yet it’s a sure way of passing important information to downstream systems.
In code
Let’s look at it with some code, with the example provided above. Let’s assume that the encoding and manifest configuration was something along the following lines (massively oversimplified for brevity)
encoding = Encoding(
name="My beautiful encoding's name")
encoding = api.encoding.encodings.create(encoding=encoding)
(...)
input_file_path = "/path/to/my/source-file.mkv"
input_stream = IngestInputStream(
input_id=input.id,
input_path=input_path)
input_stream = api.encoding.encodings.input_streams.ingest.create(
encoding_id=encoding.id,
ingest_input_stream=ingest_input_stream)
(...)
output_path = "/path/to/the/output/parent/folder"
manifest_name = "stream.mpd"
dash_manifest_default = DashManifestDefault(
encoding_id=encoding.id,
manifest_name=manifest_name,
outputs=[
EncodingOutput(
output_path=output_path,
output_id=output_id)
])
api.encoding.manifests.dash.default.create(dash_manifest_default)
(...)
bitmovin_api.encoding.encodings.start(
encoding_id=encoding.id,
start_encoding_request=StartEncodingRequest(
manifest_generator=ManifestGenerator.V2,
vod_dash_manifests=[ManifestResource(manifest_id=dash_manifest.id)]
))
I use the Python SDK here for no other reason than I like it and find it the most readable. The exact same thing can be done with any of our SDKs naturally.
Let’s modify this code now, so as to add a webhook for successful encoding, and decorate it with extra metadata I need in my downstream component. Just before the encoding start call, let’s configure our webhooks:
(actually you could even do that after starting the encoding, but I prefer to have all my rubber ducks in a row before I start running my bath…)
custom_data = {
"manifestUrl": output_path + manifest_name,
"inputFile": input_file_path,
"workflow": "test-workflow-1"
}
webhook = Webhook(
url="https://company.com/my-wehbook-listener",
method=WebhookHttpMethod.POST,
custom_data=custom_data
)
api.notifications.webhooks.encoding.encodings.finished.create_by_encoding_id(
encoding_id=encoding.id,
webhook=webhook)
The beauty of being able to define different webhooks for completed or errored encodings is that you only have to create the ones you need. Here (for the purpose of the example), we’ll just stick with an account-level webhook for errors, but nothing prevents you from creating encoding-specific ones as well with the same or different (or no) custom data.
As for my encoding? Well, it completed, and here’s what the notification payload now looks like:
{
"id": "7cf991d8-6e1c-416a-83f0-bda816a6c38e",
"eventId": "e825c2c4-1568-44d0-bf01-2feaee11a618",
"orgId": "bc1dcb27-6fc8-4a93-bc0b-66b25f62710f",
"userId": "3dd56220-4bae-4619-91ce-012dc83fe5da",
"resourceType": "ENCODING",
"type": "WEBHOOK",
"eventType": "ENCODING_FINISHED",
"state": "FIRED",
"triggeredAt": "2022-07-26T17:58:09.878389435Z",
"triggeredForResourceId": "d75e548c-444f-4509-95ca-0283ffb13256",
"customData": {
"inputFile": "/path/to/my/source-file.mkv",
"manifestUrl": "/path/to/the/output/parent/folder/stream.mpd",
"workflow": "test-workflow-1"
},
"encoding": {
"id": "d75e548c-444f-4509-95ca-0283ffb13256",
"type": "VOD",
"name": "My beautiful encoding's name"
},
"webhookId": "15662129-7f15-4b50-82f0-d3f83dda198d"
}
Limitations
Unfortunately, this is not magic. You’ll only be able to populate custom data with information that exists at the time of starting the encoding. For anything that would be determined at or after runtime (such as getting encoding performance information, or determining how many streams were skipped because of stream conditions, or actually generated in a Per-Title configuration), you will have to go back and use the APIs post-encoding to retrieve that information.
Was that useful? Can you think of a particular use case that would really benefit from this technique? Do you have an alternative idea? Let us know in the comments!