Consume Kafka record - Key and offset from kafka

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Consume Kafka record - Key and offset from kafka

KhajaAsmath Mohammed
Hi,

Is there a way to get key,timestamp and offset in consume kafka record? 

Offset: 380050   Key: urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63|+|2020-05-05   Timestamp: 2020-05-05 03:23:54.210 Headers: empty
 
{"ID":"urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63","KSQL_COL_1":2}
Reply | Threaded
Open this post in threaded view
|

Re: Consume Kafka record - Key and offset from kafka

Bryan Bende
Hello,

Not totally sure if this is what you are asking, but there isn't a way for them to be added to each record.

The key, timestamp, and offset are all added as flow file attributes, but the key is only added if you are doing one record per flow file since each record could have a different key, so there can't be a single key attribute for the whole flow file when there are N records.

Thanks,

Bryan


On Tue, May 5, 2020 at 12:15 PM KhajaAsmath Mohammed <[hidden email]> wrote:
Hi,

Is there a way to get key,timestamp and offset in consume kafka record? 

Offset: 380050   Key: urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63|+|2020-05-05   Timestamp: 2020-05-05 03:23:54.210 Headers: empty
 
{"ID":"urn:uuid:eb3b516b-9a31-4365-8ccb-e8c009073f63","KSQL_COL_1":2}