Accessing flow attributes from ExecuteStreamCommand

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Accessing flow attributes from ExecuteStreamCommand

Jean-Sebastien Vachon

Hi all,

 

I am using the ExecuteStreamCommand processor to run a python script to crunch different data and I was curious

to know if such a processor could both read and/or write from/to the flow attributes.

 

Can someone point me to the documentation if this is possible? I could not find it by myself.

 

Thanks

 

Sent from Mail for Windows 10

 

Reply | Threaded
Open this post in threaded view
|

Re: Accessing flow attributes from ExecuteStreamCommand

Mike Thomsen
There's not way at the moment to interact with the NiFi API from that processor. The closest work around would be to pass in flowfile attributes as parameters using the parameter configuration field and expression language.

On Thu, May 28, 2020 at 10:28 AM Jean-Sebastien Vachon <[hidden email]> wrote:

Hi all,

 

I am using the ExecuteStreamCommand processor to run a python script to crunch different data and I was curious

to know if such a processor could both read and/or write from/to the flow attributes.

 

Can someone point me to the documentation if this is possible? I could not find it by myself.

 

Thanks

 

Sent from Mail for Windows 10

 

Reply | Threaded
Open this post in threaded view
|

Re: Accessing flow attributes from ExecuteStreamCommand

Andy LoPresto
I think Mike is referring to the “command arguments” & “command arguments strategy” processor properties when he says “parameters”, as this is different from NiFi parameters. 

If the Python script can run in Jython (2.x only, no native libs), you could put the script into an ExecuteScript or InvokeScriptedProcessor, in which case the NiFi session API is available to your code and you can interact directly with attributes. 


Andy LoPresto
[hidden email]
[hidden email]
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 28, 2020, at 7:57 AM, Mike Thomsen <[hidden email]> wrote:

There's not way at the moment to interact with the NiFi API from that processor. The closest work around would be to pass in flowfile attributes as parameters using the parameter configuration field and expression language.

On Thu, May 28, 2020 at 10:28 AM Jean-Sebastien Vachon <[hidden email]> wrote:

Hi all,

 

I am using the ExecuteStreamCommand processor to run a python script to crunch different data and I was curious

to know if such a processor could both read and/or write from/to the flow attributes.

 

Can someone point me to the documentation if this is possible? I could not find it by myself.

 

Thanks

 

Sent from Mail for Windows 10

 


Reply | Threaded
Open this post in threaded view
|

RE: Accessing flow attributes from ExecuteStreamCommand

Jean-Sebastien Vachon
In reply to this post by Mike Thomsen

Thanks Mike

 

That was my fallback in case it was not supported out of the box.

I guess another alternative would be to create a custom processor or use some other engine/cache such as Redis to store the data if it becomes too large.

 

Thanks

 

Sent from Mail for Windows 10

 

From: [hidden email]
Sent: May 28, 2020 10:57 AM
To: [hidden email]
Subject: Re: Accessing flow attributes from ExecuteStreamCommand

 

There's not way at the moment to interact with the NiFi API from that processor. The closest work around would be to pass in flowfile attributes as parameters using the parameter configuration field and expression language.

 

On Thu, May 28, 2020 at 10:28 AM Jean-Sebastien Vachon <[hidden email]> wrote:

Hi all,

 

I am using the ExecuteStreamCommand processor to run a python script to crunch different data and I was curious

to know if such a processor could both read and/or write from/to the flow attributes.

 

Can someone point me to the documentation if this is possible? I could not find it by myself.

 

Thanks

 

Sent from Mail for Windows 10