How do you recover a workflow ?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How do you recover a workflow ?

Alessio Palma-2

Hello all,

what is the best pratice to recover a workflow gone bad ?

Currently I use a generateFlowFile processor attached to some entry point, which allows me to restart something. Start then stop and a flowfile is created, but this is not the best option.

I really miss the option to put a flowfile using a mouse click. Also some way to display a basic interface where to insert/modify some values used into some updateAttribute process helps a lot.

What do you think ? 


AP


Reply | Threaded
Open this post in threaded view
|

Re: How do you recover a workflow ?

Jeff
Hello Alessio,

Could you provide some more details about your NiFi flow?

One of the triggers I used to manually be able to start processing in my flow was to have a GetFile processor (configured with "Keep Source File" set to false) watching for files in a directory, and when I wanted to test the flow, I would just run the touch command to create a file that the GetFile processor would detect and emit a flowfile for it.

Depending on your use case, there might be a better source processor for flowfiles that you can use in your flow.

On Thu, Nov 10, 2016 at 6:55 AM Alessio Palma <[hidden email]> wrote:

Hello all,

what is the best pratice to recover a workflow gone bad ?

Currently I use a generateFlowFile processor attached to some entry point, which allows me to restart something. Start then stop and a flowfile is created, but this is not the best option.

I really miss the option to put a flowfile using a mouse click. Also some way to display a basic interface where to insert/modify some values used into some updateAttribute process helps a lot.

What do you think ? 


AP


Reply | Threaded
Open this post in threaded view
|

Re: How do you recover a workflow ?

Alessio Palma-2

The point is that I have a workflow but sometimes things go wrong and I need to manually restart it; this action requires:

1) Change some parameters ( UpdateAttribute processor )

2) Fire a new flowfile which will start again the workflow.  Perhaps this is the most obscure point. We are using nifi to execute some old cron jobs and I'm using the GenerateFlowFile ( crontab scheduling strategy ) processor to start the flow.
When the workflow did not complete I use another GenerateFlowFile processor to fire a new flowfile which allows me to execute again the flow out of the schedule.

All these points can be executed faster if I can insert the value into some kind of form into the screen and can fire a new flowfile clicking some button instead to start/stop ad additional GenerateFlowFile processor.

Perhaps I'm doing it in the wrong way. So how do you restart a workflow ? Maybe this feature can help others in the same task.
Don't know... I'm just asking.





From: Jeff <[hidden email]>
Sent: Friday, November 11, 2016 2:36:02 AM
To: [hidden email]
Subject: Re: How do you recover a workflow ?
 
Hello Alessio,

Could you provide some more details about your NiFi flow?

One of the triggers I used to manually be able to start processing in my flow was to have a GetFile processor (configured with "Keep Source File" set to false) watching for files in a directory, and when I wanted to test the flow, I would just run the touch command to create a file that the GetFile processor would detect and emit a flowfile for it.

Depending on your use case, there might be a better source processor for flowfiles that you can use in your flow.

On Thu, Nov 10, 2016 at 6:55 AM Alessio Palma <[hidden email]> wrote:

Hello all,

what is the best pratice to recover a workflow gone bad ?

Currently I use a generateFlowFile processor attached to some entry point, which allows me to restart something. Start then stop and a flowfile is created, but this is not the best option.

I really miss the option to put a flowfile using a mouse click. Also some way to display a basic interface where to insert/modify some values used into some updateAttribute process helps a lot.

What do you think ? 


AP


Reply | Threaded
Open this post in threaded view
|

Re: How do you recover a workflow ?

Jeff
From your description, it sounds like your additional GenerateFlowFile processor to manually add a flowfile is the best way to do what you're trying to do.  However, you can submit a JIRA [1] if you have ideas on how your workflow could be made easier to perform.


On Fri, Nov 11, 2016 at 2:16 AM Alessio Palma <[hidden email]> wrote:

The point is that I have a workflow but sometimes things go wrong and I need to manually restart it; this action requires:

1) Change some parameters ( UpdateAttribute processor )

2) Fire a new flowfile which will start again the workflow.  Perhaps this is the most obscure point. We are using nifi to execute some old cron jobs and I'm using the GenerateFlowFile ( crontab scheduling strategy ) processor to start the flow.
When the workflow did not complete I use another GenerateFlowFile processor to fire a new flowfile which allows me to execute again the flow out of the schedule.

All these points can be executed faster if I can insert the value into some kind of form into the screen and can fire a new flowfile clicking some button instead to start/stop ad additional GenerateFlowFile processor.

Perhaps I'm doing it in the wrong way. So how do you restart a workflow ? Maybe this feature can help others in the same task.
Don't know... I'm just asking.





From: Jeff <[hidden email]>
Sent: Friday, November 11, 2016 2:36:02 AM
To: [hidden email]
Subject: Re: How do you recover a workflow ?
 
Hello Alessio,

Could you provide some more details about your NiFi flow?

One of the triggers I used to manually be able to start processing in my flow was to have a GetFile processor (configured with "Keep Source File" set to false) watching for files in a directory, and when I wanted to test the flow, I would just run the touch command to create a file that the GetFile processor would detect and emit a flowfile for it.

Depending on your use case, there might be a better source processor for flowfiles that you can use in your flow.

On Thu, Nov 10, 2016 at 6:55 AM Alessio Palma <[hidden email]> wrote:

Hello all,

what is the best pratice to recover a workflow gone bad ?

Currently I use a generateFlowFile processor attached to some entry point, which allows me to restart something. Start then stop and a flowfile is created, but this is not the best option.

I really miss the option to put a flowfile using a mouse click. Also some way to display a basic interface where to insert/modify some values used into some updateAttribute process helps a lot.

What do you think ? 


AP