Moving nifi dataflows through environments

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Moving nifi dataflows through environments

BD International
Hello,

I need some advice about best practise when moving dataflows through our environments. 

We have three dev, test and production. We create templates for the dataflows and promote them through the environments. 

Using the custom variable registry we solved the issue of different url and envrionment specific values. 

The issue im having is to do with the ssl context and httpcontextmaps. When we import a template with an ssl context it creates a duplicate entry which then needs manual cleanup.

I was wondering what others have done to solve this issue?

Thanks

Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Moving nifi dataflows through environments

Joe Witt
Brian,

There have been issues in the past with regard to template importing
and it causing duplicate controller service entries.  But these should
be much improved.  What version are you using?

Also, the registry component that is now being worked on should help
this quite a bit.  The configuration management of flows ideas can be
found here https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows

Thanks

On Wed, Aug 9, 2017 at 11:39 AM, BD International
<[hidden email]> wrote:

> Hello,
>
> I need some advice about best practise when moving dataflows through our
> environments.
>
> We have three dev, test and production. We create templates for the
> dataflows and promote them through the environments.
>
> Using the custom variable registry we solved the issue of different url and
> envrionment specific values.
>
> The issue im having is to do with the ssl context and httpcontextmaps. When
> we import a template with an ssl context it creates a duplicate entry which
> then needs manual cleanup.
>
> I was wondering what others have done to solve this issue?
>
> Thanks
>
> Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Moving nifi dataflows through environments

BD International
Joe

Thanks for the documentation, i was running with 0.7.4 but im just de-risking nifi 1.3.0 and it seems to have solved all my dataflow environment issues, which is outstanding.

With a combination of custom properties and processor level templates ive been able to automate 99%. The only part ive had to leave as a manual step is the configuration of the remote processor group as the list of remote urls changes between environments and it can not be parameterised. 

Thanks

Brian

On 11 Aug 2017 2:59 am, "Joe Witt" <[hidden email]> wrote:
Brian,

There have been issues in the past with regard to template importing
and it causing duplicate controller service entries.  But these should
be much improved.  What version are you using?

Also, the registry component that is now being worked on should help
this quite a bit.  The configuration management of flows ideas can be
found here https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows

Thanks

On Wed, Aug 9, 2017 at 11:39 AM, BD International
<[hidden email]> wrote:
> Hello,
>
> I need some advice about best practise when moving dataflows through our
> environments.
>
> We have three dev, test and production. We create templates for the
> dataflows and promote them through the environments.
>
> Using the custom variable registry we solved the issue of different url and
> envrionment specific values.
>
> The issue im having is to do with the ssl context and httpcontextmaps. When
> we import a template with an ssl context it creates a duplicate entry which
> then needs manual cleanup.
>
> I was wondering what others have done to solve this issue?
>
> Thanks
>
> Brian
Loading...