Creating parameters from exported template?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Creating parameters from exported template?

James
Hi

For the life of me, I can't find anything related to exporting templates to new systems/environments along with the parameters.

I don't need the parameter values necessarily, but, importing a new template and then needing to create the parameters manually while setting them seems a little tedious. Is this only possible using a registry perhaps?

Any guidance would be helpful.

thanks
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

Etienne Jouvin
Hi.

In this case why don't you use variables ?

Etienne


Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
Hi

For the life of me, I can't find anything related to exporting templates to new systems/environments along with the parameters.

I don't need the parameter values necessarily, but, importing a new template and then needing to create the parameters manually while setting them seems a little tedious. Is this only possible using a registry perhaps?

Any guidance would be helpful.

thanks
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

James
Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:

> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

Etienne Jouvin
Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :
Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

John McGinn

Etienne,

I'm in the same boat as you. Currently using 1.9.2, and haven't upgraded yet, so only know of variables. I'm also concerned about migrating processor groups, as templates, from one environment to another, which cannot be done via NiFi Registry due to different networks/companies/firewalls. Currently, the variables are included as the part of the template, but unsure how the parameter context will be involved in the template. As well, those variables/parameters are now global, and not self-contained in a processor group or template, so any processor group can use any parameter context on a whim, rather than having to create/define it locally. I'm trying to plan out time for investigating the newer versions, and how parameter contexts will need to be used going forward.

John
On Tuesday, May 19, 2020, 08:01:58 AM EDT, Etienne Jouvin <[hidden email]> wrote:


Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :
Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:

> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

Bryan Bende
In reply to this post by Etienne Jouvin
The best option for moving flows across environments would be versioned flows in Nifi registry, which have full support for parameters. 

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <[hidden email]> wrote:
Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :
Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

Bryan Bende
John,

You can have a nifi registry on each network and use Nifi CLI to export and import the flows between registries. There is one limitation in that nested versioning doesn’t currently work when doing that.

Also, the parameter contexts do live outside the flow at the controller level, however in order to use one of them, the use needs read access to the context and write access to the process group. So you can setup fine grained policies on each context to restrict who can use it.

Thanks,

Bryan

On Tue, May 19, 2020 at 8:26 AM Bryan Bende <[hidden email]> wrote:
The best option for moving flows across environments would be versioned flows in Nifi registry, which have full support for parameters. 

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <[hidden email]> wrote:
Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :
Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
--
Sent from Gmail Mobile
--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

RE: Creating parameters from exported template?

nathan.english

Hi Bryan,

 

Would the same also work by using two separate git persistence providers pointed at two different git repositories? We have a remote production instance with no remote access (effectively airgapped).

 

Our plan is when we deploy to production, provide a full exported copy of the git repository, and it’s history to a locally running instance of a git server in production. Which we will then configure the production NiFi Registry to use that export of the git repository for read-only operations.

 

So two separate Nifi Registries with two independent git repo’s, the production one as a cloned export of the Dev/Test Git Repo.

 

We are planning on testing this within the next few weeks.

 

Kind Regards,

 

Nathan

 

From: Bryan Bende [mailto:[hidden email]]
Sent: 19 May 2020 15:36
To: [hidden email]
Subject: Re: Creating parameters from exported template?

 

John,

 

You can have a nifi registry on each network and use Nifi CLI to export and import the flows between registries. There is one limitation in that nested versioning doesn’t currently work when doing that.

 

Also, the parameter contexts do live outside the flow at the controller level, however in order to use one of them, the use needs read access to the context and write access to the process group. So you can setup fine grained policies on each context to restrict who can use it.

 

Thanks,

 

Bryan

 

On Tue, May 19, 2020 at 8:26 AM Bryan Bende <[hidden email]> wrote:

The best option for moving flows across environments would be versioned flows in Nifi registry, which have full support for parameters. 

 

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <[hidden email]> wrote:

Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

 

But I understand like you

 

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :

Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>

--

Sent from Gmail Mobile

--

Sent from Gmail Mobile

Reply | Threaded
Open this post in threaded view
|

Re: Creating parameters from exported template?

Bryan Bende
Hi Nathan,

Right now the git repo behind registry is mostly write-only, meaning if you add something to it outside of going through the given NiFi Registry instance, it won't be visible.

The only exception to this is when starting a brand new registry that has an empty metadata database. So if you ensure the DB is new/wiped every time that you export the git repo to production, then when you start registry the first time it will rebuild the database from the git repo and it should work.

Thanks,

Bryan

On Tue, May 19, 2020 at 8:46 AM <[hidden email]> wrote:

Hi Bryan,

 

Would the same also work by using two separate git persistence providers pointed at two different git repositories? We have a remote production instance with no remote access (effectively airgapped).

 

Our plan is when we deploy to production, provide a full exported copy of the git repository, and it’s history to a locally running instance of a git server in production. Which we will then configure the production NiFi Registry to use that export of the git repository for read-only operations.

 

So two separate Nifi Registries with two independent git repo’s, the production one as a cloned export of the Dev/Test Git Repo.

 

We are planning on testing this within the next few weeks.

 

Kind Regards,

 

Nathan

 

From: Bryan Bende [mailto:[hidden email]]
Sent: 19 May 2020 15:36
To: [hidden email]
Subject: Re: Creating parameters from exported template?

 

John,

 

You can have a nifi registry on each network and use Nifi CLI to export and import the flows between registries. There is one limitation in that nested versioning doesn’t currently work when doing that.

 

Also, the parameter contexts do live outside the flow at the controller level, however in order to use one of them, the use needs read access to the context and write access to the process group. So you can setup fine grained policies on each context to restrict who can use it.

 

Thanks,

 

Bryan

 

On Tue, May 19, 2020 at 8:26 AM Bryan Bende <[hidden email]> wrote:

The best option for moving flows across environments would be versioned flows in Nifi registry, which have full support for parameters. 

 

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <[hidden email]> wrote:

Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.

 

But I understand like you

 

Le mar. 19 mai 2020 à 13:56, James <[hidden email]> a écrit :

Hi

Thanks for the response.

According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin <[hidden email]> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <[hidden email]> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>

--

Sent from Gmail Mobile

--

Sent from Gmail Mobile