NIFI JMS consuler : AMQP 1.0 trouble

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

NIFI JMS consuler : AMQP 1.0 trouble

GAQUIERE, Olivier

Hi all

 

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.

Did anyone manage to implement the JMS consumer to consume AMQP messages ?

If yes, which client librairies did you use ?

 

Thanks in advance

 

Regards

 

Olivier

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
Reply | Threaded
Open this post in threaded view
|

Re: NIFI JMS consuler : AMQP 1.0 trouble

Peter Turcsanyi-2
Hi Olivier,

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory


Regards,
Peter


On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:

Hi all

 

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.

Did anyone manage to implement the JMS consumer to consume AMQP messages ?

If yes, which client librairies did you use ?

 

Thanks in advance

 

Regards

 

Olivier

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
Reply | Threaded
Open this post in threaded view
|

RE: NIFI JMS consuler : AMQP 1.0 trouble

GAQUIERE, Olivier

Hi Peter

 

Thanks for your answer.

I tried your solution and I got the following error :

 

ConsumeJMS[id=c7e90ec4-0172-1000-ed11-6dd4c691e385] Failed to initialize JMS Connection Factory: org.apache.nifi.processor.exception.ProcessException: Could not obtain JMS Connection Factory from JNDI

 

Do you have an idea ?

Could you give me the exact configuration of your JMS consumer ?

 

Thanks in advance

 

Olivier GAQUIERE

 

Architect Solution Managing | CSD-Practice | Space/Public/ATM

Capgemini France - Toulouse

Tel.: +33 582 28 34 61

Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert

ni prise de connaissance ni réponse immédiate de votre part.

 

De : Peter Turcsanyi <[hidden email]>
Envoyé : vendredi 19 juin 2020 14:59
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

 

Hi Olivier,

 

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].

You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.

The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory

 

 

Regards,

Peter

 

 

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:

Hi all

 

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.

Did anyone manage to implement the JMS consumer to consume AMQP messages ?

If yes, which client librairies did you use ?

 

Thanks in advance

 

Regards

 

Olivier

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Reply | Threaded
Open this post in threaded view
|

RE: NIFI JMS consuler : AMQP 1.0 trouble

GAQUIERE, Olivier
In reply to this post by Peter Turcsanyi-2

Hi again Peter

 

Here is my processor configuration

 

 

Olivier GAQUIERE

 

Architect Solution Managing | CSD-Practice | Space/Public/ATM

Capgemini France - Toulouse

Tel.: +33 582 28 34 61

Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert

ni prise de connaissance ni réponse immédiate de votre part.

 

De : Peter Turcsanyi <[hidden email]>
Envoyé : vendredi 19 juin 2020 14:59
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

 

Hi Olivier,

 

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].

You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.

The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory

 

 

Regards,

Peter

 

 

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:

Hi all

 

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.

Did anyone manage to implement the JMS consumer to consume AMQP messages ?

If yes, which client librairies did you use ?

 

Thanks in advance

 

Regards

 

Olivier

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Reply | Threaded
Open this post in threaded view
|

Re: NIFI JMS consuler : AMQP 1.0 trouble

Mark Payne
Olivier,

I don’t know much of anything about qpid. But I did notice that the JNDI Name that you provided is topicConnectionFactry rather than topicConnectionFactory. I.e., “Factory” is misspelled. Would recommend you double-check that.

Thanks
-Mark

On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <[hidden email]> wrote:

Hi again Peter
 
Here is my processor configuration
 
<image002.png>
 
Olivier GAQUIERE
 
Architect Solution Managing | CSD-Practice | Space/Public/ATM
Capgemini France - Toulouse
Tel.: +33 582 28 34 61
<image001.png>
Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert
ni prise de connaissance ni réponse immédiate de votre part.
 
De : Peter Turcsanyi <[hidden email]> 
Envoyé : vendredi 19 juin 2020 14:59
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble
 
Hi Olivier,
 
You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory
 
 
Regards,
Peter
 
 
On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:
Hi all
 
I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?
 
Thanks in advance
 
Regards
 
Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Reply | Threaded
Open this post in threaded view
|

Re: NIFI JMS consuler : AMQP 1.0 trouble

Peter Turcsanyi-2
Olivier,

The problematic property is the JNDI Provider URL. It seems the amqp:// url cannot be specified here directly (though I believe I managed to do this earlier somehow, but I cannot reproduce it now).

The Qpid JMS documentation [1] says that you need to create a properties file (eg. jndi.properties but it can have any arbitrary name):
##
connectionfactory.ConnectionFactory = amqp://localhost:5672
##

and then reference this file in the JNDI Provider URL property via file:// url or OS filesystem path.

It works for me properly. My processor config:

JNDI Initial Context Factory Class: org.apache.qpid.jms.jndi.JmsInitialContextFactory
JNDI Provider URL: ${NIFI_HOME}/ext/qpid/jndi.properties
JNDI Name of the Connection Factory: ConnectionFactory 
JNDI / JMS Client Libraries: ${NIFI_HOME}/ext/qpid/lib

Please note that you must configure the same name in the JNDI Name of the Connection Factory property as in the properties file.

Hope it will work for you too.

Best,
Peter


On Mon, Jun 22, 2020 at 2:56 PM Mark Payne <[hidden email]> wrote:
Olivier,

I don’t know much of anything about qpid. But I did notice that the JNDI Name that you provided is topicConnectionFactry rather than topicConnectionFactory. I.e., “Factory” is misspelled. Would recommend you double-check that.

Thanks
-Mark

On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <[hidden email]> wrote:

Hi again Peter
 
Here is my processor configuration
 
<image002.png>
 
Olivier GAQUIERE
 
Architect Solution Managing | CSD-Practice | Space/Public/ATM
Capgemini France - Toulouse
Tel.: +33 582 28 34 61
<image001.png>
Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert
ni prise de connaissance ni réponse immédiate de votre part.
 
De : Peter Turcsanyi <[hidden email]> 
Envoyé : vendredi 19 juin 2020 14:59
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble
 
Hi Olivier,
 
You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory
 
 
Regards,
Peter
 
 
On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:
Hi all
 
I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?
 
Thanks in advance
 
Regards
 
Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Reply | Threaded
Open this post in threaded view
|

RE: NIFI JMS consuler : AMQP 1.0 trouble

GAQUIERE, Olivier

Hi Peter

 

Thanks very much for your help !!!

It perfectly works.

 

Have a nice day

 

Olivier

 

De : Peter Turcsanyi <[hidden email]>
Envoyé : lundi 22 juin 2020 23:28
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

 

Olivier,

 

The problematic property is the JNDI Provider URL. It seems the amqp:// url cannot be specified here directly (though I believe I managed to do this earlier somehow, but I cannot reproduce it now).

 

The Qpid JMS documentation [1] says that you need to create a properties file (eg. jndi.properties but it can have any arbitrary name):

##

connectionfactory.ConnectionFactory = amqp://localhost:5672

##

 

and then reference this file in the JNDI Provider URL property via file:// url or OS filesystem path.

 

It works for me properly. My processor config:

 

JNDI Initial Context Factory Class: org.apache.qpid.jms.jndi.JmsInitialContextFactory
JNDI Provider URL: ${NIFI_HOME}/ext/qpid/jndi.properties
JNDI Name of the Connection Factory: ConnectionFactory 
JNDI / JMS Client Libraries: ${NIFI_HOME}/ext/qpid/lib

 

Please note that you must configure the same name in the JNDI Name of the Connection Factory property as in the properties file.

 

Hope it will work for you too.

 

Best,

Peter

 

 

On Mon, Jun 22, 2020 at 2:56 PM Mark Payne <[hidden email]> wrote:

Olivier,

 

I don’t know much of anything about qpid. But I did notice that the JNDI Name that you provided is topicConnectionFactry rather than topicConnectionFactory. I.e., “Factory” is misspelled. Would recommend you double-check that.

 

Thanks

-Mark



On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <[hidden email]> wrote:

 

Hi again Peter

 

Here is my processor configuration

 

<image002.png>

 

Olivier GAQUIERE

 

Architect Solution Managing | CSD-Practice | Space/Public/ATM

Capgemini France - Toulouse

Tel.: +33 582 28 34 61

<image001.png>

Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert

ni prise de connaissance ni réponse immédiate de votre part.

 

De : Peter Turcsanyi <[hidden email]
Envoyé : vendredi 19 juin 2020 14:59
À : [hidden email]
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

 

Hi Olivier,

 

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].

You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.

The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory

 

 

Regards,

Peter

 

 

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <[hidden email]> wrote:

Hi all

 

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.

Did anyone manage to implement the JMS consumer to consume AMQP messages ?

If yes, which client librairies did you use ?

 

Thanks in advance

 

Regards

 

Olivier

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.