ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

Eric Chaves
Hi folks,

 I have a flow that downloads files from an FTP server over SSL using TLS1.2. To achieve this I use curl command line in an ExecuteProcess processor. This routine has been working ok until recently when we tried it on an upgraded  NiFi server.

After tracking down the error we noticed that it was due to the updated version of open-ssl recommendation of not allowing the use of old ciphers. The FTP server in question is using TLS1.2 with a weak certificate but since it is not managed by me updating the server is not an option.

After some troubleshooting I managed to adjust my curl command and it is working when I execute it manually in a bash session on my nifi server (to be precise I ran it inside the docker container that is running the nifi) but when I execute the same command line with the ExecuteProcess processor I got the following error: "failed setting cipher list"

The curl command and argument line I'm executing is:

curl -v -slk --tlsv1.2 --ciphers 'DEFAULT:!DH' --user ${FTP_USER}:${FTP_PASS} --ftp-ssl ftp://${FTP_HOST}:${FTP_PORT}/${FTP_DIR}/

The actual verbose error from inside the ExecuteProcess processor is: 

*   Trying 200.230.161.229...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f98e691f50)
* Connected to <server-name-redacted> (<ip-address-redacted>) port <port-redacted> (#0)
< 220 ProFTPD 1.3.4d Server (...) [<ip-address-redacted>]
> AUTH SSL
< 234 AUTH SSL successful
* failed setting cipher list: 'DEFAULT:!DH'
* Closing connection 0


So it seems that some configuration either on the nifi or the ExecuteProcess is not allowing me to force my curl command to use insecure ciphers with openssl.

How can I circumvent this?

Best regards,

Eric
Reply | Threaded
Open this post in threaded view
|

Re: ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

Andy LoPresto
Hi Eric,

Can you verify a couple things?

1. The specific curl instance you’re using in the terminal and in NiFi are the same? (i.e. run this command on the terminal and in an ExecuteProcess processor: $ which curl)
2. Run curl -V to see which version of openssl curl is using in both scenarios. 
3. Run curl -vvv to see increased verbosity output. 


Andy LoPresto
[hidden email]
[hidden email]
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 22, 2020, at 8:08 AM, Eric Chaves <[hidden email]> wrote:

Hi folks,

 I have a flow that downloads files from an FTP server over SSL using TLS1.2. To achieve this I use curl command line in an ExecuteProcess processor. This routine has been working ok until recently when we tried it on an upgraded  NiFi server.

After tracking down the error we noticed that it was due to the updated version of open-ssl recommendation of not allowing the use of old ciphers. The FTP server in question is using TLS1.2 with a weak certificate but since it is not managed by me updating the server is not an option.

After some troubleshooting I managed to adjust my curl command and it is working when I execute it manually in a bash session on my nifi server (to be precise I ran it inside the docker container that is running the nifi) but when I execute the same command line with the ExecuteProcess processor I got the following error: "failed setting cipher list"

The curl command and argument line I'm executing is:

curl -v -slk --tlsv1.2 --ciphers 'DEFAULT:!DH' --user ${FTP_USER}:${FTP_PASS} --ftp-ssl <a href="ftp://${FTP_HOST}:${FTP_PORT}/${FTP_DIR}/" class="">ftp://${FTP_HOST}:${FTP_PORT}/${FTP_DIR}/

The actual verbose error from inside the ExecuteProcess processor is: 

*   Trying 200.230.161.229...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f98e691f50)
* Connected to <server-name-redacted> (<ip-address-redacted>) port <port-redacted> (#0)
< 220 ProFTPD 1.3.4d Server (...) [<ip-address-redacted>]
> AUTH SSL
< 234 AUTH SSL successful
* failed setting cipher list: 'DEFAULT:!DH'
* Closing connection 0


So it seems that some configuration either on the nifi or the ExecuteProcess is not allowing me to force my curl command to use insecure ciphers with openssl.

How can I circumvent this?

Best regards,

Eric

Reply | Threaded
Open this post in threaded view
|

Re: ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

Eric Chaves
Hi Andy, sorry for not answering before. I Just figured this one out (after a lot of trial and error). This one was tricky. ;) 

The curl being used was the same one that I ran on bash. The error was related to how I was passing the arguments to curl. In bash I was passing the argument --ciphers 'DEFAULT:!DH' with a single quote to prevent bash expansion and when I declared the arguments on the processor I did the same however it seems that the processor does some quoting on it's own and curl was getting confused with the name of the cipher.

Once I removed the quotes the command worked just fine.

Thanks for the help anyway.


Em sex., 22 de mai. de 2020 às 15:11, Andy LoPresto <[hidden email]> escreveu:
Hi Eric,

Can you verify a couple things?

1. The specific curl instance you’re using in the terminal and in NiFi are the same? (i.e. run this command on the terminal and in an ExecuteProcess processor: $ which curl)
2. Run curl -V to see which version of openssl curl is using in both scenarios. 
3. Run curl -vvv to see increased verbosity output. 


Andy LoPresto
[hidden email]
[hidden email]
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 22, 2020, at 8:08 AM, Eric Chaves <[hidden email]> wrote:

Hi folks,

 I have a flow that downloads files from an FTP server over SSL using TLS1.2. To achieve this I use curl command line in an ExecuteProcess processor. This routine has been working ok until recently when we tried it on an upgraded  NiFi server.

After tracking down the error we noticed that it was due to the updated version of open-ssl recommendation of not allowing the use of old ciphers. The FTP server in question is using TLS1.2 with a weak certificate but since it is not managed by me updating the server is not an option.

After some troubleshooting I managed to adjust my curl command and it is working when I execute it manually in a bash session on my nifi server (to be precise I ran it inside the docker container that is running the nifi) but when I execute the same command line with the ExecuteProcess processor I got the following error: "failed setting cipher list"

The curl command and argument line I'm executing is:

curl -v -slk --tlsv1.2 --ciphers 'DEFAULT:!DH' --user ${FTP_USER}:${FTP_PASS} --ftp-ssl <a href="ftp://$%7BFTP_HOST%7D:$%7BFTP_PORT%7D/$%7BFTP_DIR%7D/" target="_blank">ftp://${FTP_HOST}:${FTP_PORT}/${FTP_DIR}/

The actual verbose error from inside the ExecuteProcess processor is: 

*   Trying 200.230.161.229...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f98e691f50)
* Connected to <server-name-redacted> (<ip-address-redacted>) port <port-redacted> (#0)
< 220 ProFTPD 1.3.4d Server (...) [<ip-address-redacted>]
> AUTH SSL
< 234 AUTH SSL successful
* failed setting cipher list: 'DEFAULT:!DH'
* Closing connection 0


So it seems that some configuration either on the nifi or the ExecuteProcess is not allowing me to force my curl command to use insecure ciphers with openssl.

How can I circumvent this?

Best regards,

Eric

Reply | Threaded
Open this post in threaded view
|

Re: ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

Andy LoPresto-2
Thanks Eric. Glad to know what the issue was and this should help people in the future. Always appreciate when people follow up and document a solved problem for the benefit of the community. 

Andy LoPresto
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69


On May 24, 2020, at 16:45, Eric Chaves <[hidden email]> wrote:


Hi Andy, sorry for not answering before. I Just figured this one out (after a lot of trial and error). This one was tricky. ;) 

The curl being used was the same one that I ran on bash. The error was related to how I was passing the arguments to curl. In bash I was passing the argument --ciphers 'DEFAULT:!DH' with a single quote to prevent bash expansion and when I declared the arguments on the processor I did the same however it seems that the processor does some quoting on it's own and curl was getting confused with the name of the cipher.

Once I removed the quotes the command worked just fine.

Thanks for the help anyway.


Em sex., 22 de mai. de 2020 às 15:11, Andy LoPresto <[hidden email]> escreveu:
Hi Eric,

Can you verify a couple things?

1. The specific curl instance you’re using in the terminal and in NiFi are the same? (i.e. run this command on the terminal and in an ExecuteProcess processor: $ which curl)
2. Run curl -V to see which version of openssl curl is using in both scenarios. 
3. Run curl -vvv to see increased verbosity output. 


Andy LoPresto
[hidden email]
[hidden email]
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 22, 2020, at 8:08 AM, Eric Chaves <[hidden email]> wrote:

Hi folks,

 I have a flow that downloads files from an FTP server over SSL using TLS1.2. To achieve this I use curl command line in an ExecuteProcess processor. This routine has been working ok until recently when we tried it on an upgraded  NiFi server.

After tracking down the error we noticed that it was due to the updated version of open-ssl recommendation of not allowing the use of old ciphers. The FTP server in question is using TLS1.2 with a weak certificate but since it is not managed by me updating the server is not an option.

After some troubleshooting I managed to adjust my curl command and it is working when I execute it manually in a bash session on my nifi server (to be precise I ran it inside the docker container that is running the nifi) but when I execute the same command line with the ExecuteProcess processor I got the following error: "failed setting cipher list"

The curl command and argument line I'm executing is:

curl -v -slk --tlsv1.2 --ciphers 'DEFAULT:!DH' --user ${FTP_USER}:${FTP_PASS} --ftp-ssl <a href="ftp://$%7BFTP_HOST%7D:$%7BFTP_PORT%7D/$%7BFTP_DIR%7D/" target="_blank">ftp://${FTP_HOST}:${FTP_PORT}/${FTP_DIR}/

The actual verbose error from inside the ExecuteProcess processor is: 

*   Trying 200.230.161.229...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f98e691f50)
* Connected to <server-name-redacted> (<ip-address-redacted>) port <port-redacted> (#0)
< 220 ProFTPD 1.3.4d Server (...) [<ip-address-redacted>]
> AUTH SSL
< 234 AUTH SSL successful
* failed setting cipher list: 'DEFAULT:!DH'
* Closing connection 0


So it seems that some configuration either on the nifi or the ExecuteProcess is not allowing me to force my curl command to use insecure ciphers with openssl.

How can I circumvent this?

Best regards,

Eric