NiFi-1.10.0 memory leak

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

NiFi-1.10.0 memory leak

Anurag Sharma-2
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag
Reply | Threaded
Open this post in threaded view
|

Re: NiFi-1.10.0 memory leak

Bryan Bende
Hello,

Which part is showing that it is increasing over time?

The heap graph looks pretty standard where it goes back and forth when GC happens, but is staying in the same range.

Thanks,

Bryan


On Wed, Jul 15, 2020 at 4:21 AM Anurag Sharma <[hidden email]> wrote:
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag
Reply | Threaded
Open this post in threaded view
|

Re: NiFi-1.10.0 memory leak

Anurag Sharma-2
Hi Bryan, 

Heap and Non-heap are stable as I said. We seem to be losing memory in the Other (JVM) category. 

*******



Regards
Anurag

On Wed, Jul 15, 2020 at 6:42 PM Bryan Bende <[hidden email]> wrote:
Hello,

Which part is showing that it is increasing over time?

The heap graph looks pretty standard where it goes back and forth when GC happens, but is staying in the same range.

Thanks,

Bryan


On Wed, Jul 15, 2020 at 4:21 AM Anurag Sharma <[hidden email]> wrote:
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag
Reply | Threaded
Open this post in threaded view
|

Re: NiFi-1.10.0 memory leak

Arpad Boda-3
Anurag,

What's your Java version?

On Wed, Jul 15, 2020 at 5:15 PM Anurag Sharma <[hidden email]> wrote:
Hi Bryan, 

Heap and Non-heap are stable as I said. We seem to be losing memory in the Other (JVM) category. 

*******



Regards
Anurag

On Wed, Jul 15, 2020 at 6:42 PM Bryan Bende <[hidden email]> wrote:
Hello,

Which part is showing that it is increasing over time?

The heap graph looks pretty standard where it goes back and forth when GC happens, but is staying in the same range.

Thanks,

Bryan


On Wed, Jul 15, 2020 at 4:21 AM Anurag Sharma <[hidden email]> wrote:
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag
Reply | Threaded
Open this post in threaded view
|

Re: NiFi-1.10.0 memory leak

Anurag Sharma-2
Thanks Arpad, 

Missed that important detail. 

We tried on 2 JVMs and faced the same issue

1)
image.png

2) 
image.png


Regards
Anurag

On Wed, Jul 15, 2020 at 8:46 PM Arpad Boda <[hidden email]> wrote:
Anurag,

What's your Java version?

On Wed, Jul 15, 2020 at 5:15 PM Anurag Sharma <[hidden email]> wrote:
Hi Bryan, 

Heap and Non-heap are stable as I said. We seem to be losing memory in the Other (JVM) category. 

*******



Regards
Anurag

On Wed, Jul 15, 2020 at 6:42 PM Bryan Bende <[hidden email]> wrote:
Hello,

Which part is showing that it is increasing over time?

The heap graph looks pretty standard where it goes back and forth when GC happens, but is staying in the same range.

Thanks,

Bryan


On Wed, Jul 15, 2020 at 4:21 AM Anurag Sharma <[hidden email]> wrote:
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag
Reply | Threaded
Open this post in threaded view
|

Re: NiFi-1.10.0 memory leak

Juan Pablo Gardella
In reply to this post by Anurag Sharma-2
Which java version are you using? For the evidence you are sharing, it seems there is no memory leak. The increase of the memory could be related to the GC you are using. For example for Java 12, GC could return unused memory to the OS. Other GC will not release memory usage to the OS. For example, if you allow 20 GB of memory to the java program, maybe it will take all of them, although it really is using 1GB. Newer GCs, will use only 1 GB.

On Wed, 15 Jul 2020 at 12:15, Anurag Sharma <[hidden email]> wrote:
Hi Bryan, 

Heap and Non-heap are stable as I said. We seem to be losing memory in the Other (JVM) category. 

*******



Regards
Anurag

On Wed, Jul 15, 2020 at 6:42 PM Bryan Bende <[hidden email]> wrote:
Hello,

Which part is showing that it is increasing over time?

The heap graph looks pretty standard where it goes back and forth when GC happens, but is staying in the same range.

Thanks,

Bryan


On Wed, Jul 15, 2020 at 4:21 AM Anurag Sharma <[hidden email]> wrote:
Hi, 

We have been using NiFi-1.10.0 and with time even though heap and non-heap memories are pretty stable. We see overall memory used by the NiFi process is increasing over time. Is there a known scenario of NiFi-1.10.0 leaking native memory?

image.png

image.png

Regards
Anurag