Custom service in NAR generation failure

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Custom service in NAR generation failure

Etienne Jouvin
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Etienne Jouvin
Just for information, and for me to remember.



    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));
                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

Should not be return dependency.getVersion() ?
Because the artifact is the currently parsed artifact and not the dependency one ?



Le ven. 19 juin 2020 à 09:35, Etienne Jouvin <[hidden email]> a écrit :
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Mike Thomsen
In reply to this post by Etienne Jouvin
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Etienne Jouvin
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Bryan Bende
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Etienne Jouvin
My parent pom has this as declaration :

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.11.4</version>
    </parent>

When I studied the maven plugin, I found the following in class org.apache.nifi.extension.definition.extraction.ExtensionClassLoaderFactory.java 
    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
In this case, it search artifact in the dependencies.

If not found, check from provided dependencies (in fact from artifact that the current artifact depends on, if I well understood)
And the function is : 
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

And again if I well understood the code, it search in artifact to match the one for specific group and artifact ids, for example nifi-api.
But the version returned is not the one from the found artifact, but from the source artifact.

So that's why I explicitly set dependencies in the artifact pom to solve temporary the difficulty.

In the PR, I made the following change :
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + dependency.getVersion());
                        return dependency.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    } 

Do not know if this is the correct fix, I will way the pull request review.

Etienne
 


Le ven. 19 juin 2020 à 15:19, Bryan Bende <[hidden email]> a écrit :
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Juan Pablo Gardella
I am facing a similar problem and the workaround I used was installing the dependency in local repository.

On Fri, 19 Jun 2020 at 10:30, Etienne Jouvin <[hidden email]> wrote:
My parent pom has this as declaration :

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.11.4</version>
    </parent>

When I studied the maven plugin, I found the following in class org.apache.nifi.extension.definition.extraction.ExtensionClassLoaderFactory.java 
    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
In this case, it search artifact in the dependencies.

If not found, check from provided dependencies (in fact from artifact that the current artifact depends on, if I well understood)
And the function is : 
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

And again if I well understood the code, it search in artifact to match the one for specific group and artifact ids, for example nifi-api.
But the version returned is not the one from the found artifact, but from the source artifact.

So that's why I explicitly set dependencies in the artifact pom to solve temporary the difficulty.

In the PR, I made the following change :
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + dependency.getVersion());
                        return dependency.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    } 

Do not know if this is the correct fix, I will way the pull request review.

Etienne
 


Le ven. 19 juin 2020 à 15:19, Bryan Bende <[hidden email]> a écrit :
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Bryan Bende
In reply to this post by Etienne Jouvin
I haven't fully evaluated the fix, at a quick glance it seems correct, but I'm trying to figure out if something else is not totally correct in your poms because many other projects are using the latest NAR plugin and not having this issue, so there must be some difference that makes it work in some cases.

We have Maven archetypes for the processor and service bundles. I wonder if you could compare the resulting projects/poms with yours to see what seems different?



On Fri, Jun 19, 2020 at 9:30 AM Etienne Jouvin <[hidden email]> wrote:
My parent pom has this as declaration :

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.11.4</version>
    </parent>

When I studied the maven plugin, I found the following in class org.apache.nifi.extension.definition.extraction.ExtensionClassLoaderFactory.java 
    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
In this case, it search artifact in the dependencies.

If not found, check from provided dependencies (in fact from artifact that the current artifact depends on, if I well understood)
And the function is : 
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

And again if I well understood the code, it search in artifact to match the one for specific group and artifact ids, for example nifi-api.
But the version returned is not the one from the found artifact, but from the source artifact.

So that's why I explicitly set dependencies in the artifact pom to solve temporary the difficulty.

In the PR, I made the following change :
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + dependency.getVersion());
                        return dependency.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    } 

Do not know if this is the correct fix, I will way the pull request review.

Etienne
 


Le ven. 19 juin 2020 à 15:19, Bryan Bende <[hidden email]> a écrit :
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Etienne Jouvin
Some update.

I wanted to reproduce the error in a fresh new project. But no way to have it again.
So for the moment, I am not able to show an example.

I will give it a try later.

Sorry about that


Le ven. 19 juin 2020 à 15:48, Bryan Bende <[hidden email]> a écrit :
I haven't fully evaluated the fix, at a quick glance it seems correct, but I'm trying to figure out if something else is not totally correct in your poms because many other projects are using the latest NAR plugin and not having this issue, so there must be some difference that makes it work in some cases.

We have Maven archetypes for the processor and service bundles. I wonder if you could compare the resulting projects/poms with yours to see what seems different?



On Fri, Jun 19, 2020 at 9:30 AM Etienne Jouvin <[hidden email]> wrote:
My parent pom has this as declaration :

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.11.4</version>
    </parent>

When I studied the maven plugin, I found the following in class org.apache.nifi.extension.definition.extraction.ExtensionClassLoaderFactory.java 
    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
In this case, it search artifact in the dependencies.

If not found, check from provided dependencies (in fact from artifact that the current artifact depends on, if I well understood)
And the function is : 
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

And again if I well understood the code, it search in artifact to match the one for specific group and artifact ids, for example nifi-api.
But the version returned is not the one from the found artifact, but from the source artifact.

So that's why I explicitly set dependencies in the artifact pom to solve temporary the difficulty.

In the PR, I made the following change :
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + dependency.getVersion());
                        return dependency.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    } 

Do not know if this is the correct fix, I will way the pull request review.

Etienne
 


Le ven. 19 juin 2020 à 15:19, Bryan Bende <[hidden email]> a écrit :
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne
Reply | Threaded
Open this post in threaded view
|

Re: Custom service in NAR generation failure

Juan Pablo Gardella
I will try to reproduce it (I have a project where I can reproduce the problem) and then fill a JIRA. I hope I can reproduce it again :)

Juan

On Fri, 26 Jun 2020 at 09:19, Etienne Jouvin <[hidden email]> wrote:
Some update.

I wanted to reproduce the error in a fresh new project. But no way to have it again.
So for the moment, I am not able to show an example.

I will give it a try later.

Sorry about that


Le ven. 19 juin 2020 à 15:48, Bryan Bende <[hidden email]> a écrit :
I haven't fully evaluated the fix, at a quick glance it seems correct, but I'm trying to figure out if something else is not totally correct in your poms because many other projects are using the latest NAR plugin and not having this issue, so there must be some difference that makes it work in some cases.

We have Maven archetypes for the processor and service bundles. I wonder if you could compare the resulting projects/poms with yours to see what seems different?



On Fri, Jun 19, 2020 at 9:30 AM Etienne Jouvin <[hidden email]> wrote:
My parent pom has this as declaration :

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.11.4</version>
    </parent>

When I studied the maven plugin, I found the following in class org.apache.nifi.extension.definition.extraction.ExtensionClassLoaderFactory.java 
    private String determineProvidedEntityVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) throws ProjectBuildingException, MojoExecutionException {
        getLog().debug("Determining provided entities for " + groupId + ":" + artifactId);
        for (final Artifact artifact : artifacts) {
            if (artifact.getGroupId().equals(groupId) && artifact.getArtifactId().equals(artifactId)) {
                return artifact.getVersion();
            }
        }
        return findProvidedDependencyVersion(artifacts, groupId, artifactId);
    }
In this case, it search artifact in the dependencies.

If not found, check from provided dependencies (in fact from artifact that the current artifact depends on, if I well understood)
And the function is : 
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + artifact.getVersion());
                        return artifact.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    }

And again if I well understood the code, it search in artifact to match the one for specific group and artifact ids, for example nifi-api.
But the version returned is not the one from the found artifact, but from the source artifact.

So that's why I explicitly set dependencies in the artifact pom to solve temporary the difficulty.

In the PR, I made the following change :
    private String findProvidedDependencyVersion(final Set<Artifact> artifacts, final String groupId, final String artifactId) {
        final ProjectBuildingRequest projectRequest = new DefaultProjectBuildingRequest();
        projectRequest.setRepositorySession(repoSession);
        projectRequest.setSystemProperties(System.getProperties());
        projectRequest.setLocalRepository(localRepo);
        for (final Artifact artifact : artifacts) {
            final Set<Artifact> artifactDependencies = new HashSet<>();
            try {
                final ProjectBuildingResult projectResult = projectBuilder.build(artifact, projectRequest);
                gatherArtifacts(projectResult.getProject(), artifactDependencies);
                getLog().debug("For Artifact " + artifact + ", found the following dependencies:");
                artifactDependencies.forEach(dep -> getLog().debug(dep.toString()));

                for (final Artifact dependency : artifactDependencies) {
                    if (dependency.getGroupId().equals(groupId) && dependency.getArtifactId().equals(artifactId)) {
                        getLog().debug("Found version of " + groupId + ":" + artifactId + " to be " + dependency.getVersion());
                        return dependency.getVersion();
                    }
                }
            } catch (final Exception e) {
                getLog().warn("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API");
                getLog().debug("Unable to construct Maven Project for " + artifact + " when attempting to determine the expected version of NiFi API", e);
            }
        }
        return null;
    } 

Do not know if this is the correct fix, I will way the pull request review.

Etienne
 


Le ven. 19 juin 2020 à 15:19, Bryan Bende <[hidden email]> a écrit :
If you are not using nifi-nar-bundles as your parent (which is fine), then you should be explicitly setting versions for nifi-api and nifi-framework-api.

Otherwise how would it know to use 1.11.4 ?


On Fri, Jun 19, 2020 at 9:09 AM Etienne Jouvin <[hidden email]> wrote:
Ok, will try to just post simple thing.

The project has the following :
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
    </dependencies>
</project>

the nar project ::

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>ch.amexio.nifi.transform</groupId>
        <artifactId>nifi-transform-nar-bundles</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>nifi-transform-service-api-nar</artifactId>
    <packaging>nar</packaging>

    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <!-- Project dependencies. -->
        <dependency>
            <groupId>ch.amexio.nifi.transform</groupId>
            <artifactId>nifi-transform-service-api</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!-- NiFi dependencies. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <type>nar</type>
        </dependency>
    </dependencies>
</project>

It was then in failure.
What I did, is to change the my parent pom and add the following in dependencies
    <dependencies>
        <!-- NiFi dependencies. -->
        <!-- Required for NAR generation. -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-framework-api</artifactId>
        </dependency>
    </dependencies>


By the way, I submit a Pull Request on nifi-maven
With following change :

Etienne
 
 

Le ven. 19 juin 2020 à 13:52, Mike Thomsen <[hidden email]> a écrit :
Without seeing your POM(s), it could be several things. Try posting your POMs here or as a GitHub gist.

On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin <[hidden email]> wrote:
Hello all.

Do not know where to post the message, guide me if I should send to another mailing list.
A simple summary in first step.
I created a simple project to build a new service.
I extend the nifi-nar-bundles artifact with version 1.11.4.
My project version is currently 0.0.1-SNAPSHOT.

During NAR generation, it failed for the documentation with message :
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension Documentation
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not resolve local dependency org.apache.nifi:nifi-api:jar:0.0.1-SNAPSHOT

I am currently looking in source code of nifi-maven project, specially class ExtensionClassLoaderFactory.

What I do not understand is why it searches for version 0.0.1-SNAPSHOT on nifi-api, and not the version 1.11.4

Let me know if I should discuss about this in another thread.

Regards

Etienne