Building with gradlew on core24/core22 escapes override-build

lonroth · July 8, 2024, 4:01pm

I have a snapcraft.yaml which builds a java application. I’m building with core24.

The snapcraft.yaml implements the override-build and executes as part of the build, a shipped “gradlew” script which performs the build.

My override-build looks like this:

    override-build: |
      echo "BEGIN BUILD OF BESU - gradlew"
      ./gradlew --no-daemon installDist
      echo "COMPLETED BUILD OF BESU - gradlew"
      mkdir -p "${CRAFT_PART_INSTALL}"/bin
      cp build/install/besu/bin/besu ${CRAFT_PART_INSTALL}/bin/besu
      cp build/install/besu/bin/evmtool ${CRAFT_PART_INSTALL}/bin/evmtool

The “gradlew” command starts and completes, but, nothing AFTER this is ever executed (not even echo). This is from my build log:

2024-07-08 17:51:04.767 remove directory /root/parts/besu/build                                                                                                                                                                              
2024-07-08 17:51:09.819 :: + echo 'BEGIN BUILD OF BESU - gradlew'                                                                                                                                                                            
2024-07-08 17:51:09.819 :: BEGIN BUILD OF BESU - gradlew                                                                                                                                                                                     
2024-07-08 17:51:09.819 :: + ./gradlew --no-daemon installDist
...
...
...
2024-07-08 17:55:03.609 :: BUILD SUCCESSFUL in 3m 53s                                                                                                                                                                                        
2024-07-08 17:55:03.609 :: 95 actionable tasks: 95 executed                                                                                                                                                                                  
2024-07-08 17:55:04.485 Building utils                                                                                                                                                                                                       
2024-07-08 17:55:04.486 execute action utils:Action(part_name='utils', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))                      
2024-07-08 17:55:04.486 load state file: /root/parts/utils/state/pull                                                                                                                                                                        
2024-07-08 17:55:04.489 remove directory /root/parts/utils/build                                                                                                                                                                             
2024-07-08 17:55:04.494 Executing PosixPath('/root/parts/utils/run/build.sh')                                                                                                                                                                
2024-07-08 17:55:04.495 :: + cp --archive --link --no-dereference . /root/parts/utils/install                                                                                                                                                
2024-07-08 17:55:04.869 Building wrappers                                                                                                                                                                                                    
2024-07-08 17:55:04.869 execute action wrappers:Action(part_name='wrappers', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))                
2024-07-08 17:55:04.870 load state file: /root/parts/wrappers/state/pull                                                                                                                                                                     
2024-07-08 17:55:04.872 remove directory /root/parts/wrappers/build                                                                                                                                                                          
2024-07-08 17:55:04.879 Executing PosixPath('/root/parts/wrappers/run/build.sh')                                                                                                                                                             
2024-07-08 17:55:04.880 :: + cp --archive --link --no-dereference . /root/parts/wrappers/install                                                                                                                                             
2024-07-08 17:55:05.239 Staging besu                                                                                                                                                                                                         
2024-07-08 17:55:05.240 execute action besu:Action(part_name='besu', step=Step.STAGE, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))                        
2024-07-08 17:55:05.241 /root/parts/besu/install/bin: No such file or directory                                                                                                                                                              
2024-07-08 17:55:05.241 Launching shell on build environment...                                                                                                                                                                              
2024-07-08 17:55:05.241 Emitter: Pausing control of the terminal                                                                                                                                                                             
snapcraft-besu-on-amd64-for-amd64-33947854 ../project#

As you can see above, the string I would expect: “COMPLETED BUILD OF BESU - gradlew” never appears and the files I need to copy into place are subsequently not being copied either and failing the over all build.

It seems that the gradlew command somehow makes the rest of override-build not to execute.

I have reproduced this on core22 as well.

There seems to be a similar bug in Launchpad: Bug #1987853 “snapcraft terminates override-build script prematu...” : Bugs : Snapcraft

What is going on here and how can I fix this?

nokan · August 13, 2024, 6:00pm

I had a similar problem in an u-boot build. I think it was related to build script’s calling an executable which expects input from stdin. It breaks the snapcraft build flow some how.

I fixed that by calling that step in a sub shell. It may worth a try, changing gradlew --no-daemon.. line to

sh -c './gradlew --no-daemon installDist'

lonroth · August 13, 2024, 9:38pm

I think I tried this but couldn’t get it to work.

lonroth · August 13, 2024, 9:44pm

I also hit the proxy problem, which I had to implement features to circumvent in the canonical hosted buildfarm:

The script $SNAPCRAFT_STAGE/utils/getproxy.sh extracts the proxy variables if present and uses them for gradle to be able to fetch stuff from internet.

Its separate from this problem, but related to gradlew.

Full snapcraft.yaml here.


parts:
  besu:
    after: [utils]
    plugin: nil
    build-packages: 
      - openjdk-21-jdk-headless
    source: https://github.com/hyperledger/besu.git
    source-tag: 24.7.0
    stage-packages:
      - openjdk-21-jre-headless
      # libjemalloc requires $LD_LIBRARY_PATH:$SNAP/usr/lib
      - libjemalloc-dev
      # The jna package is needed which also needs -Djna.xxx configs to work.
      - libjna-java

    override-pull: |
      craftctl default
      craftctl set version="24.7.0-$(git rev-parse --short HEAD)"

    override-build: |
      echo "BEGIN BUILD OF BESU - gradlew"

      echo "Extracting potential https_proxy and/pr http_proxy as needed when building with Canonical buildfarm"
      PROXY_OPTS=$($SNAPCRAFT_STAGE/utils/getproxy.sh)
      GRADLE_OPTS="-Djava.net.useSystemProxies=true $PROXY_OPTS" ./gradlew --no-daemon installDist
      
      echo "COMPLETED BUILD OF BESU - gradlew"