hi,
when i am trying to build kernel from ubuntu-xenial (git://kernel.ubuntu.com/ubuntu/ubuntu-xenial.git)
$snapcraft --target-arch arm64#
…
make -j4 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image modules dtbs
It always picks -j4 option , how can i override this -j option, so that i can reduce the build time.
regards,
vinaysimha
Theoretically it should check for the amount of CPU cores your build machine has and set this value automatically, pehaps @ppisati knows if there is a way to override it …
Adding disable-parallel: true to the part definition should fix that.
Thanks for pointing to that option … though i think the OP wants the reverse … using more threads than he has cores for …
Sorry about that. I meant to of course suggest disable-parallel: false there. The default is what I accidentally posted above.
Edit: Now I’m confused… I did say true above, to disable multiple cores. That’s what you’re saying the OP wants? Else I’m not following.
To be clear: by default Snapcraft picks the number from multiprocessing.cpu_count which in Python should be the amount of cores you have available.
multiprocessing.cpu_count is located in which file.
is it possible to override from snapcraft.yaml?
so that i can harcode/override it to required number of threads. Currently in my machine if snapcraft takes lscpu (j4) it takes around 1-2 hours.
But by default if i build 4.x kernel by j24/j32 it takes around 15-20 minutes.
regards,
vinaysimha
snapcraft/_options.py: … @property def parallel_build_count(self): build_count = 1 if self.__parallel_builds: try: build_count = multiprocessing.cpu_count() except NotImplementedError: logger.warning( ‘Unable to determine CPU count; disabling parallel builds’)
return build_count
…
and:
snapcraft/plugins/kbuild.py: … def init(self, name, options, project): super().init(name, options, project) self.build_packages.extend([‘bc’, ‘gcc’, ‘make’])
self.make_targets = []
self.make_install_targets = ['install']
self.make_cmd = [
'make', '-j{}'.format(self.parallel_build_count)]
if logger.isEnabledFor(logging.DEBUG):
self.make_cmd.append('V=1')
…
if i read it correctly, parallel_build_count is determined at runtime, and
there’s no way to override it.
Patches are welcome, i guess 
calculation of threads should be based on (CPUs, cores per CPU, threads per core).
Based on my lscpu ( it should set 422 = 16 or twice of it 32)
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
how to download and build the snapcraft?
The source is at:
AFAIK you can just pip install it and run it from the tree directly to test changes… see the hacking guide linked from the README.md
multiprocessing.cpu_count() , there are no other methods in multiprocessing module to figure out the threads per core, cores per socket.
every time i modify the kernel files , i need to clean the build and rebuild kernel again with “snapcraft --target-arch arm64” command.
is there any better approach to build, create the Image and kernel snap?
Even if i modify just one .c file in kernel, i need to clean the build and build again ,
snapcraft --target-arch arm64
does not check for modified file and rebuild the kernel and pack again
please suggest.
Even if i modify just one .c file in kernel, i need to clean the build and build again ,
snapcraft --target-arch arm64
does not check for modified file and rebuild the kernel and pack again
please suggest.
Snapcraft is not make, and it doesn’t have the ability to do ‘cumulative builds’ - you’ll have to rebuild from scratch, everytime you do a modification.
rebuilding is fine, but is it possible to increase the thread level other than checking the multiprocessing.cpu_count() ? it takes me 2 hours to clean and build 
Can we have check for environmental variable for snapcraft to set thread(j) option , if it is not present then it should check for multiprocessing.cpu_count().
multiprocessing.cpu_count() already takes into account logical cpus, and CPU(s) is the total numer of threads your system is capable of.
The correct formula is: Thread(s) per core x Core(s) per socket x Socket(s) - and i bet in your case Socket(s) is equal to 1 so there’s nothing to fix, 4 is the correct number.
in my case it should be
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
422 = 16 , it can be 32 also.
4x2x2 = 16 , it can be 32 also.