#arm-linux

2023-01-05

Vince Carey (07:05:26): > @Vince Carey has joined the channel

Vince Carey (07:05:26): > set the channel description: How can bioc support arm-linux

Martin Grigorov (07:11:13): > @Martin Grigorov has joined the channel

Martin Grigorov (07:11:39): > :wave:Hello, team! Thanks for the invitation!

Martin Grigorov (07:18:02): > To answer@Vince Carey’s questions fromhttps://stat.ethz.ch/pipermail/bioc-devel/2023-January/019399.html: > 1. Building linux/arm64 Docker image should be relatively easy withdocker buildx. It uses QEMU when executed on x86_64. It might be slow but it will work! > 2. Github Actions supports Linux ARM64 only as a self-hosted runner. There is an issue for adding github-hosted Linux ARM64 runners but it is not scheduled yet by the GHA team > > 1. one can execute jobs on github-hosted runner viahttps://github.com/uraimo/run-on-arch-action/but again it is based on QEMU and it might be slow

Martin Grigorov (07:18:43): > What build/CI system is being used for the MacOS ARM64 builds (https://bioconductor.org/checkResults/3.17/bioc-mac-arm64-LATEST/) ?

Martin Grigorov (07:19:20): > and/or the Linux x86_64 builds ?

Vince Carey (08:17:25): > @Andres Wokaty@Alex Mahmoudcan you comment here? We have a bespoke system for building and reporting and distributing … but are increasing integration with GHA and exploring other avenues.

Andres Wokaty (08:17:29): > @Andres Wokaty has joined the channel

Alex Mahmoud (08:17:29): > @Alex Mahmoud has joined the channel

2023-01-06

Martin Grigorov (07:27:04): > may I ask how many machines are used for the current Linux x86_64 and Mac ARM64 builds ? And what is their hardware specs ?

Andres Wokaty (08:58:24): > We have 2 for each, one for devel and another for release. The build system was created by core developers, which is available atgithub.com/Bioconductor/BBS. These are the hardware requirements for the builders:https://github.com/Bioconductor/BBS/blob/master/Doc/Prepare-Ubuntu-20.04-HOWTO.md#12-check-hardware-requirements. Our linux builders are central builders so they’re more powerful and have 72 cores, I believe, plus about a 1.7TB SSD. The Mac ARM64 builders are slower and do not build daily. They’re slightly different since the 3.16 was our first trial and we’ll likely change it be similar to the3.17 builder, which has 8 cores, 16GB RAM, and 1TBSSD.

2023-01-10

Alex Mahmoud (19:15:28) (in thread): > For the containerized side of things, I agree that building the docker image is now possible using rocker daily builds as a base, and I can add that to the list of things to do. Once the docker image exists, there will also be a path for building binaries for the packages with the containerized builder which is still a wip. I added the high level basic steps for the containerized environment tohttps://github.com/Bioconductor/bioconductor_docker/issues/64#issuecomment-1378061482, happy to chat about it if you have the time and want to contribute. Otherwise, i’ll probably get to it in the next few weeks or so - Attachment: Comment on #64 Make an ARM image available > There is now feasible path to do this with the rocker daily builds for arm64 now available.
> High level tasks: > > ☐ Add arm64 build via https://github.com/uraimo/run-on-arch-action/ or docker buildx > ☐ Use arm64 docker image as a new base to build binaries for packages via modified gha-build (still WIP) > ☐ Make BiocManager/AnVIL::install aware of architecture when picking binaries

2023-01-11

Martin Grigorov (07:47:05) (in thread): > Thanks for the answer,@Alex Mahmoud! I will take a look at the bioconductor_docker repo and send a PR!

Martin Grigorov (12:51:04) (in thread): > https://github.com/Bioconductor/bioconductor_docker/pull/67 - Attachment: #67 Issue #64: Build Linux ARM64 image in addition to x86_64 > Build weekly images with ARM64 > > Signed-off-by: Martin Tzvetanov Grigorov mailto:mgrigorov@apache.org|mgrigorov@apache.org

Alex Mahmoud (13:16:28) (in thread): > I made a few changes to ensure that the right tags (latestvsdevelare used for arm64 since rocker doesn’t provide arm64develtag yet), and made the PR only affect the GHCR build, then merged. Assuming this goes well, I can modify the weekly build script and the docker auto-build (setup on the dockerhub side) later, but figured getting the GHCR image out there asap will be good since it doesn’t need anything else afaict, so I merged that part separately

Alex Mahmoud (13:18:06) (in thread): > Thank you for the time and effort! This was extremely helpful to move up the timeline for getting this done

Alex Mahmoud (14:00:37) (in thread): > I forgot thatlatestcorresponds to the latest released version of R, notdevel, so I added the commits to RELEASE_3_16 branch where it should hopefully now pass with R 4.2.2 and Bioc 3.16 (was failing in devel since 3.17 expects R 4.3)

Martin Grigorov (14:34:46) (in thread): > @Alex Mahmouddo you understand what is the problem athttps://github.com/Bioconductor/bioconductor_docker/actions/runs/3895511043/jobs/6650915505?

Martin Grigorov (14:35:19) (in thread): > > > BiocManager::install(version="3.17", update=TRUE, ask=FALSE) > #9 30.75 Error: Bioconductor version '3.17' requires R version '4.3'; use > #9 30.75 `BiocManager::install(version = '3.16')` with R version 4.2; see >

Martin Grigorov (14:37:15) (in thread): > let me review your changes first! in my fork the checks were failing at push time due to wrong credentials (as expected)

Martin Grigorov (14:39:27) (in thread): > everything looks good to me! debugging …

Martin Grigorov (14:55:15) (in thread): > I’m afraid we may need to revert the merge:confused:It seems r-ver and rstudiolatestcome with R 4.2.x. And theirdeveltags have noarm64support

Martin Grigorov (14:56:56) (in thread): > Now I understand your last message (I forgot thatlatestcorresponds to the latest released version of R, notdevel, so I added the commits to RELEASE_3_16 branch where it should hopefully now pass with R 4.2.2 and Bioc 3.16 (was failing in devel since 3.17 expects R 4.3)) !!!

Martin Grigorov (14:58:28) (in thread): > https://github.com/Bioconductor/bioconductor_docker/actions/runs/3895823953runs fine so far (for the arm64 jobs!). One of the amd64 jobs needs restart

Alex Mahmoud (14:58:37) (in thread): > Sorry, was in meetings, but yes, the issue was that arm64 only exists for latest release4.2.2and not latest devel version, so I had to move it to the latest RELEASE branch, and still currently waiting on the builds here:https://github.com/Bioconductor/bioconductor_docker/actions/runs/3895823953/jobs/6651613900(I think the amd64 r-ver error is transient, but not restarting until arm64 builds exit)

Alex Mahmoud (14:59:19) (in thread): > oop you found it right before I sent that:slightly_smiling_face:

Martin Grigorov (14:59:48) (in thread): > :-)

Martin Grigorov (15:01:04) (in thread): > I’ll contact the maintainers of the rocker docker images to ask for arm64 fordeveltoo

Alex Mahmoud (15:02:38) (in thread): > They are already tracking progress in this issue, if you haven’t seen it already:https://github.com/rocker-org/rocker-versioned2/issues/144

Martin Grigorov (15:05:52) (in thread): > awesome!

Martin Grigorov (16:49:34) (in thread): > The GHCR arm64 jobs may not finish in 6 hours … In the logs of the successful amd64 job I see: > > #9 64.14 > #9 65.12 * installing **binary** package 'Rcpp' ... > #9 65.17 * DONE (Rcpp) > #9 65.42 * installing **binary** package 'utf8' ... > #9 65.42 * DONE (utf8) > #9 65.63 * installing **binary** package 'sys' ... > #9 65.64 * DONE (sys) > #9 65.90 * installing **binary** package 'zip' ... > > but there are no binary packages for arm64 and they are built from sources, and it takes its time …

Alex Mahmoud (16:52:47) (in thread): > I see. I suppose the way to go would be to do it on a dedicated VM the first time. Do you think the docker image would build overall if the binary packages existed?

2023-01-12

Martin Grigorov (01:48:50) (in thread): > I am not sure. I started building/testing bioc packages on Linux ARM4 and so far all builds fine, but there might be packages which will fail for some reason

Martin Grigorov (01:55:05) (in thread): > The builds actually passed!:slightly_smiling_face:They took just 3h and 20 mins!

2023-01-20

Yikun Jiang (03:13:36): > @Yikun Jiang has joined the channel

2023-01-26

Martin Grigorov (07:28:15): > Hello!@Vince Careyasked me to share any packages that have problems on Linux ARM64. This is just a FYI. I will report it later to the maintainer athttps://support.bioconductor.orgonce the initiative to support Linux ARM64 is officially approved/announced. > > package:[https://github.com/LTLA/basilisk](https://github.com/LTLA/basilisk)$ R CMD build basilisk > * checking for file 'basilisk/DESCRIPTION' ... OK > * preparing 'basilisk': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'motivation.Rmd' using rmarkdown > trying URL '[https://repo.anaconda.com/miniconda/Miniconda3-py38_4.12.0-Linux-x86_64.sh](https://repo.anaconda.com/miniconda/Miniconda3-py38_4.12.0-Linux-x86_64.sh)' > Content type 'application/x-sh' length 76120962 bytes (72.6 MB) > ================================================== > downloaded 72.6 MB > > PREFIX=/home/biocbuild/.cache/R/basilisk/1.11.2/0 > Unpacking payload ... > /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 412: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error > /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 414: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error > Quitting from lines 71-207 (motivation.Rmd) > Error: processing vignette 'motivation.Rmd' failed with diagnostics: > conda installation failed with status code '1' > --- failed re-building 'motivation.Rmd' > > SUMMARY: processing the following file failed: > 'motivation.Rmd' >

Martin Grigorov (07:59:23): > > [https://github.com/MarioniLab/simpleSingleCell](https://github.com/MarioniLab/simpleSingleCell)$ R CMD build simpleSingleCell > * checking for file 'simpleSingleCell/DESCRIPTION' ... OK > * preparing 'simpleSingleCell': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'batch.Rmd' using rmarkdown > --- finished re-building 'batch.Rmd' > > --- re-building 'bigdata.Rmd' using rmarkdown > --- finished re-building 'bigdata.Rmd' > > --- re-building 'de.Rmd' using rmarkdown > --- finished re-building 'de.Rmd' > > --- re-building 'doublets.Rmd' using rmarkdown > --- finished re-building 'doublets.Rmd' > > --- re-building 'intro.Rmd' using rmarkdown > --- finished re-building 'intro.Rmd' > > --- re-building 'misc.Rmd' using rmarkdown > Quitting from lines 51-78 (misc.Rmd) > Error: processing vignette 'misc.Rmd' failed with diagnostics: > not all 'rnames' found or unique. > > http://bioconductor.org/checkResults/devel/workflows-LATEST/simpleSingleCell- passes without such problems

Martin Grigorov (08:39:27): > > BiocManager::install(c("tiledb"), type = "source", checkBuilt = TRUE) > > * installing **source** package 'tiledb' ... > **** package 'tiledb' successfully unpacked and MD5 sums checked > **** using staged installation > checking whether the C++ compiler works... yes > checking for C++ compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether the compiler supports GNU C++... yes > checking whether g++ -std=gnu++14 accepts -g... yes > checking for g++ -std=gnu++14 option to enable C++11 features... none needed > checking how to run the C++ preprocessor... g++ -std=gnu++14 -E > checking whether the compiler supports GNU C++... (cached) yes > checking whether g++ -std=gnu++14 accepts -g... (cached) yes > checking for g++ -std=gnu++14 option to enable C++11 features... (cached) none needed > checking whether g++ -std=gnu++14 supports C++17 features by default... no > checking whether g++ -std=gnu++14 supports C++17 features with -std=gnu++17... yes > checking for pkg-config... /usr/bin/pkg-config > checking if pkg-config knows TileDB... no > checking for stdio.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for strings.h... yes > checking for sys/stat.h... yes > checking for sys/types.h... yes > checking for unistd.h... yes > checking for tiledb/tiledb... no > configure: error: currently unsupported system Linux on aarch64 > ERROR: configuration failed for package 'tiledb' > * removing '/home/biocbuild/bbs-3.17-bioc/R/site-library/tiledb' >

Martin Grigorov (09:27:37) (in thread): > https://github.com/theislab/zellkonverterfails due to the same problem: > > R CMD build zellkonverter > * checking for file 'zellkonverter/DESCRIPTION' ... OK > * preparing 'zellkonverter': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'zellkonverter.Rmd' using rmarkdown > PREFIX=/home/biocbuild/.cache/R/basilisk/1.11.2/0 > Unpacking payload ... > /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 412: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error > /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 414: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error > Quitting from lines 42-48 (zellkonverter.Rmd) > Error: processing vignette 'zellkonverter.Rmd' failed with diagnostics: > conda installation failed with status code '1' > --- failed re-building 'zellkonverter.Rmd' >

Vince Carey (09:29:22): > This makes perfect sense. There will need to be modifications to these packages that use or create intel-specific “runtimes” – so tiledb and basilisk (which needs a miniconda to isolate r-python interoperation) will need work. I will look at the other findings soon

Martin Grigorov (09:36:30) (in thread): > https://github.com/LTLA/basilisk.utils/blob/411502fabdca972bb10b4f12115b9f7bd5b05c5d/R/installConda.R#L125-L137

Martin Grigorov (09:37:10) (in thread): > The Linux branch needs a small update

2023-01-30

Martin Grigorov (03:36:37) (in thread): > https://github.com/LTLA/basilisk.utils/pull/7 - Attachment: #7 Add support for Linux ARM64/aarch64 > There is an ongoing initiative to add support for Linux ARM64: https://stat.ethz.ch/pipermail/bioc-devel/2023-January/019398.html > > Currently https://github.com/LTLA/basilisk and
> https://github.com/theislab/zellkonverter fail with: > > > PREFIX=/home/biocbuild/.cache/R/basilisk/1.11.2/0 > Unpacking payload ... > /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 412: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error /home/biocbuild/.cache/R/basilisk/Miniconda3-py38_4.12.0-Linux-x86_64.sh: line 414: /home/biocbuild/.cache/R/basilisk/1.11.2/0/conda.exe: cannot execute binary file: Exec format error Quitting from lines 71-207 (motivation.Rmd) > Error: processing vignette 'motivation.Rmd' failed with diagnostics: conda installation failed with status code '1' > --- failed re-building 'motivation.Rmd' > > > Signed-off-by: Martin Tzvetanov Grigorov mailto:mgrigorov@apache.org|mgrigorov@apache.org

Martin Grigorov (06:33:44): > Hi! We have uploaded the first reports athttps://yikun.github.io/bioconductor/report/long-report.htmlThe common types of problems are summarized athttps://github.com/Bioconductor/BBS/pull/255#issuecomment-1407937082

2023-02-15

Martin Grigorov (03:29:50): > Hi! Currently we collect the Linux ARM64 related issues athttps://github.com/Yikun/BBS/issues. > Would it be a good idea to create a new repo underhttps://github.com/bioconductor, e.g.https://github.com/bioconductor/arm-linux, for that purpose ? > We think that this way it would be easier to convince the package maintainers to take part in this initiative and help. > The repo would have just a README file explaining its purpose and issues enabled. Everything else could be disabled. > What do you think ?

Vince Carey (05:26:12): > Thanks@Martin Grigorov, the issues page is very informative. I think at this moment it would not be a good idea to ask community developers to work on this – we are approaching the 3.17 release, and everyone has to focus on the main platforms in use. I think there are two steps of immediate interest and feasibility – first, have you identified patterns of failure on ARM64 that are common across multiple packages? If so, have the rectifications of failures led to general improvement in code robustness? That’s a piece of news that would be of interest to developers, and if “repair hints” are not too complicated, publicizing them on the bioc-devel mailing list could have impact. Second, direct engagement with developers of packages that have large failure footprints by affecting downstream packages, especially with pull requests, can be undertaken by your group. After 3.17 is out we can discuss how to bring the platform closer to general CI/CD practices of bioconductor.

Martin Grigorov (06:22:09) (in thread): > Thank you,@Vince Carey!

Martin Grigorov (06:25:37) (in thread): > Yes, we have identified few groups of issues: > 1. Same problem on x86_64 -https://github.com/Yikun/BBS/issues?q=is%3Aissue+is%3Aopen+label%3AX86_ALSO_FAILED. I guess most of these will just work on aarch64 once they are fixed for x86_64 > 2. configure: error: cannot guess build type; you must specify one. We sent PRs for them, e.g.https://github.com/UBod/msa/pull/24 > 3. Usage of x86_64 specific CFLAGS like-m64and-msse2. We work on PRs which add support for arm64 NEON

Martin Grigorov (06:26:37) (in thread): > So far several of the package maintainers expressed willingness to help with investigation and/or fixes!

Martin Grigorov (06:30:52) (in thread): > we just thought thathttps://github.com/Yikun/might look not serious to someone. But you are right that we should not make it look like it is urgent or mandatory for 3.17!

Martin Grigorov (09:06:42): > I’d be thankful if you have few spare minutes to share your opinion onhttps://github.com/TileDB-Inc/TileDB-R/issues/520

Vince Carey (10:56:39) (in thread): > Thanks again. If you see some crucial packages for which authors are not responding let me know.

Vince Carey (11:03:11): > Let’s see if I understand the situation. The Bioconductor approach to tiledb is to use an ubuntu package for it and required runtimes. But if the BBS built tiledb from source then you would be in a better position to configure the tiledb package for R as needed by ARM64. I have to ask@Hervé Pagèsto weigh in here, I think he has looked at the tiledb situation previously. I think eventually we could build tiledb infrastructure from source for linux … not sure.

Hervé Pagès (11:03:16): > @Hervé Pagès has joined the channel

Hervé Pagès (15:12:58): > Just to clarify no Ubuntu package is needed to installtiledbon Intel Linux. Pre-built TileDB Embedded libraries get downloaded from GitHub at installation time. This is what I get on my Ubuntu 20.04 LTS (Intel) laptop: > > > install.packages("tiledb") > ... > * installing **source** package 'tiledb' ... > **** package 'tiledb' successfully unpacked and MD5 sums checked > **** using staged installation > checking whether the C++ compiler works... yes > ... > checking for unistd.h... yes > checking for tiledb/tiledb... no > downloading x86_64 TileDB library... > installing TileDB for Ubuntu:22.04... > downloading[https://github.com/TileDB-Inc/TileDB/releases/download/2.14.0/tiledb-linux-x86_64-2.14.0-27eed08.tar.gz](https://github.com/TileDB-Inc/TileDB/releases/download/2.14.0/tiledb-linux-x86_64-2.14.0-27eed08.tar.gz)using inst/tiledb/{lib,include} > configure: creating ./config.status > ... > > My understanding of the discussion athttps://github.com/TileDB-Inc/TileDB-R/issues/520is that unfortunately TileDB-Inc doesn’t provide pre-built TileDB Embedded libraries for aarch64 at the moment. So in the meantimelibtiledb-devneeds to be installed on aarch64 Ubuntu. Even though Dirk doesn’t recommend this becauselibtiledb-devtends to lag behind and also lacks features as e.g. AWS S3 support. However from a Bioconductor build system perspective maybe this won’t be a problem.tiledbis a CRAN package and we only need it on the build machines to support Bioconductor packageTileDBArray, so lack of AWS S3 support will only be a problem if the vignettes/examples/unit tests inTileDBArrayneed that feature. > Anyways, what other choice is there at the moment if TileDB-Inc doesn’t provide pre-built TileDB Embedded libraries for aarch64? A source build of TileDB on the build machine? Not very realistic, not just because that’s time consuming, but also because that means the end users would need to do that too.

Hervé Pagès (15:25:32): > Just posted a shorter version of the above here:https://github.com/Yikun/BBS/issues/26#issuecomment-1431980654 - Attachment: Comment on #26 tiledb.install > > It would be lovely if TileDB could be built from source so that the R package can be more fully-featured. > > Sure but from a Bioconductor build system perspective we only need the tiledb package on the build machines to support Bioconductor package TileDBArray, so lack of AWS S3 support would only be a problem if the vignettes/examples/unit tests in TileDBArray need that feature. And I don’t know if that’s the case so we’ll see. > > In other words, as long as using libtiledb-dev allows tiledb to install and TileDBArray to pass BUILD and CHECK, then we’re good. :smiley: > > BTW any idea if/when TileDB-Inc will provide pre-built TileDB Embedded libraries for aarch64? > > Thanks!

Tom Lin (22:05:06): > @Tom Lin has joined the channel

2023-02-16

Martin Grigorov (01:47:24): > Thank you,@Hervé Pagès!

2023-02-23

Martin Grigorov (02:58:35): > Here is the latest report on Linux ARM64 -https://yikun.github.io/bioconductor-0223/report/long-report.htmlIt looks very similar to the latest Mac ARM64 report now.

Martin Grigorov (03:00:50): > There is something strange though.https://yikun.github.io/bioconductor-0223/report/Rbowtie2/kunpeng1-install.htmlstill fails about missingsimdealthough it has been fixed withhttps://github.com/wzthu/Rbowtie2/pull/8andhttps://code.bioconductor.org/browse/Rbowtie2/commits/masterI have the feeling that BBS does not clone git submodules - Attachment: #8 Add ‘simde’ as a Git submodule > Fixes #7 > > Used the following command to add the submodule: > > git submodule add --name simde -b master [https://github.com/simd-everywhere/simde-no-tests.git](https://github.com/simd-everywhere/simde-no-tests.git) src/bowtie2-2.4.4/third_party/simde

Martin Grigorov (03:02:05) (in thread): > https://github.com/Bioconductor/BBS/blob/3908dbd4f5b1927f197d2dd57ba52a266be07847/bbs/gitutils.py#L78-L86

Martin Grigorov (03:03:08) (in thread): > we’ll test it and send a PR!

Hervé Pagès (14:30:44) (in thread): > Hi@Martin Grigorov, > Thanks for fixingRbowtie2compilation issue on arm Linux. About git submodules, how do I clone a git repos with all its submodules? I’ve tried things like > > git clone --recurse-submodules[https://github.com/wzthu/Rbowtie2](https://github.com/wzthu/Rbowtie2) > > or > > git clone --shallow-submodules[https://github.com/wzthu/Rbowtie2](https://github.com/wzthu/Rbowtie2) > > but they don’t seem to work (I’d expect to seesimde-no-testsshow up underRbowtie2/src/bowtie2-2.4.4/third_party/but I don’t see it). > Note that I’m not familiar with git submodules and I don’t know what the implications of allowing them in Bioconductor packages would be (if we ever decided to do that). This is not a light change and something that we might want to discuss internally first. > In the meantime, maybe thesimde-no-testssource tree could just be copied toRbowtie2? > One more thing: have you checked that the latest bowtie2 version (2.5.1) doesn’t solve the arm Linux problem already? If that’s the case, a better fix forRbowtie2would be to update their copy of the bowtie2 source tree (they’re still using bowtie2 2.4.4). In any case it feels that the arm Linux compilation problem should preferrably be fixed upstream in bowtie2 rather than inRbowtie2. > Thanks!

Martin Grigorov (14:37:00) (in thread): > Hi@Hervé Pagès! –recurse-submodules is the way. The idea is that instead of copying the sources of a third party (without its history) you tell Git to fetch it for you as a repo inside your repo

Martin Grigorov (14:38:14) (in thread): > We have made the changes in kunpeng1 builder and re-run the build. We will have the new results later tomorrow

Martin Grigorov (14:38:53) (in thread): > bowtie2 itself uses submodules

Martin Grigorov (14:39:44) (in thread): > But if you have concerns then I’ll propose to Rbowtie2’s maintainer to use a copy

Hervé Pagès (14:49:53) (in thread): > For some reason--recurse-submodulesdoesn’t work onhttps://github.com/wzthu/Rbowtie2: > > hpages@spectre:~$ git clone --recurse-submodules[https://github.com/wzthu/Rbowtie2](https://github.com/wzthu/Rbowtie2)Cloning into 'Rbowtie2'... > remote: Enumerating objects: 3649, done. > remote: Counting objects: 100% (58/58), done. > remote: Compressing objects: 100% (39/39), done. > remote: Total 3649 (delta 29), reused 40 (delta 19), pack-reused 3591 > Receiving objects: 100% (3649/3649), 27.06 MiB | 21.75 MiB/s, done. > > As you can see, the “simde” submodule doesn’t get cloned. Does it work for you? > > But if you have concerns then I’ll propose to Rbowtie2’s maintainer to use a copy > That would be the way to go for now, since Bioconductor doesn’t allow git submodules at the moment. Thanks!

Martin Grigorov (14:53:01) (in thread): > Would you please add a comment to the PR that submodules should not be used? > I will send a new PR tomorrow

Martin Grigorov (14:53:24) (in thread): > I’m on my phone now so I can’t test it

Hervé Pagès (15:24:00) (in thread): > Done:https://github.com/wzthu/Rbowtie2/pull/8#issuecomment-1442382699 - Attachment: Comment on #8 Add ‘simde’ as a Git submodule > Unfortunately Bioconductor doesn’t support git submodules at the moment so the simde-no-tests source tree would need to be copied to Rbowtie2. > > (BTW note that something seems to be wrong in how the “simde” submodule is currently registered in Rbowtie2 anyways, because git clone --recurse-submodules [https://github.com/wzthu/Rbowtie2](https://github.com/wzthu/Rbowtie2) ignores it.)

2023-02-24

Martin Grigorov (03:50:52) (in thread): > I believe this extra commit broke the submodule:https://code.bioconductor.org/browse/Rbowtie2/commit/4c32a2d0a9d1505bb984824fb1d3095839fd2009

Martin Grigorov (03:51:19) (in thread): > https://github.com/wzthu/Rbowtie2/pull/10- replaces the submodule with a copy of the simde sources

2023-03-09

Michael Love (09:30:28): > @Michael Love has joined the channel

Michael Love (09:32:01): > hi@Martin Grigorovthanks for the detailed instructions. Could you try a quick change and then if that doesn’t work I will go with #3 or #2 > > If you have a local copy of DESeq2, can you modifylower=0tolower=1e-6https://github.com/mikelove/DESeq2/blob/master/R/helper.R#L98

Martin Grigorov (09:32:40): > Marina Nikolova sounds like another Bulgarian:slightly_smiling_face:

Martin Grigorov (09:32:47): > sure! let me try it!

Michael Love (09:33:02): > Warning in sqrt(alpha * q) : NaNsmeans that something is sneaking in negative

Michael Love (09:33:13): > oops, sorry, I hit too quick

Martin Grigorov (09:34:50): > CMD buildis running!

Martin Grigorov (09:37:48): > the build passed! NowR CMD check DESeq2_1.39.7.tar.gzis running!

Michael Love (09:39:26): > it should be fine for me to modify DESeq2 unmix to use this epsilon value, also i think there are very few users of this function anyway. weird that sqrt(0) gives NaN

Martin Grigorov (09:40:36): > > > sqrt(0) > [1] 0 >

Martin Grigorov (09:41:04) (in thread): > or you mean some othersqrtdefinition ?

Martin Grigorov (09:41:20) (in thread): > checking examples ...

Martin Grigorov (09:42:43) (in thread): > > * checking examples ... OK > * checking for unstated dependencies in 'tests' ... OK > * checking tests ... > Running 'testthat.R' > > I think the problem is gone!

Martin Grigorov (09:48:33): > > ... > * checking examples ... OK > * checking for unstated dependencies in 'tests' ... OK > * checking tests ... > Running 'testthat.R' > OK > * checking for unstated dependencies in vignettes ... OK > * checking package vignettes in 'inst/doc' ... OK > * checking running R code from vignettes ... > 'DESeq2.Rmd' using 'UTF-8'... OK > NONE > * checking re-building of vignette outputs ... OK > * checking PDF version of manual ... OK > * DONE > > Status: 1 NOTE > See > '/home/biocbuild/git/DESeq2.Rcheck/00check.log' > for details. > > all looks good now!

Martin Grigorov (09:48:59): > > --- a/R/helper.R > +++ b/R/helper.R > @@ -95,7 +95,7 @@ unmix <- function(x, pure, alpha, shift, power=1, format="matrix", quiet=FALSE) > } > res <- lapply(seq_len(ncol(x)), function(i) { > optim(par=rep(1, ncol(pure)), fn=sumLossVST, gr=NULL, i, vst, alpha, power, > - method="L-BFGS-B", lower=0, upper=100)$par > + method="L-BFGS-B", lower=1e-6, upper=100)$par > }) > } else { > stopifnot(shift > 0) >

Michael Love (10:04:29) (in thread): > somewhere in the function, a bunch of non-negative numbers are being multiplied together and added and it’s giving a sqrt(negative) error

Michael Love (10:04:50): > ok i will make this change to DESeq2 source code and note in the NEWS

Michael Love (10:08:45): > ok pushed to devel/GitHub

Martin Grigorov (10:16:29): > Awesome! Thank you!

Martin Grigorov (10:18:33) (in thread): > could it be an overflow ?!

Michael Love (10:49:10) (in thread): > maybe

2023-04-26

Vince Carey (21:49:38): > @Martin Grigorovjust wondering if you have followed the release of Bioconductor 3.17 … is the ARM linux repo up to date, and will you also have a version of the devel branch recently cut? Thanks

2023-04-27

Martin Grigorov (01:49:59) (in thread): > We have noticed the release of 3.17! Congratulations!

Martin Grigorov (01:52:15) (in thread): > The last run on Linux ARM64 ishttps://yikun.github.io/bioconductor-0327/report/We noticed that due to the R_CHECK_SUGGESTS_ONLY=true change many projects started failing also on x86_64 and decided to wait for 3.17 being released

Martin Grigorov (01:52:37) (in thread): > Now we are going to switch to 3.18/devel

Martin Grigorov (01:57:15) (in thread): > I am also going to write a short article about how to build and check a Bioc package with biconconductor/on x86_64 host withhttps://github.com/Bioconductor/bioconductor_docker/pkgs/container/bioconductorDocker ARM64 image

Martin Grigorov (01:57:45) (in thread): > This would be useful for the package maintainers to debug any potential issues

2023-05-10

Martin Grigorov (06:46:52): > Hi! We have uploaded a new report based on 3.18 athttps://yikun.github.io/bioconductor-0510/report/long-report.htmlI also wrote a short article how to build and test Bioc packages with the Docker ARM64 image on x86_64 host:https://martin-grigorov.medium.com/emulated-building-and-testing-of-bioconductor-projects-for-linux-arm64-501c1f854dc1It should be useful to package maintainers to debug issues !

Michael Love (08:26:34): > @Michael Love has left the channel

2023-05-18

Oluwafemi Oyedele (05:54:28): > @Oluwafemi Oyedele has joined the channel

2023-05-29

Martin Grigorov (14:41:38): > Hi! > > We started working on running the BBS report on openEuler 22.03 (yum/dnf based Linux distro) because this is what most of our users will actually use: > * https://github.com/martin-g/BBS/pull/1/files > * https://bio-arm.github.io/latest/report/long-report.html > There are some missing OS packages/dependencies which we will provide soon but overall everything works pretty good!

Martin Grigorov (14:48:04): > There is some interest from the community athttps://github.com/Bioconductor/BBS/pull/255#issuecomment-1554418244!:tada:I will run a new report on Ubuntu 22.04 and add a comment with a link to the latest results. > I won’t be able to answer them when/whether Linux ARM64 will become a member of the official report runners but I’d be very happy if this is in your plans for 3.18 ! - Attachment: Comment on #255 Add a Linux ARM64 builder node > I want to thank all people involved in adding support for Linux ARM64!
> In our department we are moving to ARM64 to reduce the expenses and so far everything works quite good!
> I hope to see this PR merged and Linux ARM64 being part of the official test reports soon!

2023-05-31

Hervé Pagès (11:40:39): > If you guys think you can run these builds on a daily basis, we would be happy to add them to our daily report for 3.18. I’m currently working on some adjustments to the BBS code that will allow addition of an “external builder” to the official daily builds. I’ll provide more details later.

Martin Grigorov (11:42:11) (in thread): > Yes! We can do it!

Martin Grigorov (11:42:46) (in thread): > Please ping me when you are ready to test the external builder code!

2023-06-01

Hervé Pagès (20:53:02) (in thread): > Hi@Martin Grigorov, this is ready. Seehttps://github.com/Bioconductor/BBS/issues/292 - Attachment: #292 Add Linux ARM64 builder as a node that participates to the BioC daily builds > Hi @martin-g, @Yikun > > As mentioned on our community Slack (#arm-linux channel) yesterday, your build machine will need to be configured as an “external build node” for that. This is a first for us. Yesterday and today I worked on modifying BBS to support the new “external build node” case. Make sure to pull the latest version. > > I also updated Prepare-Ubuntu-22.04-HOWTO.md to document the specificities of the “external build node” setup. This is a lighter/simpler setup than for a standalone builder. I believe that your machine is already set up as the latter so you don’t have to change anything to your current setup. However, please check sections “2.5 Add software builds to biocbuild’s crontab” and “2.6 First build report”. This will probably be new for you. > > Also your machine would need to be added to our nodes database (in BBS/nodes/), and build configuration files and scripts for it would need to be added to BBS/3.18/bioc/. More precisely, here are the changes that would need to be done to BBS: > > 1. Add your machine to our nodes database: >
> This consists in adding a new entry for it to BBS/nodes/nodespecs.py. See the entry I added at the bottom of the file for fake “external node” xps15, and add a similar entry for your machine after it. >
> You will also need to add a new subfolder in BBS/nodes/ for your machine, with a local-settings.sh
> file in it. See the BBS/nodes/xps15/ subfolder for an example (adapt the paths in local-settings.sh to reflect where things are on your machine). > 2. Add build configuration files and scripts for your machine: >
> This consists in adding a new subfolder for your machine in BBS/3.18/bioc/. See the BBS/3.18/bioc/xps15/ subfolder for an example and adapt. > > Please submit a PR whenever this is ready. > > Finally, once your machine is properly set up and was able to complete its first build run as an “external node”, we will need to discuss a way for us to retrieve the content of the ~~/bbs-3.18-bioc/products-out/ folder from it. This is where BBS will drop all the build products during the builds. We’ll need to be able to access this daily in order to include your build results in our daily report. This could typically be done via rsync i.e. think of ~~/bbs-3.18-bioc/products-out/ as something that we’d like to be able to mirror on our central builder. > > Thanks for helping Bioconductor improve its support for Linux ARM64. > > Let me know if you have questions. > > Best,
> H.

2023-06-02

Martin Grigorov (01:54:26) (in thread): > great!

Martin Grigorov (01:59:42): > I face a strange issue on the openEuler builder -https://bio-arm.github.io/latest/report/BAGS/kunpeng2-checksrc.html > > * checking re-building of vignette outputs ... OK > * checking PDF version of manual ... ERROR > Rd conversion errors: > Converting parsed Rd's to LaTeX Warning in close.connection(con) : > Problem closing connection: No space left on device > Warning in close.connection(con) : > Problem closing connection: No space left on device > > There are 750GB free disk space on/and 25GB on/tmp. Any idea what it could be ?

Martin Grigorov (02:39:29) (in thread): > many of the packages’checks fail due to this reason. > I’ve built several of them manually and all passed

Vince Carey (05:23:38) (in thread): > Is an environment variable TMPDIR defined to point to a folder where there might be a quota restriction? From the R Installation and Administration Manual: > > Various environment variables can be set to determine where R creates its per-session temporary directory. The environment variables TMPDIR, TMP and TEMP are searched in turn and the first one which is set and points to a writable area is used. If none do, the final default is /tmp on Unix-alikes and the value of R_USER on Windows. The path should be an absolute path not containing spaces28 (and it is best to avoid non-alphanumeric characters such as + or quotes). >

Martin Grigorov (05:26:19) (in thread): > Thanks!I’veexpanded/tmp to 64GB

Vince Carey (05:42:45) (in thread): > I hope it helps.@Hervé Pagèsmay have a better diagnosis.

Martin Grigorov (05:45:47) (in thread): > If itdoesn’tI’llset TMPDIR to somewhere in the disk

Martin Grigorov (10:33:02): > After updating to latest devel of BBS I get this error for prerun.sh: > > BBS> END STAGE1 loop. > BBS> ------------------------------------------------------------- > BBS> STAGE1 SUMMARY: > BBS> o Working dir: /home/biocbuild/bbs-3.18-bioc/meat > BBS> o 2195 pkg(s) listed in file meat-index.dcf > BBS> o 2195 pkg dir(s) queued and processed > BBS> o 2195 srcpkg file(s) produced > BBS> o Total time: 926.87 seconds > BBS> ------------------------------------------------------------- > BBS> START creating PACKAGES index file for target repo at Fri Jun 2 13:47:29 2023... > Error: object 'git_url' not found > Execution halted > BBS> FATAL ERROR: subprocess '/home/biocbuild/R/R-4.3.0/bin/Rscript -e "library(tools);write_PACKAGES(fields=c("git_url","git_branch","git_last_commit","git_last_commit_date"))"' returned nonzero value 1! > > Is there a problem withwritePackages?

Hervé Pagès (11:06:53) (in thread): > /tmpis part of the root partition (/) which is 218G on our Linux builders. This makes life easier. And the builds are running on a dedicated/homepartition that is 1.8T. This was expanded from 0.9T to 1.8T last year after we ran into “No space left on device” issues. > I should probably add a note herehttps://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-Ubuntu-22.04-HOWTO.md#12-check-hardware-requirementsabout/tmpsize requirement.

Andres Wokaty (13:13:23) (in thread): > @Hervé Pagèsmade a fix for it:https://github.com/Bioconductor/BBS/commit/80f9c0fd33b472e7fcfbc7744da1300437b55109.

Hervé Pagès (13:22:10) (in thread): > @Martin GrigorovBut you should not runprerun.shanymore. Onlyrun.sh. See “2.5 Add software builds to biocbuild’s crontab” inhttps://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-Ubuntu-22.04-HOWTO.md

Hervé Pagès (13:24:06) (in thread): > Alsorun.shshould be started at 3pm EST (as explained in Prepare-Ubuntu-22.04-HOWTO.md)

2023-06-05

Martin Grigorov (03:22:28) (in thread): > Thanks! I still run it as astandalonerunner to see a good report with less failures before adding it as anexternalrunner

Martin Grigorov (03:23:39) (in thread): > the last report has a regression because I usednoexecmount option when I expanded the/tmpmount. This caused errors like: > > **** testing if installed package can be loaded from temporary location > Error: package or namespace load failed for 'affyPLM' in dyn.load(file, DLLpath = DLLpath, ...): > unable to load shared object '/tmp/RtmplcQmzP/Rinst2c9e19a6e2278/00LOCK-affyPLM/00new/affyPLM/libs/affyPLM.so': > /tmp/RtmplcQmzP/Rinst2c9e19a6e2278/00LOCK-affyPLM/00new/affyPLM/libs/affyPLM.so: failed to map segment from shared object > Error: loading failed >

Martin Grigorov (03:24:01) (in thread): > I hope we can run the first report asexternaltomorrow!

Martin Grigorov (03:26:16) (in thread): > about the3PM EST: How does BBS figure out whether a report has finished ? I.e. how the master node will know when to runrsyncto fetch the output files ?

Martin Grigorov (03:33:34) (in thread): > and aboutrsync: we will need to add a static IP of the central builder to our firewall rules. Which IP(s) we should add ? > * 155.52.207.165 (nebbiolo1) > * 155.52.207.166 (nebbiolo2) > * 52.71.54.154 (master.bioconductor.org)

Martin Grigorov (07:12:50) (in thread): > I assume that nebbiolo2 will make the rsync call (https://github.com/Bioconductor/BBS/pull/293/files#diff-7c5b3923098efc8afc99952be0ab603cfda21d29718aa5b73b9e0c4a47e2239eR33)

Martin Grigorov (07:13:03) (in thread): > https://github.com/Bioconductor/BBS/pull/293

2023-06-07

Martin Grigorov (03:19:49): > Hi@Hervé Pagès! Abouthttps://github.com/Bioconductor/BBS/pull/293#issuecomment-1579708608: we see these ways: > * wget --mirrorhttp://linux-arm64-builder-ip/products-out/-> Pros: it uses plain HTTP; Cons: it is slow (1h 51mins) > * rsync -e ssh -r -P biocbuild@linux-arm64-builder-ip:/home/biocbuild/bbs-3.18-bioc/products-out .-> Pros: it is much faster than wget (here it took ~1 min to copy 524Mb from Singapore to Bulgaria). Also it is encrypted (SSH); Cons: N/A. We will need a public SSH key from you for this to work! This means you will have SSH access to the biocbuild account in case you ever need it > * rsync -e rsh ...-> Pros: no need of SSH access; Cons: it is not encrypted/secure. It requires runningrsyncdon the builder at port 873 - it will still be restricted by IP but we would prefer to not open more ports if not really needed

Hervé Pagès (12:05:25): > Plain use of modules using thersync -av host::module /destsyntax is also an option. I’d rather avoid involving SSH keys for this transfer. We don’t really care about encryption here as we don’t transfer any sensitive data. It’s all public data that will end up being displayed on the HTML report.

Martin Grigorov (15:06:22): > afaik an rsync module is just an alias to a folder, no ?

Martin Grigorov (15:06:54): > so you prefer option 3rsync -e rsh ...?

Martin Grigorov (15:17:56) (in thread): > > $ cat /etc/rsyncd.conf > # /etc/rsyncd: configuration file for rsync daemon mode > > # See rsyncd.conf man page for more options. > > # configuration example: > > # uid = nobody > # gid = nobody > # use chroot = yes > # max connections = 4 > # pid file = /var/run/rsyncd.pid > # exclude = lost+found/ > # transfer logging = yes > # timeout = 900 > # ignore nonreadable = yes > # dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2 > > # [ftp] > # path = /home/ftp > # comment = ftp export area > > [bioc] > path = /home/biocbuild/bbs-3.18-bioc/products-out/ > comment = Bioconductor report output >

Martin Grigorov (15:19:09) (in thread): > > $ rsync -av linux-arm64-ip:bioc . > opening connection using: ssh linux-arm64-ip rsync --server --sender -vvlogDtprze.iLsfxCIvu . bioc (8 args) > > Authorized users only. All activities may be monitored and reported. > martin@linux-arm64-ip's password: > > by default it uses SSH

Martin Grigorov (15:21:15) (in thread): > > rsync -e rsh -av linux-arm64-ip:bioc . > opening connection using: rsh linux-arm64-ip rsync --server --sender -vvlogDtprze.iLsfxCIvu . bioc (8 args) > > Authorized users only. All activities may be monitored and reported. > martin@linux-arm64-ip's password: > > now it uses RSH (insecure) and it asks for a password > i.e. you will need an account and you will have to send its password in plain text:confused:

Martin Grigorov (15:30:22) (in thread): > man rsyncd.confmentions anonymous access! Let me read about it!

Martin Grigorov (15:49:09) (in thread): > even when usingrsyncrsync://IP/moduleNamelocalFolderit still tried to use SSH > it could be because port 873 is filtered to me. I will ask the network admins to open this port for me and continue tomorrow

Martin Grigorov (15:53:00): > btw the run from yesterday took 19 hours, i.e. it ended at 14:24 GMT or 10:24 EST. > Today’s run wiped the contents of products-out and started populating the folder again > I see that the master starts postrun.sh at 11:00 EST, so rsync should fetch the contents between 10:24 and 11:00

Hervé Pagès (17:01:37) (in thread): > Something like that. Can’t remember exactly the difference, it’s been a long time since last time I looked into rsyncd configuration. I think defining modules allows you to mask the real path and maybe restrict what can be accessed. Anyways it doesn’t really matter from our end if the command you give us points to a module or a real path.

Hervé Pagès (17:05:52) (in thread): > We should not need an account on your machine or a password. For example, anybody can mirror CRAN withrsync -rptlzv --deletecran.r-project.org::CRAN /dir/on/local/disc. I’ve done this often from various machines. It’s very fast and doesn’t require an account on their server or any special privilege.

Hervé Pagès (17:08:46) (in thread): > That should work. Just in time for the generation of the daily report!:slightly_smiling_face:

2023-06-20

Andres Wokaty (14:26:02): > Hi@Martin GrigorovI’m updating our builders to use the new release of R 4.3.1 this week if you’d like to update kunpeng2.

Martin Grigorov (15:06:27): > Hi@Andres Wokaty! Thanks for the ping! I will update kunpeng2!

2023-06-21

Martin Grigorov (04:39:04): > one question: how do you deal with thesite-libraryfolder ? Do you copy the one from 4.3.0 ? Or create a new empty one for 4.3.1 and leave it to BBS to populate it ?

Martin Grigorov (04:40:29): > my past experience shows that leaving it to BBS leads to many failures due to missing packages which I needed to install manually. (yes, I remember@Hervé Pagèssaying that it should not be manual but …)

Martin Grigorov (05:00:26): > or maybe I should pull outR_LIBSout ofR_HOME?

Andres Wokaty (11:26:29): > We remove R and all libraries then create a new one for the BBS to repopulate. We do reinstall some packages manually to test the installation, such rJava, BiocManager, etc. as inhttps://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-Ubuntu-22.04-HOWTO.md#24-install-r. It usually takes a few days for the BBS to reinstall everything so the first report after the update isn’t great. > > OurRhome is always the same so that we don’t have to change the configuration files when we update R to a new version. I recommend doing this.

Hervé Pagès (12:29:48): > > my past experience shows that leaving it to BBS leads to many failures due to missing packages which I needed to install manually > No need to reinstall anything manually (except for testing purpose) as BBS will reinstall all the necessary deps. In your case I remember that many packages didn’t reinstall because of a slow internet connection that broke download of some big deps like the BSgenome data packages.

Martin Grigorov (14:52:17): > OK! I will leavesite-libraryempty and we will see how it goes

2023-06-23

Martin Grigorov (03:05:41): > https://bioconductor.org/checkResults/3.18/bioc-LATEST/long-report.htmlsays 1345 failed INSTALLs, against only 32 for Linux x86_64 > For examplehttps://bioconductor.org/checkResults/3.18/bioc-LATEST/abseqR/kunpeng2-install.htmlerror is:ERROR: dependencies 'VennDiagram', 'flexdashboard', 'BiocParallel', 'rmarkdown', 'knitr', 'plotly', 'BiocStyle' are not available for package 'abseqR'

Hervé Pagès (11:46:40) (in thread): > Installation ofknitrfailed with (looking atknitr.install-out.txt): > > ############################################################################## > ############################################################################## > ### > ### Running command: > ### > ### /home/biocbuild/R/R-4.3.1/bin/Rscript -e "source('/home/biocbuild/BBS/util > s/installNonTargetPkg.R');installNonTargetPkg('knitr')" > ### > ############################################################################## > ############################################################################## > > > Installing package into '/home/biocbuild/R/R-4.3.1/site-library' > (as 'lib' is unspecified) > trying URL '[https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz](https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz)' > Content type 'application/x-gzip' length 895054 bytes (874 KB) > =============================================== > downloaded 830 KB > > Error in download.file(url, destfile, method, mode = "wb", ...) : > download from '[https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz](https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz)' failed > In addition: Warning messages: > 1: In download.file(url, destfile, method, mode = "wb", ...) : > downloaded length 536835912176023587041489152849066396431755215885378601627560 > 97108221941786953991388933327197498366478368008135290086062509559865941828078127 > 73770643717048831262137586111765449802483529491827008662386243580906798949335727 > 840241334432659215154402243379200 != reported length 0 > 2: In download.file(url, destfile, method, mode = "wb", ...) : > URL '[https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz](https://cran.rstudio.com/src/contrib/knitr_1.43.tar.gz)': status was 'Fail > ure when receiving data from the peer' > Warning in download.packages(pkgs, destdir = tmpd, available = available, : > download of package 'knitr' failed > > so it’s a downloading issue. > Hundreds of Bioconductor packages depend directly or indirectly onknitr.

Martin Grigorov (14:42:30) (in thread): > let me install it manually!

Martin Grigorov (14:43:30) (in thread): > installed just fine in about ten seconds

Martin Grigorov (14:43:48) (in thread): > is there another such important package like knitr ?

Hervé Pagès (22:35:18) (in thread): > Unfortunately there are many of them. Note that the download issues on kunpeng2 seem a little bit random and don’t affect only big packages (knitris small). A quick look at the logs for the current run shows that this time 128 packages got affected: > > biocbuild@nebbiolo2:~/public_html/BBS/3.18/bioc/products-in/kunpeng2/install$ grep 'downloaded length' *.install-out.txt | wc > 128 1024 42442 >

2023-06-29

Martin Grigorov (04:07:52): > Hifails on kunpeng2 with: > > Error: C++20 standard requested but CXX20 is not defined > > but all other builders are OK! > Looking athttps://github.com/Bioconductor/BBS/blob/devel/BBS-run.py#L75-L94andhttps://github.com/Bioconductor/BBS/blob/devel/BBS-report.py#L293-L307there are definitions only up to CXX17 > Any idea why it fails only on kunpeng2 ?

Hervé Pagès (14:30:41) (in thread): > Looks like R on kunpeng2 was not configured to use C++20, probably because the compiler on the machine is too old (gcc 10.3.1 from 2020). > BTW what you see in BBS’s Python code is just what we do to generate this page:https://bioconductor.org/checkResults/3.18/bioc-LATEST/kunpeng2-NodeInfo.htmlWe don’t display info about the C++20 compiler yet.

Hervé Pagès (14:35:57) (in thread): > I’m actually surprised that some Bioconductor packages already use C++20 (NanoMethVizhasSystemRequirements: C++20in their DESCRIPTION file). I hope they have a really good reason for doing that. Want to check with them?

Martin Grigorov (14:52:41) (in thread): > according tohttps://gcc.gnu.org/projects/cxx-status.html#cxx20most of the CXX20 features are added in GCC 8 and 9

Martin Grigorov (14:53:16) (in thread): > which version of GCC is in use in nebbiolo2 (Linux x86_64) ?

Martin Grigorov (14:54:08) (in thread): > my dev machine is Ubuntu 23.04 and it provides GCC 12

Martin Grigorov (14:55:03) (in thread): > I will check with theNanoMethViz maintainers!

Hervé Pagès (14:56:03) (in thread): > nebbiolo2 has gcc 11.3.0:https://bioconductor.org/checkResults/3.18/bioc-LATEST/nebbiolo2-NodeInfo.htmlThe compiler versions for each builder are displayed by following the links in the top box of the long report:https://bioconductor.org/checkResults/3.18/bioc-LATEST/long-report.html

Martin Grigorov (14:56:50) (in thread): > https://github.com/Shians/NanoMethViz/blob/master/DESCRIPTION#L28- does this mean that it needs c++ 11 ?

Martin Grigorov (14:57:39) (in thread): > https://github.com/Shians/NanoMethViz/blob/master/DESCRIPTION#L82C26-L82C26- here it says C++17 …

Hervé Pagès (15:00:15) (in thread): > > does this mean that it needs c++ 11 > I don’t know. I’m not familiar with thecpp11package. > > here it says C++17 > That doesn’t reflect what’s in Bioconductor. This means that their GitHub repo needs to be resynced with theNanoMethVizrepo atgit.bioconductor.org

Hervé Pagès (15:00:59) (in thread): > > I will check with the NanoMethViz maintainers! > Great! Thanks

2023-07-06

Martin Grigorov (04:00:42): > Hi! Some packages are not available viaBiocManager::install(c("xyz"))and I need togit clonethem to proceed. For examplehttps://bioconductor.org/checkResults/3.18/bioc-LATEST/DESeq2/kunpeng2-buildsrc.htmlfails due to missingpasillapackage. > > > BiocManager::install("pasilla") > Bioconductor version 3.18 (BiocManager 1.30.21), R 4.3.1 (2023-06-16) > Installing package(s) 'pasilla' > Warning message: > package 'pasilla' is not available for Bioconductor version '3.18' > > A version of this package for your version of R might be available elsewhere, > see the ideas at[https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages](https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages) > > There is no build forpasillaathttps://bioconductor.org/checkResults/3.18/bioc-LATEST/but I can find it athttps://bioconductor.org/packages/3.18/data/experiment/So, to fix the build of DESeq2 I did: > > $ git clone[https://git.bioconductor.org/packages/pasilla](https://git.bioconductor.org/packages/pasilla)$ R CMD INSTALL pasilla > * installing to library '/home/biocbuild/R/R-4.3.1/site-library' > * installing **source** package 'pasilla' ... > **** using staged installation > **** data > **** inst > **** help > ***** installing help indices > **** building package indices > **** installing vignettes > **** testing if installed package can be loaded from temporary location > **** testing if installed package can be loaded from final location > **** testing if installed package keeps a record of temporary installation path > * DONE (pasilla) > > and now the build of DESeq2 also passes: > > $ R CMD build --keep-empty-dirs --no-resave-data DESeq2 > * checking for file 'DESeq2/DESCRIPTION' ... OK > * preparing 'DESeq2': > * checking DESCRIPTION meta-information ... OK > * cleaning src > * installing the package to build vignettes > * creating vignettes ... OK > * cleaning src > * checking for LF line-endings in source and make files and shell scripts > * checking for empty or unneeded directories > * building 'DESeq2_1.41.5.tar.gz' > > How is BBS supposed to deal with such packages likepasilla?

2023-07-11

Martin Grigorov (02:53:07) (in thread): > Next build should be fine -https://github.com/Shians/NanoMethViz/issues/32#issuecomment-1630084853

2023-07-18

Hervé Pagès (16:44:19): > Sorry for the late answer. Unfortunatelypasilla(data-experiment package) is not available because it depends onDEXSeq(software package) which is not available in BioC 3.18, and the reason for this is that it’s broken on Linux (and it has been since the beginning of the 3.18 builds so has never propagated): > * https://bioconductor.org/checkResults/3.18/data-experiment-LATEST/pasilla/ > * https://bioconductor.org/checkResults/3.18/bioc-LATEST/DEXSeq/nebbiolo2-checksrc.html > * https://bioconductor.org/packages/3.18/DEXSeq > This means that BBS cannot install packages that depend directly or indirectly onpasilla. BBS won’t do anything special to deal with this situation, it just reports the problem. The hope is that developers will fix important problems like this that create a bottleneck for several other packages.

2023-07-21

Martin Grigorov (07:23:02): > Hi! What could be the problem here:https://bioconductor.org/checkResults/3.18/bioc-LATEST/AffyRNADegradation/kunpeng2-checksrc.html?

Martin Grigorov (07:23:17) (in thread): > it fails with: > > ERROR: dependencies 'plogr', 'cpp11' are not available for package 'RSQLite' > * removing '/home/biocbuild/tmp/Rtmp4mfFU6/RLIBS_208f8b346d818f/RSQLite' > Warning in install.packages(cdfname, lib = lib, repos = repositories(), : > installation of package 'RSQLite' had non-zero exit status > ERROR: dependency 'RSQLite' is not available for package 'AnnotationDbi' > * removing '/home/biocbuild/tmp/Rtmp4mfFU6/RLIBS_208f8b346d818f/AnnotationDbi' >

Martin Grigorov (07:23:45) (in thread): > but those are actually installed: > > BiocManager::install("plogr") > Bioconductor version 3.18 (BiocManager 1.30.21.1), R 4.3.1 (2023-06-16) > Old packages: 'PMA', 'seriation' > Update all/some/none? [a/s/n]: n > Warning message: > package(s) not installed when version(s) same as or greater than current; use > `force = TRUE` to re-install: 'plogr' > > BiocManager::install("cpp11") > Bioconductor version 3.18 (BiocManager 1.30.21.1), R 4.3.1 (2023-06-16) > Old packages: 'PMA', 'seriation' > Update all/some/none? [a/s/n]: n > Warning message: > package(s) not installed when version(s) same as or greater than current; use > `force = TRUE` to re-install: 'cpp11' > > BiocManager::install("RSQLite") > Bioconductor version 3.18 (BiocManager 1.30.21.1), R 4.3.1 (2023-06-16) > Old packages: 'PMA', 'seriation' > Update all/some/none? [a/s/n]: n > Warning message: > package(s) not installed when version(s) same as or greater than current; use > `force = TRUE` to re-install: 'RSQLite' >

Martin Grigorov (07:56:34): > Which OS package should provideRNAfoldcommand ?https://bioconductor.org/checkResults/3.18/bioc-LATEST/GeneGA/kunpeng2-buildsrc.html

Martin Grigorov (07:57:48) (in thread): > Ubuntu 23.04 reports: > > $ RNAfold > Command 'RNAfold' not found, but can be installed with: > sudo apt install vienna-rna > > butgrep -rnH vienna BBS/Ubuntu-files/returns nothing, so I guess it could be something else

Vince Carey (08:19:47) (in thread): > @Martin Grigorovthis looks like a failure of RSQLite to declare these dependencies in DESCRIPTION.

Martin Grigorov (08:22:03) (in thread): > https://github.com/r-dbi/RSQLite/blob/main/DESCRIPTION#L55-L57

Vince Carey (08:22:15) (in thread): > Apropos cpp11 it is in the LinkingTo field. But maybe it has to be in Suggests or Imports too?

Vince Carey (08:22:28) (in thread): > @Hervé Pagèswould know

Martin Grigorov (08:24:53) (in thread): > OK! I will wait for his opinion! plogr:0.2.0 and cpp11:0.4.5 are the installed versions

Vince Carey (08:27:53) (in thread): > @Andres Wokatywould know if we have that. I see crisprScore has a mention of RNAfold binary but I am not sure it is necessary for package to pass check. (I usedcode.bioconductor.orgto find mention of RNAfold…)

Andres Wokaty (09:40:07) (in thread): > We have some packages that require special installation, like ViennaRNA which you can find athttps://www.tbi.univie.ac.at/RNA. For reference, these are the special installation packages for ubuntu:https://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-Ubuntu-22.04-HOWTO.md#3-install-additional-stuff-for-b[…]ctor-packages-with-special-needs.

Martin Grigorov (10:01:18) (in thread): > Thank you,@Andres Wokaty!

2023-07-24

Vince Carey (12:21:57): > Hi@Martin Grigorovthe ontoProc failure on kungpeng2 is pretty abrupt. Do you have any clues?

Martin Grigorov (12:23:47) (in thread): > Hi! I am on vacation until Thursday.Iwon’tbe able to take a look before that

Vince Carey (12:31:18) (in thread): > No problem. Have a great break!

2023-07-25

Hervé Pagès (16:40:43) (in thread): > Note thatRSQLiteis a CRAN package so we have no control on it. The package hascpp11in itsLinkingTofield and there’s no reason why it should also have it in itsSuggestsfield. So it’s doing the right thing. Unfortuntately this seems to cause problem with our use of*R_CHECK_SUGGESTS_ONLY*=trueon the Linux builders.*R_CHECK_SUGGESTS_ONLY*is documented here:https://cran.r-project.org/doc/manuals/r-release/R-ints.html#Tools: > > If set to a true value, running examples, tests and vignettes is done with a temporary library directory populated by all the Depends/Imports/Suggests packages. (As exceptions, packages in a ‘VignetteBuilder’ field are always made available.) Default: false (but true for CRAN submission checks: some of the regular checks use true and some use false). > I’d say the temporary library directory should be populated with all the Depends/Imports/LinkingTo/Suggests packages. It doesn’t make much sense to not include packages listed in the LinkingTo field. So to me*R_CHECK_SUGGESTS_ONLY*is conceptually flawed at the moment. Maybe using this setting for our builds is not such a good idea after all:disappointed:@Henrik BengtssonAny idea?

Henrik Bengtsson (16:40:55): > @Henrik Bengtsson has joined the channel

2023-07-26

Hervé Pagès (01:18:16) (in thread): > @Andres WokatyNice to see that RNAfold is now part of Ubuntu. We should probably update our doc to use this instead of the standalone .deb package provided by the ViennaRNA folks.

2023-07-27

Martin Grigorov (04:29:40) (in thread): > I justR CMD INSTALL,R CMD buildandR CMD check-ed ontoProc and all passed successfully (on kunpeng2 machine). I will try again few more times !

2023-07-28

Martin Grigorov (01:53:13) (in thread): > and the report is green too -https://bioconductor.org/checkResults/3.18/bioc-LATEST/ontoProc/kunpeng2-buildsrc.html

Vince Carey (15:10:16) (in thread): > Thank you. Probably an annotation hub event.

2023-08-01

Andres Wokaty (10:48:11) (in thread): > I madehttps://github.com/Bioconductor/BBS/issues/313

2023-08-16

Mervin Fansler (17:56:20): > @Mervin Fansler has joined the channel

2023-09-22

Hervé Pagès (01:12:41): > Hi@Martin Grigorov, the following code is failing on kunpeng2: > > library(ExperimentHub) > ehub <- ExperimentHub() > h5ls(ehub[["EH1040"]]) > > with the following error: > > Error in H5Fopen(file, flags = flags, fapl = fapl, native = native) : > HDF5. File accessibility. Unable to open file. > > (See herehttps://bioconductor.org/checkResults/3.18/bioc-LATEST/HDF5Array/kunpeng2-checksrc.html) > Resource EH1040 is a big HDF5 file (4.8G) and I wonder if maybe the file got corrupted/truncated when downloaded to the cache. Do you think you can remove it from the cache withremoveResources(ehub, "EH1040")? Although it could be that it will get corrupted/truncated again during the next download if kunpeng2’s internet connectivity problems are persistent. Thanks in advance!

Martin Grigorov (01:55:16) (in thread): > I’lltake a look! Thanks@Hervé Pagès!

Martin Grigorov (02:47:35) (in thread): > I’ll do it at the weekend. The system is very busy now running the daily tests ….

Martin Grigorov (02:50:13) (in thread): > I cannot even executeehub <- ExperimentHub(). It looks like it just hangs …

2023-09-24

Martin Grigorov (15:45:45) (in thread): > @Hervé Pagèsthe checks of HDF5Array passed now: > > GNU make is a SystemRequirements. > * checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK > * checking use of PKG_*FLAGS in Makefiles ... OK > * checking compiled code ... OK > * checking examples ... OK > * checking for unstated dependencies in 'tests' ... OK > * checking tests ... > Running 'run_unitTests.R' > OK > * checking PDF version of manual ... OK > * DONE > > Status: 4 NOTEs > See > '/home/biocbuild/git/HDF5Array.Rcheck/00check.log' > > It should be OK in the next report!

2023-09-25

Hervé Pagès (02:14:27) (in thread): > Thanks@Martin Grigorov. Let’s see what happens in Monday’s report.

Martin Grigorov (02:37:54) (in thread): > Thank you for your help here -https://github.com/vladchimescu/lpsymphony/pull/9#issuecomment-1730820556! I hope they react before the 3.18 release !

Martin Grigorov (02:39:28) (in thread): > I’ve opened several more PRs for other projects and mentioned them atsupport.bioconductor.orgbut just few of them have responded so far…

Hervé Pagès (14:04:14) (in thread): > HDF5Arraypasses CHECK on kunpeng2 in today’s report:tada:Thanks@Martin Grigorov > > I’ve opened several more PRs for other projects… > Let me know if you want me to chime in there too and I’ll be glad to do so. You can post the links here if you want.

Martin Grigorov (14:50:14) (in thread): > Thanks! I’ll prepare the list tomorrow!

2023-09-26

Martin Grigorov (09:54:50) (in thread): > Hi@Hervé Pagès! Here is the list of PRs and issues: > PRs:https://github.com/DAIZHENWEI/BDMMAcorrect/pull/3https://github.com/biobenkj/compartmap/pull/26https://github.com/RemyNicolle/CoRegNet-bioc/pull/1https://github.com/christpa/DMRScan/pull/15https://github.com/harrisgm/GCSscore/pull/1https://github.com/rli012/GDCRNATools/pull/24https://github.com/LihuaJulieZhu/GUIDEseq/pull/5https://support.bioconductor.org/p/9153654/(I was not able to find a personal Github repo)https://github.com/Jiefei-Wang/Travel/pull/2https://github.com/PengfanZhang/Rbec/pull/5Issues:https://github.com/FertigLab/CancerInSilico/issues/4https://github.com/HenrikBengtsson/affxparser/issues/46

Hervé Pagès (14:09:28) (in thread): > Thanks! I’ll try to go after them in the next couple of days. Some of them are already candidates for deprecation (e.g.DMRScanandTravel, seehttps://support.bioconductor.org/p/9152979/) so I’ll leave them alone.

2023-09-27

Hervé Pagès (01:48:47) (in thread): > Done:white_check_mark:

Martin Grigorov (02:44:03) (in thread): > Thanks!

Martin Grigorov (02:45:16): > https://github.com/christpa/DMRScan/pull/15#issuecomment-1736795809It appears thatkunpeng2uses a newer Pandoc (v3.1.2) while all other builders use v2.7.3. > Should I downgrade Pandoc inkunpeng2or the other builders will upgrade it soon (e.g. for 3.19) ? - Attachment: Comment on #15 Update DMRScan_vignette.Rmd YAML to not use leading spaces > > Pandoc seems to be involved. Could this be because the various Bioconductor build machines use different versions of Pandoc? > > Yes! This might be the reason!
> All build nodes but kunpeng2 use Pandoc 2.7.3, while kunpeng2 uses Pandoc 3.1.2!

Andres Wokaty (09:50:32) (in thread): > There’s no immediate plan to upgrade. Maybe downgrade if possible, especially if it’s causing a lot of errors.

Martin Grigorov (10:36:25) (in thread): > Okay!I will downgrade it!Thanks!

2023-09-29

Hervé Pagès (15:01:50): > @Martin GrigorovWe seem to have an unusual high number (369) ofR CMD checkerrors on today’s report for kunpeng2. The following error in particular seems to affect a significant number of packages (e.g.bandle,bsseq,cBioPortalData, and many other packages): > > * checking re-building of vignette outputs ... ERROR > Error(s) in re-building vignettes: > ... > > (Yes, this is a very unhelpful error message:disappointed:) > Has something changed on the machine that could be causing this? Any thoughts on that? Thanks

2023-10-02

Martin Grigorov (03:16:16) (in thread): > I didn’t change anything! I didn’t have time even for downgrading Pandoc. > I will investigate !

Martin Grigorov (03:22:02) (in thread): > 2.12 is the earliest that provides anaarch64binaries -https://github.com/jgm/pandoc/releases?page=4Hopefully it behaves similar to 2.7.3 !

Martin Grigorov (04:29:40) (in thread): > bandle’s build and check are fine today!

Hervé Pagès (13:52:18) (in thread): > Thank you. All those errors seem to be gone today. So it looks like this was just a big kunpeng2 hiccup.

Martin Grigorov (14:51:08) (in thread): > Yes, it looks like

2023-10-05

Martin Grigorov (02:17:41) (in thread): > kunpeng2now uses 2.12 andhttps://bioconductor.org/checkResults/3.18/bioc-LATEST/DMRScan/is green!

2023-10-18

Odigiri Great Alume (13:20:27): > @Odigiri Great Alume has joined the channel

2023-10-19

Hervé Pagès (00:00:14): > @Martin GrigorovIt seems that kunpeng2 can no longer finish the builds in time:https://bioconductor.org/checkResults/3.18/bioc-LATEST/kunpeng2-index.htmlSee NAs in the CHECK column towards the end of the report. > Probably a consequence of decreasingBBS_CHECK_NB_CPUfrom 25 to 20https://github.com/Bioconductor/BBS/pull/341Do you want to try to increase it to 22? (21 might not be enough). Unfortunately this might increase slightly the number of TIMEOUTs we see on the machine, which is already quite high.

Martin Grigorov (02:17:06) (in thread): > Let’s try it!

Martin Grigorov (02:33:18) (in thread): > Changed it to22! > > --- a/3.18/bioc/kunpeng2/config.sh > +++ b/3.18/bioc/kunpeng2/config.sh > @@ -14,7 +14,7 @@ export BBS_R_HOME="/home/biocbuild/R/R-4.3.1" > export R_LIBS="$BBS_R_HOME/site-library" > export BBS_NB_CPU=25 # 32 cores are available > export BBS_BUILD_NB_CPU=16 # 32 cores are available > -export BBS_CHECK_NB_CPU=20 # 32 cores are available > +export BBS_CHECK_NB_CPU=22 # 32 cores are available > > Since the current report is already running I guess we will see the effect in the next one.

Hervé Pagès (11:53:05) (in thread): > Thanks!

Andres Wokaty (13:22:02): > @Martin GrigorovI apologize for not coordinating better, but I’m going to switch to the new RELEASE_3_18 branch. I mergedhttps://github.com/Bioconductor/BBS/pull/348so you’ll just need togit pullon kungpeng2. - Attachment: #348 Initial change from devel to RELEASE_3_18 > Close #344

Martin Grigorov (13:22:47) (in thread): > Yes, I noticed!Thanks!

Martin Grigorov (13:23:31) (in thread): > There is a cron job that pulls latest devel before starting the build

Andres Wokaty (13:25:25) (in thread): > I don’t think you need to make any other changes other than to pull these changes as the configuration files will now tell it to get the new branch.

Martin Grigorov (13:31:31) (in thread): > Hm.I may have to do something!Currently there are two cron jobs - one to pull devel and another to start the build for devel. Once 3.18 is released I will have to add a~~~third cron job ~~~new build machine for 3.18!

Andres Wokaty (13:34:31) (in thread): > Let me see if I can understand kunpeng’s set up a little better.

Andres Wokaty (13:42:15) (in thread): > From looking at the documentation, it doesn’t appear to me that you need to do anything other than update the BBS code. Can you confirm,@Hervé Pagès?@Martin GrigorovAre you thinking of adding a build machine for 3.19, which will be the new devel? I’m trying to understand your last comment.

Martin Grigorov (13:51:17) (in thread): > I have to confirm with my manager but I think it would be good to have two machines - one for “release” and one for “devel”

Martin Grigorov (13:53:18) (in thread): > I can request a more powerful machine, e.g. 64/96 cores and run both branches on it too

Hervé Pagès (14:30:03) (in thread): > The switch to RELEASE_3_18 doesn’t affect the secondary nodes, only the central node (nebbiolo2 in this case) so you shouldn’t need to pull anything@Martin Grigorov

Hervé Pagès (14:40:29) (in thread): > Getting a more powerful machine would be great@Martin GrigorovNote that having Linux arm64 builds for BioC devel is more important IMO than for BioC release. So in case we only have one Linux arm64 builder, it would make more sense to use it for the 3.19 builds (upcoming devel) than for the 3.18 builds (upcoming release). This means that if we stick with kunpeng2 only, then it would make sense to switch it to build 3.19 at some point after the 3.18 release. > If kunpeng2 can be replaced with a more powerful machine, even better! > Thanks for helping us improve Bioconductor support for Linux arm64.

Martin Grigorov (14:41:48) (in thread): > I will discuss it tomorrow!

2023-10-28

Hervé Pagès (05:44:56): > @Martin GrigorovWe’ve made a small change in the BBS code that should slightly speed up the CHECK stage on kunpeng2. The change is to runR CMD checkwith--no-vignetteslike we do on the Windows and Mac builders (because they’re not very powerful either). Can you pleasegit pullthe BBS clone on kunpeng2? Thanks

Martin Grigorov (05:46:37) (in thread): > I will do it at Wednesday

Hervé Pagès (14:44:28) (in thread): > Thx

2023-11-01

Martin Grigorov (12:43:29) (in thread): > Hi! I discussed the topic with my management and it has been decided to usekunpeng2fordevel. Should I copy the 3.18 config tohttps://github.com/Bioconductor/BBS/tree/devel/3.19/bioc?

Martin Grigorov (12:44:47) (in thread): > it seems currentlykunpeng2is not used for neither 3.18 (https://bioconductor.org/checkResults/3.18/bioc-LATEST/long-report.html) nor for 3.19 (https://bioconductor.org/checkResults/3.19/bioc-LATEST/long-report.html)

Andres Wokaty (12:46:03) (in thread): > it was removed from the report on the release day because it showed a lot of NAs.@Hervé Pagèsmade an update so that maybe it would improve kunpeng2’s report to reduce the NAs recently.

Martin Grigorov (12:46:41) (in thread): > yes! this is the other thread about--no-vignettes

Martin Grigorov (12:47:06) (in thread): > I just pulled the latest BBS onkunpeng2

Martin Grigorov (12:47:15) (in thread): > Done!

Andres Wokaty (12:47:20) (in thread): > yes, so after you pull that we can add it back in; however, I can wait and you can switch to 3.19 and we can add it there?

Hervé Pagès (12:47:32) (in thread): > Thanks!

Martin Grigorov (12:47:57) (in thread): > OK! Let me open a PR for 3.19 ! Should I change anything in the config files ?

Andres Wokaty (12:48:20) (in thread): > let me look at the config file first.

Martin Grigorov (12:49:30) (in thread): > export BBS_WORK_TOPDIR="/home/biocbuild/bbs-3.18-bioc"is a candidate

Hervé Pagès (12:49:33) (in thread): > Can we have at least one more 3.18 run with--no-vignettesfirst? That way we’ll be able to compare the run time with or w/o--no-vignettes. Thanks

Andres Wokaty (12:52:17) (in thread): > Ok, let’s do as Herve suggests. > > Afterward, we can switch to 3.19. In addition to changingBBS_WORK_TOPDIR, you should also changeexport BBS_CENTRAL_ROOT_URL="http://155.52.207.165"andexport BBS_R_HOME. > > 3.19 uses R 4.4.

Martin Grigorov (12:53:38) (in thread): > .165isnebbiolo1, right ?

Andres Wokaty (12:54:01) (in thread): > yes

Martin Grigorov (12:54:10) (in thread): > OK! 3.18 is good to go at kunpeng2 !

Martin Grigorov (12:54:26) (in thread): > I will install R 4.4 tomorrow and open a PR for 3.19 !

Martin Grigorov (12:55:00) (in thread): > the switch from 3.18 to 3.19 is completely on your side, right ? I have nothing to do there ?!

Andres Wokaty (12:55:59) (in thread): > update your cron jobs to run 3.19

Martin Grigorov (12:56:10) (in thread): > right! just realized!

Martin Grigorov (12:56:35) (in thread): > OK! I will prepare the PR for 3.19 and I will wait for you to tell me when to update the cron jobs !

Hervé Pagès (12:56:38) (in thread): > Nothing to do there. You just need to copyBBS/3.18/bioc/kunpeng2toBBS/3.19/bioc/kunpeng2and make the changes that Jen mentioned above.

Hervé Pagès (12:57:16) (in thread): > Oh yes, and edit the crontab. Thanks@Andres Wokaty!

Andres Wokaty (13:10:04) (in thread): > @Martin GrigorovCan you also removemaptoolsandrgeosif they’re still installed? They were removed from CRAN last month:https://github.com/Bioconductor/BBS/issues/300. - Attachment: #300 Remove rgeos and maptools from builders > CRAN will archive rgeos, maptools, rgdal October 2023: https://stat.ethz.ch/pipermail/r-packages/2023/001825.html. The following packages import or declared one of these as a dependency and were initially contacted June 2023: > > • GEOExplorer > • GateFinder > • flowDensity > > As of June 2023, rgdal isn’t installed on the builders; however, rgeos and maptools are, but they should be removed before the release.

Hervé Pagès (13:11:26) (in thread): > Starting with a fresh R 4.4 install should take care of that.

2023-11-02

Martin Grigorov (09:39:22) (in thread): > I guess I have to installhttps://cran.r-project.org/src/base-prerelease/R-devel.tar.gzfor 4.4 ? How often do I need to update it ?

Andres Wokaty (10:01:42) (in thread): > Yes, we’ll update periodically. Let’s try to keep the updates together and I’ll try to give advanced notice when we need to update R.

Martin Grigorov (10:02:06) (in thread): > :thumbsup:

Andres Wokaty (14:54:29) (in thread): > Let me know when I should remove kunpeng2 from the release report and add it to the devel report. Thank you!

Martin Grigorov (14:57:12) (in thread): > https://bioconductor.org/checkResults/3.18/bioc-LATEST/long-report.htmlhas many timeouts in the BUILD phase

Martin Grigorov (14:59:06) (in thread): > OK! I will update the cron job now and if we have similar results for 3.19 I will ask for more powerful machine

Martin Grigorov (15:00:54) (in thread): > done!

Andres Wokaty (15:02:12) (in thread): > ok, i will add kunpeng2 to the 3.19 report and remove it from 3.18. (I know it just started but i am going to try to fix it for the report tomorrow.)

Martin Grigorov (15:03:09) (in thread): > but usually with a new R installation I expect a lot of errors due to missing dependencies. I install them manually:confused:For some reason the transitive dependencies are not automatically installed …

Andres Wokaty (15:10:18) (in thread): > even after a few days you still need to install them manually?

Martin Grigorov (15:10:48) (in thread): > I am not sure how long I waited last time

Andres Wokaty (15:14:01) (in thread): > @Hervé PagèsI want to confirm that I just need to add kunpeng2 to the 3.19 report and add the cronjob to fetch the report artifacts?

Hervé Pagès (18:54:35) (in thread): > Hmm… I didn’t know you could also get these tarballs from the prerelease folder. This used to be for the tarballs that the CRAN folks would make available well… during prerelease times. The rest of the time we normally grab them from the daily snaphot folder here:https://stat.ethz.ch/R/daily/I’ve not checked this for a long time so they must have change this. Or this is unintentional, and I would not rely too much on the latest daily snaphots being also available under the prerelease folder. What’s even more intriguing is that theR-devel_2023-11-01_r85459.tar.gztarball from the prerelease folder and theR-devel_2023-11-01.tar.gztarball from the daily snaphots don’t have exactly the same content, despite both being from svn revision 85459 (but I checked and the differences are only indoc/html/R-admin.htmland very minor, so not a big deal). > > For some reason the transitive dependencies are not automatically installed … > If I recall correctly, the installation failures that we’ve seen so far on kunpeng2 were usually the consequences of some big packages (typically data packages) not getting properly downloaded and ending up corrupted. This is very likely related to a problem with the quality of kunpeng2’s connection to the internet. > > I want to confirm that I just need to add kunpeng2 to the 3.19 report and add the cronjob to fetch the report artifacts? > Yep. You might want to make sure kunpeng2 has finished (by checking~/public_html/BBS/3.19/bioc/products-in/kunpeng2/on nebbiolo1) before adding it to the report. Since this is its first run with R 4.4, it might take significantly longer.

Hervé Pagès (19:03:59) (in thread): > @Martin GrigorovPlease make sure that rsync module114.119.187.96::biocpoints to the right place (typically/home/biocbuild/bbs-3.19-bioc/products-out). This might require updating your rsync daemon configuration.

Hervé Pagès (19:14:42) (in thread): > @Andres WokatyAfter Martin has addressed the above, check the rsync command manually and make sure it does the right thing before adding it to the crontab. A good check is to make sure that~/public_html/BBS/3.19/bioc/products-in/kunpeng2/install/PACKAGESand~/public_html/BBS/3.19/bioc/src/contrib/PACKAGESare identical.

Hervé Pagès (19:23:44) (in thread): > One more thing@Martin Grigorov, is kunpeng2 still running openEuler 22.03 LTS SP1 or have you applied SP2 to it already? Thx

Andres Wokaty (20:44:15) (in thread): > It’s frustrating that there are differences in the 2 tarballs. I just checked the version numbers (assuming the rest was the same). Thanks for guidance.

Hervé Pagès (21:24:30) (in thread): > It is. And confusing. I suspect it’s not even expected that the prerelease folder gets populated with the daily snapshots. Maybe a leftover cron job from the last R release that should have been stopped after the release or something like that, idk.

2023-11-03

Martin Grigorov (07:26:15) (in thread): > @Hervé PagèsRsync and HTTPD configs are updated to point to 3.19 and the services are restarted!

Martin Grigorov (07:33:23) (in thread): > it is still SP1. Let’s see whether we will keep the VM for 3.19 or we will use a new bigger one. If we use a bigger one then I will have to configure it from scratch and I will install SP2, otherwise I will update the current one to SP2 some weekend when the build is not running

Martin Grigorov (07:51:02) (in thread): > > BBS> o Working dir: /home/biocbuild/bbs-3.19-bioc/meat > BBS> o 1484 srcpkg file(s) in working dir > BBS> o 1484 srcpkg file(s) queued and processed > BBS> o Total time: 14111.01 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Fri Nov 3 11:29:01 2023. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.19-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. > > the build ended much earlier than usual! I guess due to the missing dependencies

Martin Grigorov (08:32:54) (in thread): > here is a list of the missing packages. Should I install them manually or should I leave it to the next run ? - File (Plain Text): missing-pkgs.txt

Andres Wokaty (10:29:08) (in thread): > Can you leave it for the next run so we can compare if some of them got installed?

Martin Grigorov (10:34:33) (in thread): > Sure!

Andres Wokaty (16:20:24) (in thread): > I can’t seem to rsync from nebbiolo1. I’m following up with@Hervé Pagèsto see if we missed anything.

Andres Wokaty (16:51:27) (in thread): > @Martin GrigorovI’m using what was in our crontab for nebbiolo2 to rsync: > > /usr/bin/rsync -azv 114.119.187.96::bioc /home/biocbuild/public_html/BBS/3.19/bioc/products-in/kunpeng2 > > However, I get the following > > rsync error: error in socket IO (code 10) at clientserver.c(139) [Receiver=3.2.7] > rsync: [Receiver] failed to connect to 114.119.187.96 (114.119.187.96): Connection timed out (110) >

2023-11-07

Martin Grigorov (07:13:38) (in thread): > I think this is due to a firewall rule ! Let me ask the network admins to add netbiolo1 to the allowed IPs !

Martin Grigorov (07:15:04) (in thread): > @Andres WokatyYou make this request from155.52.207.165, right ?

Martin Grigorov (07:16:30) (in thread): > Here is the list of missing packages for today. At Nov 5th they were155and today they are61! - File (Plain Text): 23.11.07-missing-pkgs.txt

Martin Grigorov (07:54:11) (in thread): > @Andres WokatyThe firewall rule has been added

Andres Wokaty (09:20:21) (in thread): > The IP is correct. Thanks!

Andres Wokaty (09:32:47) (in thread): > Unfortunately, I’m still getting a failed to connect error.

Martin Grigorov (09:33:31) (in thread): > please trynmap -p 873 114.119.187.96?

Martin Grigorov (09:39:19) (in thread): > > $ telnet 114.119.187.96 873 > Trying 114.119.187.96... > Connected to 114.119.187.96. > Escape character is '^]'. > @RSYNCD: 31.0 > ^] > telnet> Connection closed. >

Andres Wokaty (09:39:31) (in thread): > > nmap -p 873 114.119.187.96 > Starting Nmap 7.80 ([https://nmap.org](https://nmap.org)) at 2023-11-07 09:36 EST > Nmap scan report for ecs-114-119-187-96.compute.hwclouds-dns.com (114.119.187.96) > Host is up (0.00081s latency). > > PORT STATE SERVICE > 873/tcp filtered rsync > > Nmap done: 1 IP address (1 host up) scanned in 2.15 seconds >

Martin Grigorov (09:40:04) (in thread): > and you do this from 155.52.207.165 ?

Andres Wokaty (09:40:13) (in thread): > yes

Martin Grigorov (09:40:29) (in thread): > OK! I’ll ask again the admins to double check

Andres Wokaty (09:42:24) (in thread): > I can’t seem to connect via telnet; it’s hanging on ‘trying’

Martin Grigorov (10:17:10) (in thread): > The network admin sent me a screenshot and it looks OK. 155.52.207.165 must be allowed

Andres Wokaty (12:23:05) (in thread): > i’m still having some trouble with it and will likely reach out to IT.

Hervé Pagès (12:48:17) (in thread): > FWIW I just tried the rsync command from nebbiolo2 (155.52.207.166) and it works fine so it really seems like a problem with nebbiolo1 being blocked somehow.

Andres Wokaty (12:49:12) (in thread): > yes, i tried rsync, telnet, and nmap on nebbiolo2, which all worked fine.

Andres Wokaty (12:49:31) (in thread): > I’ll reach out to Nikos.

Martin Grigorov (12:52:01) (in thread): > Do you know what is 155.52.47.146 ?

Martin Grigorov (12:52:59) (in thread): > It is also allowed andits comments say ‘bioconductor-rsync’

Andres Wokaty (12:54:45) (in thread): > maybe the ip is wrong? that should be nebbiolo2.

Hervé Pagès (12:55:02) (in thread): > oh, that’s also nebbiolo2, got it! You need to allow 155.52.47.135 for nebbiolo1, sorry for the confusion (these machines have 2 IPs)

Hervé Pagès (12:55:44) (in thread): > one internal and one external IP each

Andres Wokaty (12:55:56) (in thread): > this must be it!

Martin Grigorov (12:56:32) (in thread): > Ok! It will be added after 5-6 hours, when China wakes up

Andres Wokaty (12:56:56) (in thread): > thank you!

Martin Grigorov (13:21:28) (in thread): > It seems Chinadoesn’tsleep:blush:Please try again!

Hervé Pagès (13:24:37) (in thread): > Thx Martin, works for me on nebbiolo1.@Andres Wokaty?

Andres Wokaty (13:27:15) (in thread): > looking good! thank you:slightly_smiling_face:

Martin Grigorov (13:27:30) (in thread): > Great!

Hervé Pagès (15:20:37) (in thread): > @Andres WokatyThis means the rsync command can now be enabled in the crontab on nebbiolo1.

2023-11-08

Hervé Pagès (14:33:06): > @Martin GrigorovLooking at the build logs for kunpeng2, we see many INSTALL errors (213). 35 of them are caused by a download error with an error message that says “downloaded length 5368359121760235870414891528490663964317552158853786016275609710822194178695399138893332719749836647836800813529008606250955986594182807812773770643717048831262137586111765449802483529491827008662386243580906798949335727840241334432659215154402243379200 != reported length 0”. Yes this is the crazy “downloaded length” number that we see! Even more intriguing, we see this exact same number for the 35 download errors! Very very strange. > FWIW, here is the full list of packages that have this error: > {brgedata} > BSgenome.Drerio.UCSC.danRer7 > BSgenome.Ggallus.UCSC.galGal3 > BSgenome.Hsapiens.1000genomes.hs37d5 > BSgenome.Hsapiens.NCBI.GRCh38 > BSgenome.Hsapiens.UCSC.hg18 > BSgenome.Mmulatta.UCSC.rheMac10 > BSgenome.Mmusculus.UCSC.mm10 > BSgenome.Mmusculus.UCSC.mm8 > BSgenome.Mmusculus.UCSC.mm9 > BSgenome.Mmusculus.UCSC.mm9.masked > BSgenome.Rnorvegicus.UCSC.rn4 > BSgenome.Rnorvegicus.UCSC.rn5 > BSgenome.Rnorvegicus.UCSC.rn6 > BSgenome.Rnorvegicus.UCSC.rn7 > ccdata > ChIPXpressData > curatedOvarianData > KEGGdzPathwaysGEO > metaMSdata > minionSummaryData > MMDiffBamSubset > msdata > msPurityData > mtbls2 > oligoData > pd.genomewidesnp.5 > reactome.db > RGMQLlib > RNAseqData.HNRNPC.bam.chr14 > SNPlocs.Hsapiens.dbSNP155.GRCh37 > SNPlocs.Hsapiens.dbSNP155.GRCh38 > tximportData > XtraSNPlocs.Hsapiens.dbSNP144.GRCh37 > yeastRNASeq >

Martin Grigorov (14:52:46) (in thread): > my only solution for this problem is to download them manually

Hervé Pagès (15:01:18) (in thread): > An important question is whether this problem is caused by kunpeng2’s flaky connection to the internet or if there is something deeper going on with R’sdownload.file()function on openEuler. If the latter, then openEuler users are also going to struggle to install packages. > How do you manually download the packages? Have you tried to usedownload.file()for that? Would be good to be able to reproduce this error at the R command line.

Martin Grigorov (15:07:49) (in thread): > last time I used BiocManager::install() for the data packages

Martin Grigorov (15:08:14) (in thread): > andR CMD INSTALL ...for the non-data

Martin Grigorov (15:09:32) (in thread): > but I have never seen such long numbers

Hervé Pagès (16:01:19) (in thread): > so you see these errors when you try to install data packages withBiocManager::install()?

2023-11-09

Martin Grigorov (02:14:15) (in thread): > brgedatais already installed > > > BiocManager::install(c("brgedata"), type = "source", checkBuilt = TRUE) > Bioconductor version 3.19 (BiocManager 1.30.22), R Under development (unstable) > (2023-11-01 r85459) > Warning message: > package(s) not installed when version(s) same as or greater than current; use > `force = TRUE` to re-install: 'brgedata' > > > > >

Martin Grigorov (02:14:56) (in thread): > > BSgenome.Drerio.UCSC.danRer7 > > installs fine: > > > BiocManager::install(c("BSgenome.Drerio.UCSC.danRer7"), type = "source", checkBuilt = TRUE) > Bioconductor version 3.19 (BiocManager 1.30.22), R Under development (unstable) > (2023-11-01 r85459) > Installing package(s) 'BSgenome.Drerio.UCSC.danRer7' > trying URL '[https://bioconductor.org/packages/3.19/data/annotation/src/contrib/BSgenome.Drerio.UCSC.danRer7_1.4.0.tar.gz](https://bioconductor.org/packages/3.19/data/annotation/src/contrib/BSgenome.Drerio.UCSC.danRer7_1.4.0.tar.gz)' > Content type 'application/x-gzip' length 330303588 bytes (315.0 MB) > ================================================== > downloaded 315.0 MB > > * installing **source** package 'BSgenome.Drerio.UCSC.danRer7' ... > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > **** help > ***** installing help indices > **** building package indices > **** testing if installed package can be loaded from temporary location > **** testing if installed package can be loaded from final location > **** testing if installed package keeps a record of temporary installation path > * DONE (BSgenome.Drerio.UCSC.danRer7) > > The downloaded source packages are in > '/home/biocbuild/tmp/RtmpfJVRor/downloaded_packages' >

Hervé Pagès (10:51:51) (in thread): > It worked this time but I was wondering if sometimes you get the “downloaded length XXXXXXX != reported length 0” error when doing this.

Martin Grigorov (10:57:28) (in thread): > The download is still running.No such errors so far!

Hervé Pagès (11:02:50) (in thread): > You’re trying on another package? FWIW the biggest isSNPlocs.Hsapiens.dbSNP155.GRCh38(> 5GB) > But if this is an intermittent problem maybe you want to trydownload.file()in a loop on a small file?

Martin Grigorov (11:05:16) (in thread): > I started them all

Martin Grigorov (11:06:24) (in thread): > What is the full command?I am not an R developer…

Hervé Pagès (11:42:09) (in thread): > Let’s use a medium size package: > > url <- "[https://bioconductor.org/packages/3.19/data/experiment/src/contrib/brgedata_1.25.0.tar.gz](https://bioconductor.org/packages/3.19/data/experiment/src/contrib/brgedata_1.25.0.tar.gz)" > for (i in 1:20) download.file(url, tempfile()) > > If that does not work reliably then we’ll need to start looking into using a different download method (that can be controlled viadownload.file()’smethodargument). Thanks!

Hervé Pagès (11:54:42) (in thread): > You’re using the same R as the builds right? i.e. you’re starting R with~/bbs-3.19-bioc/R/bin/Rand from the biocbuild account. Just to be sure.

Martin Grigorov (12:35:10) (in thread): > yes!

Martin Grigorov (12:37:14) (in thread): > I just started the code. It downloads the file for the first iteration

Martin Grigorov (12:40:27) (in thread): > the first download finished without issues

Martin Grigorov (12:45:51) (in thread): > all packages have been successfully installed with BiocManager::install(). Let’s see the next build !

Hervé Pagès (13:14:11) (in thread): > > all packages have been successfully installed with BiocManager::install() > Great. Thanks! > > I just started the code. It downloads the file for the first iteration > So the 20 iterations went fine?

Martin Grigorov (13:15:03) (in thread): > Still running but so far everything is fine!

2023-11-10

Hervé Pagès (18:41:24) (in thread): > Thanks@Andres Wokatyfor adding kunpeng2 to the report.@Martin GrigorovAll the download errors are gone on today’s report:tada:There’s still a high number of INSTALL error though. I’m looking at it and will post some details soon.

Hervé Pagès (19:09:11): > Today’s build report with results for kunpeng2:https://bioconductor.org/checkResults/3.19/bioc-LATEST/long-report.htmlA major cause of INSTALL errors on kunpeng2 is because CRAN packageSeuratcould not be installed e.g.https://bioconductor.org/checkResults/3.19/bioc-LATEST/APL/kunpeng2-install.htmlThis affects 30+ packages. > > The reason for this is that it depends on CRAN packageleidenwhich has the following INSTALL error on kunpeng2: > > Installing package into '/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library' > (as 'lib' is unspecified) > trying URL '[https://cran.rstudio.com/src/contrib/leiden_0.4.3.tar.gz](https://cran.rstudio.com/src/contrib/leiden_0.4.3.tar.gz)' > Content type 'application/x-gzip' length 3246718 bytes (3.1 MB) > ================================================== > downloaded 3.1 MB > > * installing **source** package 'leiden' ... > **** package 'leiden' successfully unpacked and MD5 sums checked > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > **** help > ***** installing help indices > **** building package indices > **** installing vignettes > **** testing if installed package can be loaded from temporary location > > ***** caught segfault ***** > address 0x90, cause 'memory not mapped' > > Traceback: > 1: py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error) > 2: (function() { Sys.setenv(PYTHONPATH = newpythonpath) on.exit(Sys.setenv(PYTHONPATH = oldpythonpath), add = TRUE) py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error)})() > 3: doTryCatch(return(expr), name, parentenv, handler) > 4: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 5: tryCatchList(expr, classes, parentenv, handlers) > 6: tryCatch({ oldpythonpath <- Sys.getenv("PYTHONPATH") newpythonpath <- Sys.getenv("RETICULATE_PYTHONPATH", unset = paste(config$pythonpath, system.file("python", package = "reticulate"), sep = .Platform$path.sep)) local({ Sys.setenv(PYTHONPATH = newpythonpath) on.exit(Sys.setenv(PYTHONPATH = oldpythonpath), add = TRUE) py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error) })}, error = function(e) { Sys.setenv(PATH = oldpath) if (is.na(curr_session_env)) { Sys.unsetenv("R_SESSION_INITIALIZED") } else { Sys.setenv(R_SESSION_INITIALIZED = curr_session_env) } stop(e)}) > 7: initialize_python() > 8: ensure_python_initialized(required_module = module) > 9: import(module) > 10: doTryCatch(return(expr), name, parentenv, handler) > 11: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 12: tryCatchList(expr, classes, parentenv, handlers) > 13: tryCatch({ import(module) TRUE}, error = clear_error_handler(FALSE)) > 14: reticulate::py_module_available("pandas") > 15: fun(libname, pkgname) > 16: doTryCatch(return(expr), name, parentenv, handler) > 17: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 18: tryCatchList(expr, classes, parentenv, handlers) > 19: tryCatch(fun(libname, pkgname), error = identity) > 20: runHook(".onAttach", ns, dirname(nspath), nsname) > 21: attachNamespace(ns, pos = pos, deps, exclude, include.only) > 22: doTryCatch(return(expr), name, parentenv, handler) > 23: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 24: tryCatchList(expr, classes, parentenv, handlers) > 25: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return && !quietly) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)}) > 26: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) > 27: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage")) > 28: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)) > 29: doTryCatch(return(expr), name, parentenv, handler) > 30: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 31: tryCatchList(expr, classes, parentenv, handlers) > 32: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call, nlines = 1L) prefix <- paste("Error in", dcall, ": ") LONG <- 75L sm <- strsplit(conditionMessage(e), "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && isTRUE(getOption("show.error.messages"))) { cat(msg, file = outFile) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))}) > 33: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE))) > 34: tools:::.test_load_package("leiden", "/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library/00LOCK-leiden/00new") > An irrecoverable exception occurred. R is aborting now ... > ERROR: loading failed > * removing '/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library/leiden' > > The downloaded source packages are in > '/home/biocbuild/tmp/Rtmpo60CDx/downloaded_packages' > Warning message: > In install.packages(pkg, repos = .NON_TARGET_REPOS, dependencies = FALSE, : > installation of package 'leiden' had non-zero exit status > > A segfault at load time! > Note thatleidenis doing a lot of complicated Python/reticulate business in their.onAttachhook (seehttps://github.com/TomKellyGenetics/leiden/blob/75bc9f5f2480f47416c53d5291dd385e806b695f/R/leiden.R#L494-L617), trying to load many python modules, and for some reason that is failing on kunpeng2. > > Problem with putting heavy business like this in the.onAttachor.onLoadhook is that if something goes wrong then the package cannot be loaded, so it makes it very hard to debug the problem. Which is why packages should not do this. > > I don’t have a solution or workaround yet. Just putting this here for the record for now.

Hervé Pagès (19:54:49): > Another cause of INSTALL errors is from packages that try to access AnnotationHub or ExperimentHub in their.onLoadhook. Loading them fails with the following error on kunpeng2: > > **** testing if installed package can be loaded from temporary location > Error: package or namespace load failed for 'coMethDMR': > .onLoad failed in loadNamespace() for 'coMethDMR', details: > call: NULL > error: Corrupt Cache: index file > See AnnotationHub's TroubleshootingTheCache vignette section on corrupt cache > cache: /home/biocbuild/.cache/R/ExperimentHub > filename: experimenthub.index.rds > Error: loading failed > Execution halted > ERROR: loading failed > > Possibly related to kunpeng2’s downloading struggles.@Lori ShepherdI wonder if there is a simple command you could recommend to fix the index file.

2023-11-13

Martin Grigorov (05:30:39) (in thread): > https://github.com/TomKellyGenetics/leiden/issues/26- it seems it is not failing only on Linux ARM64 - Attachment: #26 Package installation segfault error on CentOS > Hi there, I’m attempting to install leiden on CentOS 7.9.2009 and have encountered the following installation error. Would appreciate any assistance. Thanks! > > > > install.packages("leiden") > Installing package into ‘/home/an252/R-4.2.1/library’ > (as ‘lib’ is unspecified) > trying URL '[https://mirrors.nics.utk.edu/cran/src/contrib/leiden_0.4.3.tar.gz](https://mirrors.nics.utk.edu/cran/src/contrib/leiden_0.4.3.tar.gz)' > Content type 'application/x-gzip' length 3246718 bytes (3.1 MB) > ================================================== > downloaded 3.1 MB > > * installing **source** package ‘leiden’ ... > **** package ‘leiden’ successfully unpacked and MD5 sums checked > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > **** help > ***** installing help indices > **** building package indices > **** installing vignettes > **** testing if installed package can be loaded from temporary location > sh: line 1: 15516 Segmentation fault R_TESTS= '/n/app/R/4.2.1-gcc-9.2.0/lib64/R/bin/R' --no-save --no-restore --no-echo 2>&1 < '/tmp/Rtmpz2zILL/file3c7f4ff851a1' > > ***** caught segfault ***** > address 0x90, cause 'memory not mapped' > > Traceback: > 1: py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error) > 2: (function() { Sys.setenv(PYTHONPATH = newpythonpath) on.exit(Sys.setenv(PYTHONPATH = oldpythonpath), add = TRUE) py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error)})() > 3: doTryCatch(return(expr), name, parentenv, handler) > 4: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 5: tryCatchList(expr, classes, parentenv, handlers) > 6: tryCatch({ oldpythonpath <- Sys.getenv("PYTHONPATH") newpythonpath <- Sys.getenv("RETICULATE_PYTHONPATH", unset = paste(config$pythonpath, system.file("python", package = "reticulate"), sep = .Platform$path.sep)) local({ Sys.setenv(PYTHONPATH = newpythonpath) on.exit(Sys.setenv(PYTHONPATH = oldpythonpath), add = TRUE) py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error) })}, error = function(e) { Sys.setenv(PATH = oldpath) if ([is.na](http://is.na)(curr_session_env)) { Sys.unsetenv("R_SESSION_INITIALIZED") } else { Sys.setenv(R_SESSION_INITIALIZED = curr_session_env) } stop(e)}) > 7: initialize_python(required_module, use_environment) > 8: ensure_python_initialized(required_module = module) > 9: import(module) > 10: doTryCatch(return(expr), name, parentenv, handler) > 11: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 12: tryCatchList(expr, classes, parentenv, handlers) > 13: tryCatch({ import(module) TRUE}, error = clear_error_handler(FALSE)) > 14: reticulate::py_module_available("pandas") > 15: fun(libname, pkgname) > 16: doTryCatch(return(expr), name, parentenv, handler) > 17: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 18: tryCatchList(expr, classes, parentenv, handlers) > 19: tryCatch(fun(libname, pkgname), error = identity) > 20: runHook(".onAttach", ns, dirname(nspath), nsname) > 21: attachNamespace(ns, pos = pos, deps, exclude, include.only) > 22: doTryCatch(return(expr), name, parentenv, handler) > 23: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 24: tryCatchList(expr, classes, parentenv, handlers) > 25: tryCatch({ attr(package, "LibPath") [is.na](- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return && !quietly) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)}) > 26: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) > 27: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage")) > 28: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)) > 29: doTryCatch(return(expr), name, parentenv, handler) > 30: tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 31: tryCatchList(expr, classes, parentenv, handlers) > 32: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call, nlines = 1L) prefix <- paste("Error in", dcall, ": ") LONG <- 75L sm <- strsplit(conditionMessage(e), "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if ([http://is.na)(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w ](http://is.na)(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w ) LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && isTRUE(getOption("show.error.messages"))) { cat(msg, file = outFile) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))}) > 33: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE))) > 34: tools:::.test_load_package("leiden", "/home/an252/R-4.2.1/library/00LOCK-leiden/00new") > An irrecoverable exception occurred. R is aborting now ... > ERROR: loading failed > * removing ‘/home/an252/R-4.2.1/library/leiden’ > > The downloaded source packages are in > ‘/tmp/Rtmp6ZFOiM/downloaded_packages’ > Warning message: > In install.packages("leiden") : > installation of package ‘leiden’ had non-zero exit status > > >

Martin Grigorov (05:43:35) (in thread): > > $ ll /home/biocbuild/.cache/R/ExperimentHub > total 7.3M > -rw-r--r-- 1 biocbuild biocbuild 271K Nov 6 21:28 1babb872364939_1babb872364939_hub_index.rds > -rw-r--r-- 1 biocbuild biocbuild 6.7M Nov 6 21:28 1bac597308bd04_experimenthub.sqlite3 > -rw-r--r-- 1 biocbuild biocbuild 271K Nov 6 21:28 1bacc7b213e0e_1bacc7b213e0e_hub_index.rds > -rw-r--r-- 1 biocbuild biocbuild 20K Nov 6 21:28 BiocFileCache.sqlite > -rw------- 1 biocbuild biocbuild 0 Nov 6 21:28 BiocFileCache.sqlite.LOCK > > the files are small. I could just delete them and let the cache repopulate with the next build

Martin Grigorov (05:51:33) (in thread): > > R CMD INSTALL coMethDMR > * installing to library '/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library' > * installing **source** package 'coMethDMR' ... > **** using staged installation > **** R > **** data > **** inst > **** byte-compile and prepare package for lazy loading > **** help > ***** installing help indices > **** building package indices > **** installing vignettes > **** testing if installed package can be loaded from temporary location > **** testing if installed package can be loaded from final location > **** testing if installed package keeps a record of temporary installation path > * DONE (coMethDMR) > > and the new cache files are: > > ll ~/.cache/R/ExperimentHub/ > total 7.0M > -rw-r--r-- 1 biocbuild biocbuild 271K Nov 13 10:46 37724512f08563_37724512f08563_hub_index.rds > -rw-r--r-- 1 biocbuild biocbuild 6.7M Nov 13 10:46 377245654d8db8_experimenthub.sqlite3 > -rw-r--r-- 1 biocbuild biocbuild 20K Nov 13 10:46 BiocFileCache.sqlite > -rw------- 1 biocbuild biocbuild 0 Nov 13 10:46 BiocFileCache.sqlite.LOCK >

Martin Grigorov (09:07:07): > https://bioconductor.org/checkResults/3.19/bioc-LATEST/BASiCS/nebbiolo1-install.html- the package fails on all builder nodes but for some reason it is still reported asOKon Linux x86_64

Martin Grigorov (09:11:49) (in thread): > but now I face the same error for packageDOSE: > > R CMD INSTALL DOSE > * installing to library '/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library' > * installing **source** package 'DOSE' ... > **** using staged installation > **** R > **** data > **** inst > **** byte-compile and prepare package for lazy loading > Error: .onLoad failed in loadNamespace() for 'HPO.db', details: > call: NULL > error: Corrupt Cache: index file > See AnnotationHub's TroubleshootingTheCache vignette section on corrupt cache > cache: /home/biocbuild/.cache/R/AnnotationHub > filename: annotationhub.index.rds > Execution halted > ERROR: lazy loading failed for package 'DOSE' > * removing '/home/biocbuild/R/R-4.4-devel-2023.11.02/site-library/DOSE' >

Andres Wokaty (09:51:22) (in thread): > It also has the'weave'error that masks other issues. Thanks for pointing it out. I made an issue to look into it later:https://github.com/Bioconductor/BBS/issues/366 - Attachment: #366 Investigate basics failures > basics fails on all builders, passes on x86_64 linux with errors. See https://bioconductor.org/checkResults/3.19/bioc-LATEST/BASiCS/nebbiolo1-install.html

Hervé Pagès (11:45:05) (in thread): > I’m not surprised!

2024-01-11

Lori Shepherd (11:43:34): > @Lori Shepherd has joined the channel

Lori Shepherd (11:50:06) (in thread): > sorry didnt realize this channel existed and I wasn’t notified because of it. The index is created as part of the initial download of the hub. You can try to fix it as documented in the recommended vignette sectionhttps://bioconductor.org/packages/release/bioc/vignettes/AnnotationHub/inst/doc/TroubleshootingTheCache.html#index-filebut likely better to remove the cache and reset completely

2024-01-16

Andres Wokaty (10:53:45): > Hi@Martin Grigorov, Happy new year! I’m going to update the devel builders today because there’s some new behavior related tohttps://github.com/RcppCore/Rcpp/issues/1287that is only on more recent R devel. When you have time, can you update the R on kunpeng2? Thank you. - Attachment: #1287 A fresh -Wformat-security issue under r-devel > Update 2023-11-28: If you came here because of a similar message in your package please read on and see particularly <https://github.com/RcppCore/Rcpp/issues/1287#issuecomment-1829886024|this comment below> for the fairly simple fix. > > * * * > > While working on an update for RQuantLib with a few r-devel discovered minor changes, CRAN and I both came across a new nag this time from -Wformat-security.. Our glue code in src/attributes.cpp does > > > << " if (rcpp_isError_gen) {" << std::endl > << " SEXP rcpp_msgSEXP_gen = Rf_asChar(rcpp_result_gen);" << std::endl > << " UNPROTECT(1);" << std::endl > << " Rf_error(CHAR(rcpp_msgSEXP_gen));" << std::endl > << " }" << std::endl > > > and the Rf_error(someCharvariablehere) now makes the compiler bark under -Wformat-security: > > > RcppExports.cpp:180:18: warning: format string is not a string literal (potentially insecure) [-Wformat-security] > > > The fix is pretty easy: add a "%s". I will take care of that shortly.

Martin Grigorov (11:50:09): > Happy new year,@Andres Wokaty!I’llupdate it tomorrow!

2024-01-17

Martin Grigorov (08:11:18): > I am going to installhttps://cran.r-project.org/src/base-prerelease/R-devel_2024-01-16_r85812.tar.xz

Martin Grigorov (08:31:14): > Done! The next few reports most probably will have higher rate of errors! I will install manually the problematic (big) packages once the reports are available!

2024-01-20

Michael Kaufman (15:21:26): > @Michael Kaufman has joined the channel

2024-02-01

Andres Wokaty (16:06:41): > Hi@Martin GrigorovWould it be possible to installReactomeGSA.dataon kunpeng2:https://bioconductor.org/checkResults/devel/bioc-LATEST/ReactomeGSA/kunpeng2-buildsrc.html?

Martin Grigorov (16:35:50) (in thread): > Itravel and I have n access to the VM. I will do it early next week!

Andres Wokaty (16:36:05) (in thread): > Great! Safe travels!

2024-02-22

Andres Wokaty (12:21:31): > Hi@Martin Grigorov, I noticed kunpeng2 hasn’t been finishing the build so I have removed it from the reports. (I’m also having an issue with merida1 not finishing too.) Could I ask you to look at kunpeng2 when you have time? Thank you.

Martin Grigorov (12:51:59): > I will take a look soon!

Martin Grigorov (14:22:40): > > bbs.jobs.processJobQueue> DONE JOB BioNAR (211/2243) ON SLOT 1/16 after 59.08 seconds [retcode=1] > BBS> No such process: 1320955 > OSError: [Errno 28] No space left on device > > let me clean it up!

Martin Grigorov (14:30:08): > I’ve cleaned up ~300Gb. It should be enough for a new build run! I will clean some more tomorrow !

2024-02-23

Martin Grigorov (03:13:05) (in thread): > I cleaned up another ~300Gb in /home/biocbuild/tmp folder. Is it safe to delete the contents of this folder before every build run ?

Andres Wokaty (13:20:56) (in thread): > Sorry, I didn’t see your message earlier. It’s probably okay. I don’t know if large things are cached there, which might slow things down a bit; however, I am going to switch the devel builders to run every other day to allow more time for each run. Devel will run on Monday, Wednesday, Friday at the same time but only these days. You’ll want to set kunpeng2 to follow the cadence when you have time. Thank you.

2024-02-24

Martin Grigorov (01:56:28) (in thread): > I’ve updated the crontab entry to00 19 * * 1,3,5!

2024-03-06

Andres Wokaty (10:32:49): > Hi@Martin Grigorovwhen you have time, can you update R on kunpeng2? I would like to update the devel builders by next week. Thank you.

Martin Grigorov (10:33:55) (in thread): > OK! I will do it tomorrow!

Andres Wokaty (12:50:06) (in thread): > I forgot to mention that I am going to switch the devel builders back to daily builds as they were before. This is all in preparation for the release. Could you change your cronjob as well to the previous configuration?

2024-03-07

Martin Grigorov (03:29:13) (in thread): > restored the cron job to0-5!

Martin Grigorov (03:48:31) (in thread): > R has been updated tohttps://stat.ethz.ch/R/daily/R-devel.tar.gz! Some package builds may/will fail in the coming days! I will install manually the big packages when I have a list of them (failed/timed out ones) tomorrow

Andres Wokaty (09:07:03) (in thread): > Sounds great! Thank you.

2024-03-18

Andres Wokaty (14:36:05): > Hi@Martin GrigorovI will update the devel builders again tomorrow to incorporate recent bug fixes. Please update kunpeng2 when you have time.

Martin Grigorov (14:37:49) (in thread): > I can do it tomorrow too!

2024-03-20

Martin Grigorov (08:01:43) (in thread): > Updated it to2024-03-19 r86153!

2024-04-16

Andres Wokaty (13:06:55): > Hi@Martin GrigorovI am updating the devel builders with the beta release. Please update kunpeng2 when you have time. Thank you:slightly_smiling_face:

2024-04-22

Martin Grigorov (07:21:18): > Hi! Is kunpeng2 currently excluded from the reports ?https://bioconductor.org/checkResults/3.19/bioc-LATEST/long-report.html

Martin Grigorov (07:23:29): > https://bioconductor.org/checkResults/3.19/bioc-LATEST/NanoMethViz/kunpeng2-install.html- generated at Apr 18thhttps://bioconductor.org/checkResults/3.19/bioc-LATEST/NanoMethViz/- generated at Apr 20th (no kunpeng2)

Andres Wokaty (11:38:30): > I think there might be an issue somewhere. I will take a look.

Andres Wokaty (11:46:59): > Hrm, I didn’t do anything because the report was being generated, but it is there now.

Martin Grigorov (11:50:50): > Yes.It is there now!

Hervé Pagès (14:08:08): > I removed it from Saturday’s report because it didn’t build

2024-04-24

Andres Wokaty (12:31:53): > @Martin Grigorovkunpeng2 seems to be having an issue. I only see that nebbiolo1 was able to retrieve install-related artifacts, so I am removing it to rerun the report. Could you check if kungpeng2 was able to finish the build?

Martin Grigorov (14:03:08): > it stopped at: > > ... > BBS> o [1376/2290] git_last_commit (cd333dc) for package multiscan has not changed --> skip > BBS> o [1377/2290] downloading multistateQTL_0.99.1.tar.gz ... ok > BBS> o [1378/2290] git_last_commit (9352a6f) for package multiWGCNA has not changed --> skip > BBS> o [1379/2290] git_last_commit (9c32120) for package multtest has not changed --> skip > BBS> o [1380/2290] git_last_commit (4c13a83) for package mumosa has not changed --> skip > BBS> o [1381/2290] git_last_commit (f7bd333) for package MungeSumstats has not changed --> skip > BBS> o [1382/2290] git_last_commit (817a9f3) for package muscat has not changed --> skip > BBS> o [1383/2290] git_last_commit (c123fb2) for package muscle has not changed --> skip > BBS> o [1384/2290] downloading musicatk_1.13.1.tar.gz ... 3 failed attempts => EXIT at 2024-04-23 20:20:19. >

Martin Grigorov (14:06:40): > I just tried to start it manually and it logged: > > BBS> ============================================================== > BBS> [STAGE2] STARTING STAGE2 at Wed Apr 24 17:58:41 2024 > BBS> [waitForTargetRepoToBeReady] Unable to access[http://155.52.207.165/BBS/3.19/bioc/src/contrib/PACKAGES](http://155.52.207.165/BBS/3.19/bioc/src/contrib/PACKAGES). Looks like the target repo is not ready yet! > BBS> [waitForTargetRepoToBeReady] -> will wait 3 minutes before trying again ... >

Martin Grigorov (14:07:10): > http://155.52.207.165/BBS/3.19/bioc/is empty

Hervé Pagès (18:45:25): > You started the builds too early. The central builder populated this folder around 2:32 pm EST. This is why the satellite builders are scheduled to start building at 3pm EST.

2024-04-25

Martin Grigorov (02:43:27): > got it! > I started it just to see whether it can download new packages > Now it is running fine! (this is the cronjob!)

Martin Grigorov (08:24:32): > today’s build finished successfully: > > .jobs.processJobQueue> Finished. > > BBS> END STAGE4 loop. > BBS> ------------------------------------------------------------- > BBS> STAGE4 SUMMARY: > BBS> o Working dir: /home/biocbuild/bbs-3.19-bioc/meat > BBS> o 2163 srcpkg file(s) in working dir > BBS> o 2163 srcpkg file(s) queued and processed > BBS> o Total time: 30858.29 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Thu Apr 25 11:54:18 2024. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.19-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. >

2024-04-26

Robert Castelo (13:10:59): > @Robert Castelo has joined the channel

Robert Castelo (13:16:11): > Hi everyone, one of the packages I maintain,GenomicScores, is not building in kungpeng2 (see reporthere) because it is missing the annotation package MafH5.gnomAD.v4.0.GRCh38, which we added during this devel cycle and which GenomicScores uses in its vignette. Lori advised me to ask here that somebody please installsMafH5.gnomAD.v4.0.GRCh38manually in kungpeng2. Is it that possible? Thanks!! - Attachment (Bioconductor): GenomicScores > Provide infrastructure to store and access genomewide position-specific scores within R and Bioconductor. - Attachment (Bioconductor): MafH5.gnomAD.v4.0.GRCh38 (development version) > Store minor allele frequency data from the Genome Aggregation Database (gnomAD version 4.0) for the human genome version GRCh38.

Andres Wokaty (17:34:41): > @Martin GrigorovI apologize that in my previous messagehttps://community-bioc.slack.com/archives/CEQ04GKEC/p1713463375556859?thread_ts=1711501233.971639&cid=CEQ04GKECwhen you asked about removing the cache because kunpeng2 has adatabase disk image is malformederror that I only gave you information about reinstating the AnnotationHub cache but not the ExperimentHub cache. When you remove these two caches, it’s better to reinstate them both with: > > library(AnnotationHub) > library(ExperimentHub) > ah <- AnnotationHub() > eh <- ExperiementHub() > - Attachment: Attachment > @Martin Grigorov I am not very familiar with the database disk image is malformed error. If you remove the AnnotationHub cache, can you reinstate it in R with > > library(AnnotationHub) > ah <- AnnotationHub()

2024-04-27

Martin Grigorov (03:02:16) (in thread): > > > library(ExperimentHub) > > ah <- AnnotationHub() > snapshotDate(): 2024-04-02 > > eh <- ExperiementHub() > Error in ExperiementHub() : could not find function "ExperiementHub" > > eh <- ExperimentHub() > snapshotDate(): 2024-04-19 > > > Save workspace image? [y/n/c]: y > biocbuild@kunpeng2 ~> rm -rf ~/.cache/R/ExperimentHub/* > biocbuild@kunpeng2 ~> rm -rf ~/.cache/R/AnnotationHub/* > biocbuild@kunpeng2 ~> R > > R version 4.4.0 beta (2024-04-15 r86425) -- "Puppy Cup" > ... > > library(AnnotationHub) > Loading required package: BiocGenerics > > Attaching package: 'BiocGenerics' > > The following objects are masked from 'package:stats': > > IQR, mad, sd, var, xtabs > > The following objects are masked from 'package:base': > > anyDuplicated, aperm, append, as.data.frame, basename, cbind, > colnames, dirname, do.call, duplicated, eval, evalq, Filter, Find, > get, grep, grepl, intersect, is.unsorted, lapply, Map, mapply, > match, mget, order, paste, pmax, pmax.int, pmin, pmin.int, > Position, rank, rbind, Reduce, rownames, sapply, setdiff, table, > tapply, union, unique, unsplit, which.max, which.min > > Loading required package: BiocFileCache > Loading required package: dbplyr > > library(ExperimentHub) > > ah [======================================================================| 100% > > snapshotDate(): 2024-04-02 > ](- AnnotationHub() > ) eh <- ExperimentHub() > snapshotDate(): 2024-04-19 > > re-installed them! > But the dates before and after are still the same

2024-04-29

Lori Shepherd (07:01:30) (in thread): > @Martin Grigorovmaking sure you saw this too?

Martin Grigorov (07:02:13) (in thread): > Thanks! Indeed I didn’t notice it! Let me check !

Martin Grigorov (07:05:00) (in thread): > > > BiocManager::install("MafH5.gnomAD.v4.0.GRCh38") > Bioconductor version 3.19 (BiocManager 1.30.22), R 4.4.0 beta (2024-04-15 > r86425) > Installing package(s) 'MafH5.gnomAD.v4.0.GRCh38' > trying URL '[https://bioconductor.org/packages/3.19/data/annotation/src/contrib/MafH5.gnomAD.v4.0.GRCh38_3.19.0.tar.gz](https://bioconductor.org/packages/3.19/data/annotation/src/contrib/MafH5.gnomAD.v4.0.GRCh38_3.19.0.tar.gz)' > Content type 'application/x-gzip' length 1487709773 bytes (1418.8 MB) > == > > it is being installed !

Martin Grigorov (07:06:44) (in thread): > > assertive.sets > dnet > ensurer > exomeCopy > IHW > lpsymphony > Ringo > Rmpi > rsbml > seqbias > tiledb > VarfromPDB > > my script does not detect this package as a missing dependency … I guess it is being installed with different means than being a dependency in DESCRIPTION … > I will add it to my list ofstaticdeps to install after updating R!

Robert Castelo (08:00:09) (in thread): > Thanks!! MafH5.gnomAD.v4.0.GRCh38 is used in the vignette of GenomicScores, and consequently appears in theSuggestsfield of itsDESCRIPTIONfile. In principle, if you install software packages, such as GenomicScores, with the argumentdependencies=TRUE,BiocManager::install()should install all the packages listed inSuggests.

Martin Grigorov (08:01:03) (in thread): > > Content type 'application/x-gzip' length 1487709773 bytes (1418.8 MB) > ================================================== > downloaded 1418.8 MB > > * installing **source** package 'MafH5.gnomAD.v4.0.GRCh38' ... > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > Warning: replacing previous import 'utils::findMatches' by 'S4Vectors::findMatches' when loading 'MafH5.gnomAD.v4.0.GRCh38' > **** help > ***** installing help indices > **** building package indices > **** testing if installed package can be loaded from temporary location > Warning: replacing previous import 'utils::findMatches' by 'S4Vectors::findMatches' when loading 'MafH5.gnomAD.v4.0.GRCh38' > **** testing if installed package can be loaded from final location > Warning: replacing previous import 'utils::findMatches' by 'S4Vectors::findMatches' when loading 'MafH5.gnomAD.v4.0.GRCh38' > **** testing if installed package keeps a record of temporary installation path > * DONE (MafH5.gnomAD.v4.0.GRCh38) > > The downloaded source packages are in > '/home/biocbuild/tmp/Rtmp6hwK5V/downloaded_packages' > Old packages: 'shinytest2' >

Martin Grigorov (08:03:30) (in thread): > https://bioconductor.org/checkResults/3.19/bioc-LATEST/GenomicScores/kunpeng2-buildsrc.htmlis generated at Apr 27th

Martin Grigorov (08:04:05) (in thread): > > biocbuild@kunpeng2 ~> ll /home/biocbuild/bbs-3.19-bioc/products-out/buildsrc/GenomicScores.buildsrc-out.txt > -rw-r--r-- 1 biocbuild biocbuild 1.3K Apr 28 22:57 /home/biocbuild/bbs-3.19-bioc/products-out/buildsrc/GenomicScores.buildsrc-out.txt > biocbuild@kunpeng2 ~> less /home/biocbuild/bbs-3.19-bioc/products-out/buildsrc/GenomicScores.buildsrc-out.txt > biocbuild@kunpeng2 ~> cat /home/biocbuild/bbs-3.19-bioc/products-out/buildsrc/GenomicScores.buildsrc-out.txt > ############################################################################## > ############################################################################## > ### > ### Running command: > ### > ### /home/biocbuild/R/R-beta-2024-04-15_r86425/bin/R CMD build --keep-empty-dirs --no-resave-data GenomicScores > ### > ############################################################################## > ############################################################################## > > > * checking for file 'GenomicScores/DESCRIPTION' ... OK > * preparing 'GenomicScores': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'GenomicScores.Rmd' using rmarkdown > > Quitting from lines at lines 118-120 [unnamed-chunk-1] (GenomicScores.Rmd) > Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics: > Corrupt Cache: index file > See AnnotationHub's TroubleshootingTheHubs vignette section on corrupt cache > cache: /home/biocbuild/.cache/R/AnnotationHub > filename: annotationhub.index.rds > --- failed re-building 'GenomicScores.Rmd' > > SUMMARY: processing the following file failed: > 'GenomicScores.Rmd' > > Error: Vignette re-building failed. > Execution halted > > at Apr 28th the error is due to AnnotationHub …

Martin Grigorov (08:04:48) (in thread): > I re-installed AnnotationHub the other day by request from@Andres Wokaty. Let’s see the next report !

Martin Grigorov (08:05:15) (in thread): > I will test the build of GenomicScores manually !

Martin Grigorov (08:06:53) (in thread): > > biocbuild@kunpeng2 ~/git> git clone[https://git.bioconductor.org/packages/GenomicScores](https://git.bioconductor.org/packages/GenomicScores)Cloning into 'GenomicScores'... > remote: Enumerating objects: 1810, done. > remote: Counting objects: 100% (1810/1810), done. > remote: Compressing objects: 100% (1490/1490), done. > remote: Total 1810 (delta 1209), reused 471 (delta 296), pack-reused 0 > Receiving objects: 100% (1810/1810), 2.90 MiB | 2.45 MiB/s, done. > Resolving deltas: 100% (1209/1209), done. > biocbuild@kunpeng2 ~/git> R CMD build GenomicScores > * checking for file 'GenomicScores/DESCRIPTION' ... OK > * preparing 'GenomicScores': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'GenomicScores.Rmd' using rmarkdown > > Quitting from lines at lines 118-120 [unnamed-chunk-1] (GenomicScores.Rmd) > Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics: > Corrupt Cache: index file > See AnnotationHub's TroubleshootingTheHubs vignette section on corrupt cache > cache: /home/biocbuild/.cache/R/AnnotationHub > filename: annotationhub.index.rds > --- failed re-building 'GenomicScores.Rmd' > > SUMMARY: processing the following file failed: > 'GenomicScores.Rmd' > > Error: Vignette re-building failed. > Execution halted >

Martin Grigorov (08:08:12) (in thread): > do I need to do something about it ?

Lori Shepherd (08:08:26) (in thread): > Thats an issue with AnnotationHub again – I’m not sure why the file is getting corrupt

Martin Grigorov (08:08:31) (in thread): > > library(AnnotationHub) > Loading required package: BiocGenerics > > Attaching package: 'BiocGenerics' > > The following objects are masked from 'package:stats': > > IQR, mad, sd, var, xtabs > > The following objects are masked from 'package:base': > > anyDuplicated, aperm, append, as.data.frame, basename, cbind, > colnames, dirname, do.call, duplicated, eval, evalq, Filter, Find, > get, grep, grepl, intersect, is.unsorted, lapply, Map, mapply, > match, mget, order, paste, pmax, pmax.int, pmin, pmin.int, > Position, rank, rbind, Reduce, rownames, sapply, setdiff, table, > tapply, union, unique, unsplit, which.max, which.min > > Loading required package: BiocFileCache > Loading required package: dbplyr > > library(ExperimentHub) > > ah <- AnnotationHub() > > snapshotDate(): 2024-04-02 > Error: Corrupt Cache: index file > See AnnotationHub's TroubleshootingTheHubs vignette section on corrupt cache > cache: /home/biocbuild/.cache/R/AnnotationHub > filename: annotationhub. > > eh <- ExperimentHub() > snapshotDate(): 2024-04-19 >

Robert Castelo (08:17:48) (in thread): > GenomicScoresbuilds finein every other platform of the BBS and also in my own linux and macOS (x86) machines, so as the error message suggests and Lori says, the cache is corrupt. The AnnotationHub documentation includes asubsectionon how to deal with this error, but I guess the fastest would be to remove altogether/home/biocbuild/.cache, which would force the BBS to build the cache again, but I do not know if this would be the reocommended action,@Lori Shepherd?

Lori Shepherd (08:18:12) (in thread): > yes

Martin Grigorov (08:19:25) (in thread): > OK! I will wait for the daily build to finish and delete the cache !

Lori Shepherd (08:20:34) (in thread): > just annotation should be sufficient and I would do as@Andres Wokatyrecommended and after deleting it before the builds get to it, reinstate it manually with a > > library(AnnotationHub) > ah = AnnotationHub() >

Martin Grigorov (09:19:52) (in thread): > buildandcheckare OK for GenomicScores after reinstating AnnotationHub !

Robert Castelo (09:23:45) (in thread): > Great, thank you very much!!!

2024-04-30

Robert Castelo (03:42:40) (in thread): > Hi@Martin Grigorov, I’m afraid the corrupt cache problem is persisting for a number of packages, in my case as a maintainer, now not only GenomicScores, but also atena, if we look at the overal stats here:https://bioconductor.org/checkResults/3.19/bioc-LATEST/long-report.htmlthere are 135 packages that are not building in kungpeng2, which is over twice as much as in the other platforms.

Martin Grigorov (03:44:15) (in thread): > do you have a concrete example of the cache problem ?

Martin Grigorov (03:44:37) (in thread): > many projects didn’t bother to fix their Linux ARM64 build …:confused:

Robert Castelo (03:57:20) (in thread): > If you’re referring to a concrete package not building, this would be the report for GenomicScores:https://bioconductor.org/checkResults/3.19/bioc-LATEST/GenomicScores/kunpeng2-buildsrc.htmland this other one, for instance, for GenomicRanges:https://bioconductor.org/checkResults/3.19/bioc-LATEST/GenomicRanges/kunpeng2-buildsrc.htmlbut, if you refer about whether I have an example on how the cache goes corrupt, unfortunately I cannot reproduce the problem in my own computer. From a very naive perspective, I’d think that it could arise when building in parallel packages that access the AnnotationHub or the ExperimentHub and attempt to update that cache concurrently, but this is just a guess.

Martin Grigorov (04:00:07) (in thread): > @Lori ShepherdAny hints how to debug what exactly is corrupted ?

Martin Grigorov (04:05:29) (in thread): > > biocbuild@kunpeng2 ~/.c/R/AnnotationHub> ll > total 6.6G > -rw-r--r-- 1 biocbuild biocbuild 74M Apr 29 23:49 11466920bdd750_81731 > -rw-r--r-- 1 biocbuild biocbuild 788K Apr 29 23:52 11a1a286c1b33_98511 > -rw-r--r-- 1 biocbuild biocbuild 259M Apr 30 00:07 12332335a1217f_113795 > ... > -rw-r--r-- 1 biocbuild biocbuild 40K Apr 30 07:19 BiocFileCache.sqlite > -rw------- 1 biocbuild biocbuild 0 Apr 29 12:19 BiocFileCache.sqlite.LOCK > ... > -rw-r--r-- 1 biocbuild biocbuild 2.1M Apr 30 03:26 289f934a9b592_289f934a9b592_hub_index.rds > -rw-r--r-- 1 biocbuild biocbuild 125M Apr 29 20:01 289f977e95d65_annotationhub.sqlite3 > > there are: > * many files without extension. > * BiocFileCache.sqlite > * annotationhub.sqlite3

Martin Grigorov (04:05:38) (in thread): > > sqlite3 ~/.cache/R/AnnotationHub/289f977e95d65_annotationhub.sqlite3 > SQLite version 3.37.2 2022-01-06 13:25:41 > Enter ".help" for usage hints. > sqlite> .tables > biocversions rdatapaths schema_info test > input_sources recipes statuses timestamp > location_prefixes resources tags >

Martin Grigorov (04:06:11) (in thread): > > BiocFileCache.sqlite contains: > sqlite> .tables > metadata resource >

2024-05-09

Martin Grigorov (09:51:42): > kunpeng2 has been updated to R 4.4.0 and Bioc 3.20. The build later today will be the first for 3.20! I will install the big data packages once they appear in the logs!

Andres Wokaty (11:25:39) (in thread): > Just a reminder: the devel builders are building every other day, so a report is produced Monday, Wednesday, Friday.

Martin Grigorov (11:34:57) (in thread): > Ok! I will update the cron job!

Martin Grigorov (14:53:28) (in thread): > I will let it run today, so I could install the failed big data packages tomorrow !

2024-05-19

David Rach (12:52:42): > @David Rach has joined the channel

2024-05-22

Martin Grigorov (03:46:54): > kunpeng2 went out of disk space with the last run! > Deleting 4 of the old R-4.4-devel installations freed half a TB!:slightly_smiling_face:

2024-06-07

Hervé Pagès (14:14:50): > Hi Martin, the machines collecting the build results from kunpeng2 have changed. The IP addresses of the new machines are 155.52.47.135 and 155.52.47.146. So please make the necessary changes on your side so we can keep collecting build results from kunpeng2 by running the usualrsync -azv 114.119.187.96::bioc ...command from these new machines. Thanks!

Martin Grigorov (15:40:41): > Thanks, Herve!I’llsend a PR tomorrow!

Hervé Pagès (18:24:46): > No need for a PR. It’s a change that needs to happen on your firewall. Right now I get a “Connection timed out” when I runrsync -azv 114.119.187.96::bioc ...from the new collecting machines.

2024-06-10

Martin Grigorov (02:12:10) (in thread): > I’ve requested the changes to our network admins!

Martin Grigorov (23:29:19) (in thread): > @Hervé Pagèsthe new static IPs have been enabled!Please test and let me know!

2024-06-11

Hervé Pagès (00:58:33) (in thread): > Oh my.. I sent you the wrong IP addresses:face_with_rolling_eyes:The correct ones are 149.165.171.124 and 149.165.154.78. I’m so sorry! Thanks@Martin Grigorov

Martin Grigorov (01:34:10) (in thread): > :grinning:

Martin Grigorov (03:38:23) (in thread): > @Hervé Pagèsdone! Please check again !

Hervé Pagès (21:07:03) (in thread): > I still get a timeout: > > biocbuild@bbscentral2:~$ rsync -azv 114.119.187.96::bioc /home/biocbuild/public_html/BBS/3.20/bioc/products-in/kunpeng2 > rsync: [Receiver] failed to connect to 114.119.187.96 (114.119.187.96): Connection timed out (110) > rsync error: error in socket IO (code 10) at clientserver.c(139) [Receiver=3.2.7] > > bbscentral2’s IP address is 149.165.154.78.

2024-06-12

Martin Grigorov (04:59:37) (in thread): > I’ve been told that the problem should be fixed now!

Hervé Pagès (17:34:56) (in thread): > Hmm.. still no luck with this. Same error:disappointed:

Martin Grigorov (21:53:18) (in thread): > Hi

Martin Grigorov (21:54:04) (in thread): > What about the other machine (the second IP address)?

Martin Grigorov (21:54:54) (in thread): > 149.165.171.124

Martin Grigorov (21:55:43) (in thread): > Does it work from there or fails the same way?

Martin Grigorov (22:00:29) (in thread): > Also could you please verify that 149.165.154.78 is correct?Could it be that the machine has two network interfaces and uses another one for the connection?!

2024-06-17

Hervé Pagès (15:05:03) (in thread): > I triple-checked 149.165.154.78. This is the only IP address of the machine that I know of. Can you make sure that your rsync daemon is listening on port 873 (the default) and that the IT people have allowed incoming connections to that port? > It doesn’t work from the other machine either (149.165.171.124).

Martin Grigorov (15:24:44) (in thread): > Let me check the port!

Martin Grigorov (15:31:49) (in thread): > > root@kunpeng2 ~# netstat -antp | grep 873 > tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 1151/rsync > tcp6 0 0 :::873 :::* LISTEN 1151/rsync >

Martin Grigorov (15:32:01) (in thread): > looks good

Martin Grigorov (15:36:58) (in thread): > I have asked the network admins to confirm the port!

Martin Grigorov (15:38:19) (in thread): > Please try again before the end of your business hours!

Hervé Pagès (16:51:11) (in thread): > > biocbuild@bbscentral2:~$ rsync -azv 114.119.187.96::bioc /home/biocbuild/public_html/BBS/3.20/bioc/products-in/kunpeng2 > rsync: [Receiver] failed to connect to 114.119.187.96 (114.119.187.96): Connection timed out (110) > rsync error: error in socket IO (code 10) at clientserver.c(139) [Receiver=3.2.7] > > Note that the command still works if I do it on the previous central builders (155.52.207.165 and 155.52.207.166). So the only thing that the IT people need to do is edit the rules on the firewall to replace the previous IP addresses with the new ones. Doesn’t look like they did that.

2024-06-19

Martin Grigorov (01:20:19) (in thread): > @Hervé Pagèsplease check again

Hervé Pagès (13:33:13) (in thread): > It works!:tada:However, rsync module114.119.187.96::biocseems to contain the results of the BioC 3.19 builds (e.g. it has results forBiocVersion3.19.1 andBiocGenerics0.50.0). Are the 3.20 builds running on kunpeng2? If so, the114.119.187.96::biocmodule should point to the results of the 3.20 builds. Thanks!

Martin Grigorov (13:35:11) (in thread): > Yes, 3.20 builds are running!I’llcheck the rsyncd config in few hours

Martin Grigorov (14:43:39) (in thread): > I’ve updated the config and restarted the service

Martin Grigorov (14:45:40) (in thread): > the last few builds failed with:

Martin Grigorov (14:45:45) (in thread): > > BBS> Done cloning CRAN-style package repo[http://149.165.154.78/BBS/3.20/bioc/src/contrib/](http://149.165.154.78/BBS/3.20/bioc/src/contrib/)to /home/biocbuild/bbs-3.20-bioc/target-repo/ > BBS> Copying /home/biocbuild/bbs-3.20-bioc/target-repo/PACKAGES to /home/biocbuild/bbs-3.20-bioc/products-out/ ... OK > BBS> Extracting the 2242 source tarballs in /home/biocbuild/bbs-3.20-bioc/target-repo/ > to /home/biocbuild/bbs-3.20-bioc/meat/ ... OK > BBS> [STAGE2] cd BBS_WORK_TOPDIR/STAGE2_tmp > BBS> [STAGE2] Update non-target packages (1st run) ... OK > BBS> [get_list_of_target_pkgs] downloading meat-index.dcf ... ok > BBS> [build_pkg_dep_graph] 2242 pkgs written to target_pkgs.txt > BBS> [build_pkg_dep_graph] Calling build_pkg_dep_graph() defined in /home/biocbuild/BBS/utils/build_pkg_dep_graph.R to generate file pkg_dep_graph.txt ... ERROR! > BBS> [build_pkg_dep_graph] Command /home/biocbuild/R/R-devel_2024-01-16_r85812/bin/Rscript -e "source('/home/biocbuild/BBS/utils/build_pkg_dep_graph.R');build_pkg_dep_graph('target_pkgs.txt',outfile='pkg_dep_graph.txt')" returned an error (127) => EXIT. >

Hervé Pagès (18:09:15) (in thread): > /home/biocbuild/R/R-devel_2024-01-16_r85812/bin/Rscript: Is that really the path to Rscript on kunpeng2? This is old! The 3.20 builds should use the released R 4.4.0 (released about 2 months ago), or even better R 4.4.1 (released last Friday). Please update. > Also may I suggest that you install R at a fixed location? This means thatBBS_R_HOMEinBBS/3.20/bioc/kunpeng2/config.shshould be set to something like > > export BBS_R_HOME="/home/biocbuild/R" > > or > > export BBS_R_HOME="$BBS_WORK_TOPDIR/R" # <-- that's how it's set on nebbiolo2, see BBS/3.20/bioc/nebbiolo2/config.sh > > rather than > > export BBS_R_HOME="/home/biocbuild/R/R-devel_2024-01-16_r85812" > > That way you won’t need to submit a PR each time you update R on kunpeng2. > Thanks!

2024-06-20

Martin Grigorov (07:04:43) (in thread): > It was using R-4.4.0 but I have a crontab job that updates (git pull –rebase) BBS and it doesgit stash && git pull --rebase && git stash pop. I didn’t send a PR before I also need to set custom timeout for kunpeng2 and the stashing was needed anyway …

Martin Grigorov (07:05:39) (in thread): > I think I have used ~/R as a symlink to the actual version in the past but you recommended against it. I will change it now!

Martin Grigorov (07:07:44) (in thread): > is there a way to use custom Renviron.bioc for kunpeng2 only ? > > diff --git a/3.20/Renviron.bioc b/3.20/Renviron.bioc > index ce07e160..accc5f6e 100644 > --- a/3.20/Renviron.bioc > +++ b/3.20/Renviron.bioc > @@ -55,7 +55,7 @@ *R_CHECK_SUGGESTS_ONLY*=true > # Raise an error if using ordered comparisons on language objects > _R_COMPARE_LANG_OBJECTS=eqonly > > -R_DEFAULT_INTERNET_TIMEOUT=600 > +R_DEFAULT_INTERNET_TIMEOUT=6000 >

Martin Grigorov (07:11:00) (in thread): > https://github.com/Bioconductor/BBS/pull/410

Hervé Pagès (12:02:30) (in thread): > Thanks! > > I think I have used ~/R as a symlink to the actual version in the past but you recommended against it. > I’d be surprised. I use symlinks a lot on my own laptop to manage many different versions of R/BioC:wink:

Martin Grigorov (13:06:02) (in thread): > is there a way to use custom Renviron.bioc for kunpeng2 only ? I.e. to add overwrites which should be applied on top of BBS/3.20/Renviron,bioc

Hervé Pagès (14:52:37) (in thread): > No easy way to do that. It’s actually a feature to use the same Renviron.bioc everywhere and to expose it to the user. The goal is to try to make things as reproducible as possible by avoiding hidden settings.

2024-06-28

Martin Grigorov (06:12:32) (in thread): > Are there any other issues with kunpeng2 ? When it could be re-added tohttps://bioconductor.org/checkResults/3.20/bioc-LATEST/long-report.html?

Hervé Pagès (10:07:05) (in thread): > I believe that@Andres Wokatytook care of it already. Should happen today or tomorrow.

Andres Wokaty (10:21:39) (in thread): > I’ve added it back, but I realized I forgot to turn on the cronjob to collect the build products. So I may run the report without kungpeng2 then run it again to include kunpeng2 when collection is finished today. kungpeng2 will be included in the reports going forward.

2024-07-03

Andres Wokaty (13:52:09): > @Martin GrigorovI updated the builders to R 4.4.1. I know kunpeng2 recently rejoined so I think it’s okay to update R on kunpeng2 when you have time.

Martin Grigorov (15:46:19) (in thread): > I’llupdate it tomorrow!

2024-07-04

Martin Grigorov (07:22:33) (in thread): > > biocbuild@kunpeng2 ~/R> R --version > R version 4.4.0 (2024-04-24) -- "Puppy Cup" > Copyright (C) 2024 The R Foundation for Statistical Computing > Platform: aarch64-unknown-linux-gnu > > R is free software and comes with ABSOLUTELY NO WARRANTY. > You are welcome to redistribute it under the terms of the > GNU General Public License versions 2 or 3. > For more information about these matters see[https://www.gnu.org/licenses/](https://www.gnu.org/licenses/). > > biocbuild@kunpeng2 ~/R> unlink R > biocbuild@kunpeng2 ~/R> R --version > /home/biocbuild/bin/R: line 15: /home/biocbuild/R/R/bin/R: No such file or directory > biocbuild@kunpeng2 ~/R [127]> ln -s R-4.4.1 R > biocbuild@kunpeng2 ~/R> R --version > R version 4.4.1 (2024-06-14) -- "Race for Your Life" > Copyright (C) 2024 The R Foundation for Statistical Computing > Platform: aarch64-unknown-linux-gnu > > R is free software and comes with ABSOLUTELY NO WARRANTY. > You are welcome to redistribute it under the terms of the > GNU General Public License versions 2 or 3. > For more information about these matters see[https://www.gnu.org/licenses/](https://www.gnu.org/licenses/). > > Done!

Martin Grigorov (07:25:51) (in thread): > I started a new build manually, so it can download the packages. Then I will see the failed downloads and install them manually !

2024-07-09

Hervé Pagès (20:27:28): > @Martin GrigorovCan you check the status of kunpeng2? It doesn’t seem to be producing any build artefacts.

2024-07-10

Martin Grigorov (02:15:46) (in thread): > I can’t reach the VM today … I’ll ask someone to reboot it!

Martin Grigorov (03:52:44) (in thread): > > biocbuild@kunpeng2 ~/B/3/b/kunpeng2 (devel)> ./run.sh > > BBS> ============================================================== > BBS> [STAGE2] STARTING STAGE2 at Wed Jul 10 07:41:35 2024 > BBS> products-out subdir: /home/biocbuild/bbs-3.20-bioc/products-out/install > BBS> Start cloning CRAN-style package repo[http://149.165.154.78/BBS/3.20/bioc/src/contrib/](http://149.165.154.78/BBS/3.20/bioc/src/contrib/)to /home/biocbuild/bbs-3.20-bioc/target-repo/ ... > BBS> o downloading PACKAGES ... > > it seems to hang early …

Martin Grigorov (03:54:32) (in thread): > > wget[http://149.165.154.78/BBS/3.20/bioc/src/contrib/](http://149.165.154.78/BBS/3.20/bioc/src/contrib/)--2024-07-10 07:44:09--[http://149.165.154.78/BBS/3.20/bioc/src/contrib/](http://149.165.154.78/BBS/3.20/bioc/src/contrib/)Connecting to 149.165.154.78:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: unspecified [text/html] > Saving to: 'index.html' > > index.html [ <=> ] 1.51M 1.10MB/s in 1.4s > > 2024-07-10 07:44:11 (1.10 MB/s) - 'index.html' saved [1581522] > > biocbuild@kunpeng2 ~/B/3/b/kunpeng2 (devel)> tail index.html > <tr><td valign="top"><img src="/icons/text.gif" alt="[TXT]"></td><td><a href="zitools.buildnovig-out.txt">zitools.buildnovig-out.txt</a></td><td align="right">2024-07-09 14:52 </td><td align="right">400 </td><td> </td></tr> > <tr><td valign="top"><img src="/icons/unknown.gif" alt="[ ]"></td><td><a href="zitools.buildnovig-summary.dcf">zitools.buildnovig-summary.dcf</a></td><td align="right">2024-07-09 14:52 </td><td align="right">368 </td><td> </td></tr> > <tr><td valign="top"><img src="/icons/compressed.gif" alt="[ ]"></td><td><a href="zitools_0.99.3.tar.gz">zitools_0.99.3.tar.gz</a></td><td align="right">2024-07-09 14:52 </td><td align="right"> 32K</td><td> </td></tr> > <tr><td valign="top"><img src="/icons/text.gif" alt="[TXT]"></td><td><a href="zlibbioc.buildnovig-out.txt">zlibbioc.buildnovig-out.txt</a></td><td align="right">2024-07-09 14:52 </td><td align="right">402 </td><td> </td></tr> > <tr><td valign="top"><img src="/icons/unknown.gif" alt="[ ]"></td><td><a href="zlibbioc.buildnovig-summary.dcf">zlibbioc.buildnovig-summary.dcf</a></td><td align="right">2024-07-09 14:52 </td><td align="right">372 </td><td> </td></tr> > <tr><td valign="top"><img src="/icons/compressed.gif" alt="[ ]"></td><td><a href="zlibbioc_1.51.1.tar.gz">zlibbioc_1.51.1.tar.gz</a></td><td align="right">2024-07-09 14:52 </td><td align="right">166K</td><td> </td></tr> > <tr><th colspan="5"><hr></th></tr> > </table> > <address>Apache/2.4.52 (Ubuntu) Server at 149.165.154.78 Port 80</address> > </body></html> > > reaching the repo is not the problem

Martin Grigorov (04:22:30) (in thread): > it runs now, but it looks slow …

Martin Grigorov (05:16:56) (in thread): > I’m waiting for a response from the cloud services support team

Hervé Pagès (13:14:41) (in thread): > ok, thanks for checking

2024-07-11

Sathish Kumar (06:05:41): > @Sathish Kumar has joined the channel

Martin Grigorov (08:04:37) (in thread): > the support team confirmed that there is a problem but there is no solution yet

2024-07-18

Martin Grigorov (02:58:57) (in thread): > https://serverfault.com/a/670081- this seems to fixed the connectivity issue! I’ve started a build! - Attachment (Server Fault): Cannot SSH: debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY > We have a server on Amazon EC2 running SSH is on a standard (22) port. > I placed my public key at the /.ssh/authorized_keys file. > The fun thing is that yesterday it was working great…

2024-07-24

Martin Grigorov (02:51:14) (in thread): > > biocbuild@kunpeng2 ~/b/log> tail -f kunpeng2-20240721-run.log > BBS> o Working dir: /home/biocbuild/bbs-3.20-bioc/meat > BBS> o 2101 srcpkg file(s) in working dir > BBS> o 2101 srcpkg file(s) queued and processed > BBS> o Total time: 30411.81 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Mon Jul 22 12:31:44 2024. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.20-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. > ^[[A^C⏎ biocbuild@kunpeng2 ~/b/log [SIGINT]> tail kunpeng2-20240722-run.log > BBS> o Working dir: /home/biocbuild/bbs-3.20-bioc/meat > BBS> o 2105 srcpkg file(s) in working dir > BBS> o 2105 srcpkg file(s) queued and processed > BBS> o Total time: 30577.68 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Tue Jul 23 11:22:52 2024. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.20-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. > biocbuild@kunpeng2 ~/b/log> tail kunpeng2-20240723-run.log > bbs.jobs.processJobQueue> Command: /home/biocbuild/R/R/bin/R CMD check --install=check:lipidr.install-out.txt --library=/home/biocbuild/R/R/site-library --no-vignettes --timings lipidr_2.19.0.tar.gz > > bbs.jobs.processJobQueue> DONE JOB IsoBayes (981/2106) ON SLOT 22/22 after 555.09 seconds [retcode=0] > BBS> No such process: 2376966 > BBS> Copying IsoBayes.Rcheck to /home/biocbuild/bbs-3.20-bioc/products-out/checksrc/ ... OK > BBS> Copying IsoBayes.checksrc-out.txt to /home/biocbuild/bbs-3.20-bioc/products-out/checksrc/ ... OK > BBS> Copying IsoBayes.checksrc-summary.dcf to /home/biocbuild/bbs-3.20-bioc/products-out/checksrc/ ... OK > > bbs.jobs.processJobQueue> START JOB LiquidAssociation (1026/2106) ON SLOT 22/22 > bbs.jobs.processJobQueue> Command: /home/biocbuild/R/R/bin/R CMD check --install=check:LiquidAssociation.install-out.txt --library=/home/biocbuild/R/R/site-library --no-vignettes --timings LiquidAssociation_1.59.0.tar.gz > > all looks good again!

Martin Grigorov (02:51:46) (in thread): > @Hervé Pagès@Andres WokatyI think kunpeng2 could be added back to the list of builders

2024-07-30

Andres Wokaty (15:02:50) (in thread): > Sorry for the delay. I’ve added kunpeng2 back and we should see it in the reports again starting from tomorrow’s report.

Martin Grigorov (16:08:56) (in thread): > Thank you,@Andres Wokaty!

2024-08-02

Andres Wokaty (11:06:42): > Hi@Martin Grigorov, I’m getting rsync errors when trying to fetch from kungpeng2. I wonder if it is because it’s rejecting the machine because the IP is different? About 2 weeks ago we had to associate a new IP with bbscentral2, which is now149.165.152.87because the previous one got accidentally unallocated by the hosting service. Can you make sure this IP is allowed to rsync on kungpeng2?

Martin Grigorov (11:40:39): > Pretty sure this is the reason!I’llrequest an update of the firewall rule!

2024-08-05

Martin Grigorov (03:47:38): > @Andres WokatyThe new IP has been added to the allowed ones! Please check!

Andres Wokaty (12:51:58) (in thread): > Thank you! I’ve added kunpeng2 to the report again. It should appear on tomorrow’s report.

Martin Grigorov (13:08:53) (in thread): > Great! Thank you!

2024-10-09

Andres Wokaty (12:55:15): > Hi@Martin Grigorovcould you install meme suite on kunpeng2 when convenient? You can followhttps://github.com/Bioconductor/BBS/pull/422/filesas an example.

Martin Grigorov (13:12:55): > Sure!I’lldo it tomorrow!

2024-10-10

Martin Grigorov (02:48:22): > The installation went well! > > biocbuild@kunpeng2 ~/git [1]> ll /opt/meme/ > total 16K > drwxr-xr-x 2 root root 4.0K Oct 10 06:09 bin > drwxr-xr-x 3 root root 4.0K Oct 10 06:09 lib > drwxr-xr-x 3 root root 4.0K Oct 10 06:09 libexec > drwxr-xr-x 3 root root 4.0K Oct 10 06:09 share > biocbuild@kunpeng2 ~/git> > biocbuild@kunpeng2 ~/git> ll /opt/meme/bin/ > total 27M > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 ame > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 centrimo > -rwxr-xr-x 1 root root 74K Oct 10 06:09 dreme > -rwxr-xr-x 1 root root 72K Oct 10 06:09 dust > -rwxr-xr-x 1 root root 2.2M Oct 10 06:09 fimo > -rwxr-xr-x 1 root root 1.9M Oct 10 06:09 glam2 > -rwxr-xr-x 1 root root 149K Oct 10 06:09 glam2scan > -rwxr-xr-x 1 root root 2.2M Oct 10 06:09 gomo > -rwxr-xr-x 1 root root 1.9M Oct 10 06:09 mast > -rwxr-xr-x 1 root root 2.4M Oct 10 06:09 mcast > -rwxr-xr-x 1 root root 2.1M Oct 10 06:09 meme > -rwxr-xr-x 1 root root 68K Oct 10 06:09 meme-chip > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 momo > -rwxr-xr-x 1 root root 78K Oct 10 06:09 purge > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 sea > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 spamo > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 streme > -rwxr-xr-x 1 root root 297K Oct 10 06:09 tgene > -rwxr-xr-x 1 root root 2.0M Oct 10 06:09 tomtom > -rwxr-xr-x 1 root root 70K Oct 10 06:09 xstreme > > but the build of the package fails with: > > biocbuild@kunpeng2 ~/git> git clone[https://git.bioconductor.org/packages/MotifPeeker](https://git.bioconductor.org/packages/MotifPeeker)... > ... > biocbuild@kunpeng2 ~/git> R CMD build MotifPeeker > * checking for file 'MotifPeeker/DESCRIPTION' ... OK > * preparing 'MotifPeeker': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'MotifPeeker.Rmd' using rmarkdown > sh: -c: line 1: syntax error near unexpected token `(' > sh: -c: line 1: `'echo' Using MulticoreParam object for BiocParallel (workers = 2).' > sh: -c: line 1: syntax error near unexpected token `(' > sh: -c: line 1: `'echo' Using MulticoreParam object for BiocParallel (workers = 2).' > Starting STREME run for 4 sets of sequences. > sh: -c: line 1: syntax error near unexpected token `(' > sh: -c: line 1: `'echo' Using MulticoreParam object for BiocParallel (workers = 2).' > STREME run complete. > sh: -c: line 1: syntax error near unexpected token `(' > sh: -c: line 1: `'echo' Using MulticoreParam object for BiocParallel (workers = 2).' > Generating download buttons... > > Quitting from lines 193-212 [run-motifpeeker] (MotifPeeker.Rmd) > Error: processing vignette 'MotifPeeker.Rmd' failed with diagnostics: > there is no package called 'downloadthis' > --- failed re-building 'MotifPeeker.Rmd' > > --- re-building 'troubleshooting.Rmd' using rmarkdown > --- finished re-building 'troubleshooting.Rmd' > > SUMMARY: processing the following file failed: > 'MotifPeeker.Rmd' > > Error: Vignette re-building failed. > Execution halted >

Lori Shepherd (10:05:01): > Interesting – the package does properly list it in the DESCRIPTION and it does exist on CRANhttps://cran.r-project.org/web/packages/downloadthis/index.htmlis there some way to see if somehow downloadthis did not install properly?

2024-10-14

Charlotte Soneson (01:42:58): > @Charlotte Soneson has joined the channel

Charlotte Soneson (01:52:47): > At the moment I’m seeing this error during vignette building for several of ourbasilisk-using packages (e.g.sketchR,velociraptor) onkunpeng2: > > OSError: Could not find/load shared object file > > The other platforms don’t seem to have this issue. Is there anything we should do on our side to address this?

Martin Grigorov (01:54:40): > I’ll take a look !

Martin Grigorov (03:05:12): > it is not very clear what the problem is. It is easy to reproduce but the lack of details makes it hard to debug …

Martin Grigorov (03:58:59): > I will need help by someone more familiar with reticulate/basilisk and/or sketchR to debug the issue

Charlotte Soneson (04:04:48): > Happy to try to help - what can I do? From thesketchRside, it fails in the chunk on lines 145-149 in the vignette, which is not the first place where thebasiliskenvironment is used, so I would think that the environment construction/activation etc is fine. Forvelociraptorit fails in the chunk on lines 76-82, which is also a chunk that uses python. Perhaps one of the python dependencies is not compatible with the platform.

Martin Grigorov (04:06:31): > I can try issueing commands in the REPL until something breaks and gives more details which library is missing. Any ideas for such command ?

Charlotte Soneson (04:09:24): > ForsketchR, the R-side command that fails is > > idx800scs <- scsampler(SingleCellExperiment::reducedDim(pbmc3k, "PCA"), > N = 800, seed = 123) > > which under the hood should do the following in python: > > import scsampler as scs > scs.scsampler(mat, n_obs = 800, copy = FALSE, random_state = 123, random_split = 1) > > wherematis a 2d numpy array.

Martin Grigorov (04:11:16): > locally it fails with: > > biocbuild@kunpeng2 ~/git [127]> R CMD build sketchR > * checking for file 'sketchR/DESCRIPTION' ... OK > * preparing 'sketchR': > * checking DESCRIPTION meta-information ... OK > * installing the package to build vignettes > * creating vignettes ... ERROR > --- re-building 'sketchR.Rmd' using rmarkdown > > Quitting from lines 134-138 [unnamed-chunk-4] (sketchR.Rmd) > Error: processing vignette 'sketchR.Rmd' failed with diagnostics: > ModuleNotFoundError: No module named 'rpytools' > Run `reticulate::py_last_error()` for details. > --- failed re-building 'sketchR.Rmd' > > SUMMARY: processing the following file failed: > 'sketchR.Rmd' > > Error: Vignette re-building failed. > Execution halted > > The lines are different thanhttps://bioconductor.org/checkResults/3.20/bioc-LATEST/sketchR/kunpeng2-buildsrc.html

Martin Grigorov (04:11:39): > it needsrpytools

Martin Grigorov (04:12:59): > grep -rnHi rpytools sketchRreturns nothing

Charlotte Soneson (04:14:45): > Hm, interesting - could it be an issue with different PYTHONPATHs (https://github.com/rstudio/reticulate/issues/1570)?

Martin Grigorov (04:16:32): > it could be! Usuallykunpeng2does not useexport RETICULATE_PYTHON=...but for debugging I tried with it (/home/biocbuild/.pyenv/versions/3.10.15/bin/python3).Let me unset it !

Martin Grigorov (04:19:35): > yes, this was the reason. Now it fails with the same error: > > Quitting from lines 145-149 [unnamed-chunk-5] (sketchR.Rmd) > Error: processing vignette 'sketchR.Rmd' failed with diagnostics: > OSError: Could not find/load shared object file >

Martin Grigorov (04:23:30): > I am copying the commands from sketchR.rmd one by one in the REPL

Martin Grigorov (04:27:10): > > > idx800scs <- scsampler(SingleCellExperiment::reducedDim(pbmc3k, "PCA"), N = 800, seed = 123) > Error in checkForRemoteErrors(lapply(cl, recvResult)) : > one node produced an error: OSError: Could not find/load shared object file > Run `reticulate::py_last_error()` for details. > > reticulate::py_last_error() > > ── Python Exception Message ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── > Traceback (most recent call last): > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 122, in _find_and_load_hook > return _run_hook(name, _hook) > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 96, in _run_hook > module = hook() > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 120, in _hook > return *find_and_load(name, import*) > ModuleNotFoundError: No module named 'geosketch' > > ── R Traceback ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── > ▆ > 1. └─reticulate::import("geosketch") > 2. └─reticulate:::py_module_import(module, convert = convert) > See `reticulate::py_last_error()$r_trace$full_call` for more details. >

Martin Grigorov (04:28:06): > ideas what to do now ?

Charlotte Soneson (04:30:10): > Hm, interesting - thegeosketchmodule was supposedly used in the previous code chunk:thinking_face:

Charlotte Soneson (04:30:42): > I have to run to a meeting now, sorry - will come back in a bit

Martin Grigorov (04:31:54): > OK!

Martin Grigorov (04:32:29): > geosketch()is not used in a previous code snippet.

Charlotte Soneson (04:33:05): > It should be - on line 134-138

Martin Grigorov (04:34:01): > oh, right! let me run it

Martin Grigorov (04:35:46): > > > idx800gs <- geosketch(SingleCellExperiment::reducedDim(pbmc3k, "PCA"), N = 800, seed = 123) > > idx800scs <- scsampler(SingleCellExperiment::reducedDim(pbmc3k, "PCA"), N = 800, seed = 123) > Error in checkForRemoteErrors(lapply(cl, recvResult)) : > one node produced an error: OSError: Could not find/load shared object file > Run `reticulate::py_last_error()` for details. > > reticulate::py_last_error() > > ── Python Exception Message ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── > Traceback (most recent call last): > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 122, in _find_and_load_hook > return _run_hook(name, _hook) > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 96, in _run_hook > module = hook() > File "/home/biocbuild/R/R-4.4.1/site-library/reticulate/python/rpytools/loader.py", line 120, in _hook > return *find_and_load(name, import*) > ModuleNotFoundError: No module named 'geosketch' > > ── R Traceback ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── > ▆ > 1. └─reticulate::import("geosketch") > 2. └─reticulate:::py_module_import(module, convert = convert) > See `reticulate::py_last_error()$r_trace$full_call` for more details. >

Martin Grigorov (04:36:10): > geosketch(...)passes, butscsampler(...)fails …

Martin Grigorov (04:37:12): > > > head(idx800gs) > [1] 3 4 5 6 10 11 > > length(idx800gs) > [1] 800 > > all seems good withgeosketch(...)

Charlotte Soneson (06:55:28): > (Quick break between meetings) A couple of other things that may be worth trying: > * Running thescsampler()line without first runninggeosketch()- I’m confused as to why the former tries to loadgeosketch > * Perhapsreticulate::py_last_error()$r_trace$full_callwill give some more ideas > * And maybe trying the same with the code in thevelociraptorvignette will give some more hints - it also raises the same kind of error, but doesn’t have eithergeosketchorscsamplerin the environment (there are many other shared python though).

Michael Stadler (07:01:06): > @Michael Stadler has joined the channel

2024-10-22

Lori Shepherd (14:35:13): > @Martin Grigorovthere was a question on our bioc-devel mailing list for ANCOMBChttps://bioconductor.org/checkResults/3.20/bioc-LATEST/ANCOMBC/kunpeng2-install.htmlR package CVXR is not available but is listed on CRANhttps://cran.r-project.org/web/packages/CVXR/index.html. Can you check to see if there was a reason it might not have installed properly?

Martin Grigorov (15:19:36) (in thread): > Sure!I’lltake a look tomorrow morning!

2024-10-23

Martin Grigorov (02:34:24) (in thread): > Fixed it! > The builder was missing Rust to build some dependencies. > I replied on the mailing list!

2024-11-19

Martin Grigorov (01:43:05): > @Andres WokatyIs it time to update kunpeng to 3.21 ?https://bioconductor.org/checkResults/3.21/bioc-LATEST/long-report.html

2024-11-20

Andres Wokaty (16:10:09) (in thread): > I’ve removed kunpeng2 from the report. Let me know when I should add it to 3.21.

2024-11-26

Martin Grigorov (03:12:05) (in thread): > I just noticed your response! I will work on this today!

Martin Grigorov (05:35:53) (in thread): > OK. I think it is ready! R is updated to R-devel from 2024-11-24. Rsyncd is updated to deliver 3.21, and the cron job is updated to run 3.21

Andres Wokaty (10:28:17) (in thread): > Thanks! I’ve set up the central builder to rsync kunpeng2 artifacts.

Martin Grigorov (10:29:56) (in thread): > Great! > Some big packages are expected to fail in the first few runs!I’llkeep an eye!

2024-11-27

Andres Wokaty (10:14:24) (in thread): > @Martin GrigorovI am experiencing an rsync failure trying to fetch from kunpeng2. I wonder if the IP is allowed: 149.165.171.124?

Martin Grigorov (10:15:32) (in thread): > Maybe it is not.Is it a new IP since 3.20 ?

Andres Wokaty (10:19:32) (in thread): > This IP isn’t new but it is a different machine.

Martin Grigorov (10:20:04) (in thread): > I’llrequest to add it the firewall rules

Andres Wokaty (10:20:09) (in thread): > Thank you!

Andres Wokaty (10:20:49) (in thread): > I thought maybe it would already be in the firewall rules but maybe we haven’t completed a full cycle with both of the machines yet.

2024-12-02

Martin Grigorov (10:37:50) (in thread): > @Andres WokatyI’vebeen told that the IP is added to the rules.Please try again!

Andres Wokaty (10:40:20) (in thread): > Thank you. I am trying. I think there is also a configuration issue on my end.

Andres Wokaty (11:05:46) (in thread): > Does the rsync module114.119.187.96::biocpoint to/home/biocbuild/bbs-3.21-bioc/products-out? > > When I tryrsync, I get > > @ERROR: chroot failed > rsync error: error starting client-server protocol (code 5) at main.c(1863) [Receiver=3.2.7] >

Martin Grigorov (11:08:45) (in thread): > > [bioc] > path = /home/biocbuild/bbs-3.21-bioc/products-out/ > comment = Bioconductor 3.21 report output > read only = yes > alist = yes > #dont compress = *.dcf *.txt > #auth users = bioc > #secrets file = /etc/rsync.bioc.secrets >

Martin Grigorov (11:09:24) (in thread): > > root@kunpeng2 ~# systemctl status rsyncd.service > ● rsyncd.service - fast remote file copy program daemon > Loaded: loaded (/usr/lib/systemd/system/rsyncd.service; enabled; vendor preset: disabled) > Active: active (running) since Tue 2024-11-26 08:31:54 UTC; 6 days ago > Main PID: 2199867 (rsync) > Tasks: 1 (limit: 410321) > Memory: 720.0K > CGroup: /system.slice/rsyncd.service > └─ 2199867 /usr/bin/rsync --daemon --no-detach > > Nov 26 08:31:54 kunpeng2 systemd[1]: Started fast remote file copy program daemon. >

Martin Grigorov (11:09:47) (in thread): > > root@kunpeng2 ~# netstat -antp | grep 873 > tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 2199867/rsync > tcp6 0 0 :::873 :::* LISTEN 2199867/rsync >

Andres Wokaty (11:10:26) (in thread): > Thanks for sharing that. I’m not sure what if any configurations should be made on this machine, but I pinged@Hervé Pagèsabout it in case I’m missing something.

Martin Grigorov (11:11:18) (in thread): > for 3.21 I’ve just updated thepathand restarted the service

Martin Grigorov (11:12:15) (in thread): > you can trynmap -p 873 the-IP-addrto see whether the firewall rule is OK

Andres Wokaty (11:15:34) (in thread): > I can reach it. > > nmap -p 878 114.119.187.96 > Starting Nmap 7.80 ([https://nmap.org](https://nmap.org)) at 2024-12-02 11:14 EST > Nmap scan report for ecs-114-119-187-96.compute.hwclouds-dns.com (114.119.187.96) > Host is up (0.21s latency). > > PORT STATE SERVICE > 878/tcp closed unknown > > Nmap done: 1 IP address (1 host up) scanned in 0.77 seconds >

Martin Grigorov (11:15:50) (in thread): > 873

Andres Wokaty (11:16:45) (in thread): > thanks! > > nmap -p 873 114.119.187.96 > Starting Nmap 7.80 ([https://nmap.org](https://nmap.org)) at 2024-12-02 11:16 EST > Nmap scan report for ecs-114-119-187-96.compute.hwclouds-dns.com (114.119.187.96) > Host is up (0.22s latency). > > PORT STATE SERVICE > 873/tcp open rsync > > Nmap done: 1 IP address (1 host up) scanned in 0.46 seconds >

Martin Grigorov (11:17:01) (in thread): > looks good!

Andres Wokaty (11:23:20) (in thread): > unfortunately, i still get a chroot error. i will try again later.

Andres Wokaty (13:21:15) (in thread): > Hi@Martin GrigorovHerve said I don’t need any special settings. He checkedbbscentral2, which had previously rsynced fromkunpeng2and now it also has the error. Have other settings changed for rsync?

Hervé Pagès (13:29:27) (in thread): > rsyncsees thebiocmodule but the module seems somehow misconfigured: > > biocbuild@nebbiolo1:~$ rsync 114.119.187.96:: > bioc Bioconductor 3.21 report output > > @Martin GrigorovAre you sure the path associated with the module is valid?

Martin Grigorov (13:30:01) (in thread): > please try again

Andres Wokaty (13:30:47) (in thread): > I think it is working now

Andres Wokaty (13:31:11) (in thread): > At least no error, but I can check again when files are available.

Martin Grigorov (13:31:31) (in thread): > I justtouched a file in the folder -content.txt

Andres Wokaty (13:31:39) (in thread): > weird

Martin Grigorov (13:32:37) (in thread): > I manually created the folder. I expected that the runner will create it. I will add it to doc with the upgrade steps for 3.22

Martin Grigorov (13:34:20) (in thread): > > BBS> ============================================================== > BBS> [STAGE2] STARTING STAGE2 at Mon Dec 2 18:20:23 2024 > BBS> [waitForTargetRepoToBeReady] Unable to access[http://149.165.171.124/BBS/3.21/bioc/src/contrib/PACKAGES](http://149.165.171.124/BBS/3.21/bioc/src/contrib/PACKAGES). Looks like the target repo is not ready yet! > BBS> [waitForTargetRepoToBeReady] -> will wait 3 minutes before trying again ... > > It seems I need to update config.sh ?!

Martin Grigorov (13:34:30) (in thread): > diff-ing it against the others

Andres Wokaty (13:35:08) (in thread): > So, the prerun, which makes the repo, hasn’t started yet.

Hervé Pagès (13:35:54) (in thread): > Yes BBS should take care of creating the folder. The folder is typically located at/home/biocbuild/bbs-3.21-bioc/products-out/(that’s where it is on our builders).

Martin Grigorov (13:36:14) (in thread): > I use the same folder names

Martin Grigorov (13:38:08) (in thread): > Should I change some setting to solveBBS> [waitForTargetRepoToBeReady] Unable to accesshttp://149.165.171.124/BBS/3.21/bioc/src/contrib/PACKAGES. Looks like the target repo is not ready yet!?

Hervé Pagès (13:38:51) (in thread): > No. You just started the builds too early. Kill them and let them start at the usual time.

Martin Grigorov (13:39:52) (in thread): > I have the cron job since a week but it seems it didn’t start at all so far. That’s why I tried to run it manually to see whether there are any errors.

Martin Grigorov (13:40:39) (in thread): > another question: I see the other builders useexport BBS_PRODUCT_TRANSMISSION_MODE="asynchronous",kunpeng2usesnone. Should I change it ?

Hervé Pagès (13:40:57) (in thread): > If the cron job didn’t start at all so far that explains why/home/biocbuild/bbs-3.21-bioc/products-out/was never created:wink:

Martin Grigorov (13:41:14) (in thread): > or maybe this is because kunpeng2 is an “external” builder ?!

Martin Grigorov (13:42:55) (in thread): > I think the cron jobs didn’t start because ~/bbs-3.21-bioc/log/ is not existing too

Hervé Pagès (13:43:07) (in thread): > External builders create/home/biocbuild/bbs-3.21-bioc/products-out/like all other builders. > But you’re right:export BBS_PRODUCT_TRANSMISSION_MODE="asynchronous"is bad. Should be set tononeinstead.@Andres Wokaty?

Andres Wokaty (13:43:21) (in thread): > yes, the log should exist.

Hervé Pagès (13:44:39) (in thread): > yep, seehttps://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-Ubuntu-22.04-HOWTO.md#create-bbs-xy-bioc-directory-structure

Andres Wokaty (13:45:00) (in thread): > the configuration is set tononein the repo:https://github.com/Bioconductor/BBS/blob/devel/3.21/bioc/kunpeng2/config.sh

Martin Grigorov (13:45:31) (in thread): > kunpeng2 usesnonebut nebbiolo1 and kjohnson3 usesasync

Andres Wokaty (13:45:57) (in thread): > yes, because they retrieve products different ways, but kunpeng2 should benone

Martin Grigorov (13:46:05) (in thread): > that’s why I asked whether I have to change it too

Martin Grigorov (13:46:07) (in thread): > got it !

Hervé Pagès (13:46:29) (in thread): > Oh, I misunderstood what@Martin Grigorovwas saying. kunpeng2 usesnoneand that is what it should use because it’s an external builder.

Martin Grigorov (13:47:52) (in thread): > I just rebooted the VM. It should be ready for the cron run

Andres Wokaty (13:49:13) (in thread): > I will check in a few hours ifbbscentral1is able to get files fromkunpeng2. if it is, i will add it to tomorrow’s report.

Martin Grigorov (13:49:30) (in thread): > thanks!

Andres Wokaty (18:01:35) (in thread): > I see that we’re getting files finally. I’ve added it back to the report, although sometimes the first run takes a little longer.

2024-12-03

Martin Grigorov (02:19:56) (in thread): > the run finished successfully: > > bbs.jobs.processJobQueue> Finished. > > BBS> END STAGE4 loop. > BBS> ------------------------------------------------------------- > BBS> STAGE4 SUMMARY: > BBS> o Working dir: /home/biocbuild/bbs-3.21-bioc/meat > BBS> o 204 srcpkg file(s) in working dir > BBS> o 204 srcpkg file(s) queued and processed > BBS> o Total time: 1022.55 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Tue Dec 3 01:42:19 2024. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.21-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. >

Martin Grigorov (02:20:22) (in thread): > I will see which big packages timed out and install them manually !

Andres Wokaty (10:19:57) (in thread): > Great. I see that we got all the files. It should be on the report.

2024-12-04

Martin Grigorov (05:45:33): > @Hervé PagèsRhtslib fails to compile on ARM64 with this error: > > gcc -g -O2 -fpic -fvisibility=hidden -I. -I/usr/local/include -D_FILE_OFFSET_BITS=64 -c -o htscodecs/htscodecs/rANS_static4x16pr.o htscodecs/htscodecs/rANS_static4x16pr.c > htscodecs/htscodecs/rANS_static4x16pr.c: In function 'have_neon': > htscodecs/htscodecs/rANS_static4x16pr.c:1022:35: error: 'HWCAP_ASIMD' undeclared (first use in this function) > 1022 | return (getauxval(AT_HWCAP) & HWCAP_ASIMD) != 0; > | ^~~~~~~~~~~~~~~~~~~~~ > htscodecs/htscodecs/rANS_static4x16pr.c:1022:35: note: each undeclared identifier is reported only once for each function it appears in > > At Bioconda this problem is worked around with the following patch -https://github.com/bioconda/bioconda-recipes/blob/master/recipes/htslib/arm_hwcap.patchHow to solve it here ?

Martin Grigorov (06:03:02) (in thread): > https://github.com/samtools/htscodecs/issues/88

Martin Grigorov (06:04:25) (in thread): > I think the issue comes from the fact that I installed GCC 14 from Conda … Now it uses Conda’s sysroot with GLibc 2.17https://github.com/samtools/htscodecs/issues/88

Martin Grigorov (13:16:29) (in thread): > I’ve downgraded GCC to 12.x to be able to install Rhtslib and the other packages. Once they are installed I will build GCC 14.2.x from sources

Hervé Pagès (14:16:51) (in thread): > Thanks! This code is from the HTSlib project:https://github.com/samtools/htslib(Rhtslibis just a wrapper around HTSlib). IIRCRhtslibcompiles fine on all other patforms and used to compile fine on kunpeng2, until now. Also the package has not changed in more than a year so not sure what’s going on.

Martin Grigorov (14:46:41) (in thread): > It is a problem with Conda-forge gcc/gxx/sysroot packages

2024-12-12

Lori Shepherd (09:30:18): > @Martin GrigorovJust looking at the latest build report and it looks like almost all packages are ERROR at one stage or another on kunpeng2. Do we know the reason for the catastrophic failures?

Martin Grigorov (09:54:43): > No idea.I’llcheck later!

Martin Grigorov (11:32:58): > The problem is the old version of GCC (10.x) and libstdc++. Let me re-build R 4.5.0 with GCC 12.x and see whether it will help

Martin Grigorov (14:05:25): > It should be better tomorrow! Installed GCC 14.2.0 with a newer libstdc++ and several of the failed packages now passed !

2024-12-16

Lori Shepherd (10:39:50): > https://support.bioconductor.org/p/9160872/

Martin Grigorov (10:42:23): > I’veupgraded the OS to 24.03 LTS and now there are two libs which are still older than required- libcurl and libicui18n.I’llbuild them from sources tomorrow!

Martin Grigorov (10:43:25): > I’llreply at the forum too!

Hervé Pagès (14:22:01): > Thanks@Martin GrigorovIn the meantime I’m excluding kunpeng2 from the report generation (the machine seems to be in a state that breaks the report generation).

2024-12-17

Martin Grigorov (02:35:36): > > * installing **source** package 'ReUseData' ... > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > Error in dyn.load(file, DLLpath = DLLpath, ...) : > unable to load shared object '/home/biocbuild/R/R-4.5.0-devel_2024-11-24/site-library/git2r/libs/git2r.so': > libgit2.so.1.3: cannot open shared object file: No such file or directory > Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load > > Do you have an idea why it tries to load.so.1.3specifically ? The system provideslibgit2.so.1.7andlibgit2.solinks to.so.1.7ifgit2r.sotried to loadlibgit2.soit would work. > I worked it around by symlinking.so.1.3to.so.1.7

Martin Grigorov (03:00:48): > Similar issue: > > * installing **source** package 'RBioFormats' ... > **** using staged installation > **** R > **** inst > **** byte-compile and prepare package for lazy loading > Error in dyn.load(file, DLLpath = DLLpath, ...) : > unable to load shared object '/home/biocbuild/R/R-4.5.0-devel_2024-11-24/site-library/tiff/libs/tiff.so': > libtiff.so.5: cannot open shared object file: No such file or directory > Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load > > The OS provides: > > /usr/lib64/libtiff.so > /usr/lib64/libtiff.so.6 > /usr/lib64/libtiff.so.6.0.2 >

Martin Grigorov (03:06:26): > I think I fixed the most common issues (libicu and libcurl). > I just started run.sh. Once it finish I will check again for failed builds

Vince Carey (12:54:09): > The way I solved the libtiff problem on ubuntu was to download the sources and install by hand. Then EBImage would install; I think RBioFormats would also be aided.

Martin Grigorov (13:21:35): > I worked around the issues by adding symlinks from .so.MISSING to .so

Martin Grigorov (13:22:25): > Only libicu was older on the system.All other missing libs were older than the provided ones

Martin Grigorov (13:23:31): > But I wonder why it looked for .so.SOMETHING instead of just .so

Vince Carey (14:13:44): > oh, interesting trick, i missed that. seems a bit risky….

2024-12-19

Martin Grigorov (03:50:09): > at the moment only 31 packages have problems: > {affxparser} > ArrayExpress > celda > cellHTS2 > ChIPXpressData > cn.mops > CopyNumberPlots > crlmm > cytomapper > DAPAR > DiffBind > enrichR > epiR > GLAD > HTqPCR > liger > oligo > pasilla > PICS > pRoloc > puma > rsbml > simpleSeg > SpatialFeatureExperiment > spatialreg > spdep > spicyR > tiledb > TumourMethData > twilight > VanillaICE >

Martin Grigorov (03:50:42): > I think it is OK to re-enable kunpeng2

Martin Grigorov (03:52:55): > @Hervé Pagès@Andres WokatyWe would like to add a second VM for building 3.20 (release branch) too. Same kind of setup askunpeng2. Is this OK with you ?

Hervé Pagès (12:13:56): > > I think it is OK to re-enable kunpeng2 > Will do. FWIW I see 214 INSTALL failures (189 BioC packages, 25 CRAN packages), 294 BUILD failures, and 83 CHECK failures today on kunpeng2. > > We would like to add a second VM for building 3.20 (release branch) too. > I’ll let Jen decide when they are back. The Linux arm64 builds are informative only i.e. they provide useful feedback to developers to help them support their package on Linux arm64. Building BioC devel on Linux arm64 achieves that. However, note that we don’t propagate anything produced by these builds. In particular we don’t produce package binaries for the Linux arm64 platform. So it’s not clear to me what benefits we hope to get by running the Linux arm64 builds on BioC 3.20 that we don’t get already from the 3.21 builds. Sounds like more work for everybody for not much added value.

Martin Grigorov (12:16:30) (in thread): > Our users want to see a list of packages which are usable on Linux ARM64 for the maintained versions of Bioconductor

Martin Grigorov (12:16:44) (in thread): > This is our reason

Hervé Pagès (12:48:09): > kunpeng2 is back:https://bioconductor.org/checkResults/3.21/bioc-LATEST/long-report.html

Martin Grigorov (12:49:50): > Thanks! Tomorrow I will see what else could be fixed !

2024-12-23

Andres Wokaty (11:33:04): > Hi@Martin Grigorov, could you installscikit-learnforseqArchRon kunpeng2 when you have time:https://bioconductor.org/checkResults/3.21/bioc-LATEST/seqArchR/kunpeng2-checksrc.html?

Martin Grigorov (12:10:10): > Sure!

Martin Grigorov (23:41:56) (in thread): > @Andres Wokatywhat do you think about having a build runner for the RELEASE branch on Linux ARM64 ?

2024-12-24

Martin Grigorov (07:06:05): > Installed it but now it fails with: > > * checking for unstated dependencies in 'tests' ... OK > * checking tests ... > Running 'testthat.R' > ERROR > Running the tests in 'tests/testthat.R' failed. > Last 13 lines of output: > ══ Failed tests ════════════════════════════════════════════════════════════════ > ── Error ('test_seqArchR_main.R:149:5'): seqArchR (cv) works when timeFlag is FALSE ── > Error: cannot find an open port. For manually specifying the port, see > ?SnowParam > Backtrace: > ▆ > 1. └─seqArchR::seqArchR(...) at test_seqArchR_main.R:149:5 > 2. └─seqArchR:::perform_setup(...) > 3. └─BiocParallel::MulticoreParam(workers = crs, tasks = crs) > 4. └─BiocParallel:::.snowPort() > 5. └─BiocParallel:::.stop("cannot find an open port. For manually specifying the port, see ?SnowParam") > > [ FAIL 1 | WARN 1 | SKIP 4 | PASS 104 ] > Error: Test failures > Execution halted > * checking for unstated dependencies in vignettes ... OK > * checking package vignettes ... OK > * checking re-building of vignette outputs ... OK > * checking PDF version of manual ... OK > * DONE > > Status: 1 ERROR >

2024-12-26

Andres Wokaty (10:29:34) (in thread): > I appreciate Hervé’s comments. While it’s some additional work for me, it’s more work for you,@Martin Grigorov. If you want to proceed, the configuration can be the same as kunpeng2. You’ll also need to allow149.165.152.87.

Martin Grigorov (10:35:46) (in thread): > Thank you,@Andres Wokaty!I’llprepare the system and let you know when I am ready!Merry Christmas!

Andres Wokaty (10:40:25) (in thread): > Happy holidays!

Andres Wokaty (14:03:48) (in thread): > I notice the build reporthttps://bioconductor.org/checkResults/3.21/bioc-LATEST/seqArchR/kunpeng2-checksrc.htmlshows it can’tsklearn. Ubuntu 24 by default uses virtual environments so I see that it’s looking along the same path on kunpeng2, but I wonder if that’s where it actually is.

Martin Grigorov (14:18:01) (in thread): > openEuler still uses Python 3.11 so envs are not mandatory (as in 3.12) > I installed it with pip and it worked well the other day. I will debug it tomorrow!

2024-12-27

Martin Grigorov (04:32:19) (in thread): > > biocbuild@kunpeng2 ~/git [1]> python -m pip install scikit-learn > Requirement already satisfied: scikit-learn in /home/biocbuild/miniconda3/lib/python3.11/site-packages (1.6.0) > Requirement already satisfied: numpy>=1.19.5 in /home/biocbuild/miniconda3/lib/python3.11/site-packages (from scikit-learn) (2.2.1) > Requirement already satisfied: scipy>=1.6.0 in /home/biocbuild/miniconda3/lib/python3.11/site-packages (from scikit-learn) (1.14.1) > Requirement already satisfied: joblib>=1.2.0 in /home/biocbuild/miniconda3/lib/python3.11/site-packages (from scikit-learn) (1.4.2) > Requirement already satisfied: threadpoolctl>=3.1.0 in /home/biocbuild/miniconda3/lib/python3.11/site-packages (from scikit-learn) (3.5.0) > > already installed !

Martin Grigorov (04:32:54) (in thread): > > biocbuild@kunpeng2 ~/git> R CMD check seqArchR_1.11.0.tar.gz > * using log directory '/home/biocbuild/git/seqArchR.Rcheck' > * using R Under development (unstable) (2024-11-24 r87369) > * using platform: aarch64-unknown-linux-gnu > * R was compiled by > aarch64-unknown-linux-gnu-gcc (GCC) 14.2.0 > GNU Fortran (GCC) 14.2.0 > * running under: openEuler 24.03 (LTS) > * using session charset: UTF-8 > * checking for file 'seqArchR/DESCRIPTION' ... OK > * checking extension type ... Package > * this is package 'seqArchR' version '1.11.0' > * package encoding: UTF-8 > * checking package namespace information ... OK > * checking package dependencies ... OK > * checking if this is a source package ... OK > * checking if there is a namespace ... OK > * checking for hidden files and directories ... OK > * checking for portable file names ... OK > * checking for sufficient/correct file permissions ... OK > * checking whether package 'seqArchR' can be installed ... OK > * checking installed package size ... OK > * checking package directory ... OK > * checking 'build' directory ... OK > * checking DESCRIPTION meta-information ... OK > * checking top-level files ... OK > * checking for left-over files ... OK > * checking index information ... OK > * checking package subdirectories ...Warning: program compiled against libxml 212 using older 211 > OK > * checking code files for non-ASCII characters ... OK > * checking R files for syntax errors ... OK > * checking whether the package can be loaded ... OK > * checking whether the package can be loaded with stated dependencies ... OK > * checking whether the package can be unloaded cleanly ... OK > * checking whether the namespace can be loaded with stated dependencies ... OK > * checking whether the namespace can be unloaded cleanly ... OK > * checking loading without being on the library search path ... OK > * checking whether startup messages can be suppressed ... OK > * checking dependencies in R code ... OK > * checking S3 generic/method consistency ... OK > * checking replacement functions ... OK > * checking foreign function calls ... OK > * checking R code for possible problems ... OK > * checking Rd files ... OK > * checking Rd metadata ... OK > * checking Rd cross-references ... OK > * checking for missing documentation entries ... OK > * checking for code/documentation mismatches ... OK > * checking Rd \usage sections ... OK > * checking Rd contents ... OK > * checking for unstated dependencies in examples ... OK > * checking installed files from 'inst/doc' ... OK > * checking files in 'vignettes' ... OK > * checking examples ... OK > * checking for unstated dependencies in 'tests' ... OK > * checking tests ... > Running 'testthat.R' > ERROR > Running the tests in 'tests/testthat.R' failed. > Last 13 lines of output: > ══ Failed tests ════════════════════════════════════════════════════════════════ > ── Error ('test_seqArchR_main.R:149:5'): seqArchR (cv) works when timeFlag is FALSE ── > Error: cannot find an open port. For manually specifying the port, see > ?SnowParam > Backtrace: > ▆ > 1. └─seqArchR::seqArchR(...) at test_seqArchR_main.R:149:5 > 2. └─seqArchR:::perform_setup(...) > 3. └─BiocParallel::MulticoreParam(workers = crs, tasks = crs) > 4. └─BiocParallel:::.snowPort() > 5. └─BiocParallel:::.stop("cannot find an open port. For manually specifying the port, see ?SnowParam") > > [ FAIL 1 | WARN 1 | SKIP 4 | PASS 104 ] > Error: Test failures > Execution halted > * checking for unstated dependencies in vignettes ... OK > * checking package vignettes ... OK > * checking re-building of vignette outputs ... OK > * checking PDF version of manual ... OK > * DONE > > Status: 1 ERROR > See > '/home/biocbuild/git/seqArchR.Rcheck/00check.log' > for details. > > same error as the last time

2024-12-28

Pascal-Onaho (07:54:56): > @Pascal-Onaho has joined the channel

2024-12-29

Yahya Jahun (04:01:27): > @Yahya Jahun has joined the channel

2024-12-30

Martin Grigorov (04:31:37) (in thread): > https://github.com/Bioconductor/BBS/pull/432- a PR adding the details of the new VM and updating the ones for kunpeng2 - Attachment: #432 Add a build node named taishan for 3.20 on Linux ARM64 > As agreed at Slack (https://community-bioc.slack.com/archives/C04HZMP49LH/p1735226974631609?thread_ts=1734628436.266339&cid=C04HZMP49LH)

Martin Grigorov (04:33:20) (in thread): > I am currently running a full build (run.sh) on the new VM (taishan) . I’ll let you know once it finishes successfully ! > The firewall rule is already set !

Martin Grigorov (14:20:19) (in thread): > > bbs.jobs.processJobQueue> Finished. > > BBS> END STAGE4 loop. > BBS> ------------------------------------------------------------- > BBS> STAGE4 SUMMARY: > BBS> o Working dir: /home/biocbuild/bbs-3.20-bioc/meat > BBS> o 513 srcpkg file(s) in working dir > BBS> o 513 srcpkg file(s) queued and processed > BBS> o Total time: 2612.84 seconds > BBS> ------------------------------------------------------------- > BBS> [STAGE4] DONE at Mon Dec 30 17:46:11 2024. > BBS> START writing BBS_EndOfRun.txt ticket. > BBS> cd BBS_MEAT_PATH > BBS> Copying BBS_EndOfRun.txt to /home/biocbuild/bbs-3.20-bioc/products-out/ ... OK > BBS> END writing BBS_EndOfRun.txt ticket. >

Martin Grigorov (14:20:42) (in thread): > I will check for timed out downloads and install them manually

2025-01-02

Martin Grigorov (11:05:03) (in thread): > @Andres WokatyI thinktaishanis ready to be added to the build runners

Andres Wokaty (12:06:44) (in thread): > Can you share the IP with me?

2025-01-06

Andres Wokaty (10:12:28) (in thread): > I get connection refused when I attempt torsyncwithtaishan. I used the same command we use forkunpeng2, only withtaishanIPs and paths.

Martin Grigorov (12:03:10) (in thread): > Could you please send me again the static IP of your system?

Martin Grigorov (12:04:46) (in thread): > This is whatI’verequested “Please add 149.165.152.87:873 to the list of IP:port that are allowed to reach 49.0.202.179” and the network admins confirmed that it is done

Andres Wokaty (12:05:39) (in thread): > That looks right; it’s the IP listedhttps://github.com/Bioconductor/BBS/blob/b822ae67e79fcde66377f48b2ecf036fb8f99373/3.20/bioc/taishan/config.sh#L25.

Martin Grigorov (12:06:22) (in thread): > I’llask the admins to re-check

Martin Grigorov (15:44:11) (in thread): > @Andres Wokatycould you please test‘nmap-p 873 49.0.202.179’ ?

2025-01-07

Martin Grigorov (01:12:06) (in thread): > Here is the response from the network admins: “149.165.152.87 is already in the white list, and port 873 is allowed”

Andres Wokaty (10:35:19) (in thread): > I will try on Thursday as that machine is down for maintenance.

Martin Grigorov (10:35:35) (in thread): > OK!

2025-01-08

Andres Wokaty (15:20:08) (in thread): > The machine finished maintenance early; however, I still get that it’s closed: > > sudo nmap -p 873 49.0.202.179 > Starting Nmap 7.80 ([https://nmap.org](https://nmap.org)) at 2025-01-08 15:16 EST > Nmap scan report for ecs-49-0-202-179.compute.hwclouds-dns.com (49.0.202.179) > Host is up (0.22s latency). > > PORT STATE SERVICE > 873/tcp closed rsync > > Nmap done: 1 IP address (1 host up) scanned in 0.84 seconds >

2025-01-09

Martin Grigorov (10:46:43) (in thread): > @Andres WokatyCould you please try again ?

Andres Wokaty (10:55:41) (in thread): > I’m finally able to rsync with taishan. I’ll finish set up. The release builds start running at 13:00 ET Monday and Thursday. Taishan should follow this cadence.

Martin Grigorov (10:56:40) (in thread): > Great!

Martin Grigorov (10:57:06) (in thread): > What UTC time is this ?

Andres Wokaty (10:58:19) (in thread): > 6:00pm UTC

2025-01-11

Hervé Pagès (18:22:55) (in thread): > I see that the builds on the other 3.20 builders (nebbiolo2, merida1, etc..) start at 3 pm EST on Mondays and Thursdays. Isn’t that 8pm UTC@Andres Wokaty@Martin Grigorov?

2025-01-13

Martin Grigorov (02:19:28) (in thread): > I can easily change it to whatever time you tell me!

Andres Wokaty (10:13:36) (in thread): > Thanks@Hervé Pagès. The time I gave was the prerun script rather than the run. taishan should start building at 8pm UTC. Sorry for the mistake@Martin Grigorov

Martin Grigorov (10:14:43) (in thread): > Updated it to 00 20 * * 1,4 ... !

Andres Wokaty (12:20:10) (in thread): > taishan’s on the report today:https://bioconductor.org/checkResults/3.20/bioc-LATEST/long-report.html

Martin Grigorov (13:09:03) (in thread): > Great!I’llcheck the errors tomorrow!

2025-01-14

Martin Grigorov (02:52:12) (in thread): > Pandoc 2.x was missing on taishan. This caused most of the BUILD errors. It is installed now!

2025-01-25

Rahman Nurudeen (10:26:47): > @Rahman Nurudeen has joined the channel

2025-02-07

Andres Wokaty (12:20:08): > Hi@Martin GrigorovI noticed kunpeng2 hasn’t been finishing its build recently so I’ve had to remove it from the report. Can you take a look when you have time?

Martin Grigorov (16:22:40) (in thread): > I’llcheck it at Sunday! Thanks!

2025-02-10

Martin Grigorov (02:21:55) (in thread): > > File "/home/biocbuild/BBS/BBS-run.py", line 789, in <module> > tee: /home/biocbuild/bbs-3.21-bioc/log/kunpeng2-20250210-run.log: No space left on device > bbs.fileutils.remake_dir(products_out_path, ignore_errors=True) > File "/home/biocbuild/BBS/bbs/fileutils.py", line 79, in remake_dir > os.mkdir(dirpath) > OSError: [Errno 28] No space left on device: '/home/biocbuild/bbs-3.21-bioc/products-out' >

Martin Grigorov (02:29:39) (in thread): > I’ve cleaned ~500GB disk space. It should be OK now!run.shstarts fine !

Martin Grigorov (02:30:12) (in thread): > I’ve also added a cron job that deletes old content in $TMPDIR

2025-02-18

Martin Grigorov (08:02:40): > @Andres WokatyThe kunpeng2 VM had some issues and has been fully reset! I will set it up in the next few days !

Andres Wokaty (09:05:59) (in thread): > Thank you. I noticed it still wasn’t completing builds. When it’s set up, I’ll check its performance and add it back to the report if things are going well.

2025-02-24

Martin Grigorov (07:54:35) (in thread): > @Andres WokatyI think kunpeng2 is ready to be enabled again. I ran the build twice and installed the big data packages manually

Andres Wokaty (12:29:35) (in thread): > Yes, i was just looking at rerunning the report for it. Rerunning it shortly.

Martin Grigorov (12:46:15) (in thread): > Thank you!

2025-03-05

Andres Wokaty (10:14:46): > Hi@Martin GrigorovI’m updating R on the release builders. There was an update of R 4.4 last week:https://cran.r-project.org/. When you have time, maybe you want update taishan?

Martin Grigorov (11:03:26) (in thread): > I’llupgrade it tomorrow!Thanks for the notice!

2025-03-06

Martin Grigorov (04:56:19) (in thread): > I’ve updated it to 4.4.3 !

Martin Grigorov (04:56:57) (in thread): > I just started a build to download the packages. The timed out ones will be installed manually during the weekend !

2025-03-21

Andres Wokaty (11:35:32): > Hi@Martin GrigorovWe had to undo some commits to basilisk on devel. On kunpeng2, I recommend deleting the basilisk directory inmeatand I also removed the basilisk cache. You can seehttps://community-bioc.slack.com/archives/CLUJWDQF4/p1742502429060189for a little more context and you could opt to do that instead. - Attachment: Attachment > For developers who use basilisk: I prematurely merged a PR to the sources that makes a transition to virtual environment management of python infrastructure. This change has been reverted and the current devel version of 1.19.3 is in the intended state. If you have a checkout of basilisk source code from git, you may have to use the commands git fetch && git reset --hard origin/devel to get a clean source checkout. Feel free to DM me with concerns, and I will summarize back as appropriate.

Martin Grigorov (11:39:48) (in thread): > OK.I will do it tomorrow!

2025-03-22

Martin Grigorov (04:33:04) (in thread): > Done! > > biocbuild@kunpeng2 ~> rm -rf bbs-3.21-bioc/meat/basilisk* > biocbuild@kunpeng2 ~> rm -rf .cache/R/basilisk >

2025-03-24

Hervé Pagès (14:28:13) (in thread): > Just as an FYI everything in themeatfolder is automatically replaced at each build so there’s no need to delete anything manually.

2025-04-04

Andres Wokaty (15:08:58): > Hi@Martin GrigorovI’m configuring our builders for 3.22 (R-4.5) for the release on 4/15 so you may want to prepare taishan in next few weeks.

Martin Grigorov (16:14:25) (in thread): > I’ll do itin the week after next!

Martin Grigorov (16:14:53) (in thread): > Should I do something for kunpeng2 ?

Andres Wokaty (16:16:08) (in thread): > we want current the devel builders to be very stable for the release so I don’t think I will update those machines until after the release. So for now, no. If this changes, I will reach out.

Martin Grigorov (16:16:54) (in thread): > Ok! Thank you!

Andres Wokaty (16:17:03) (in thread): > thank you:slightly_smiling_face:

2025-04-14

Martin Grigorov (02:12:31) (in thread): > @Andres WokatyIs it time to configure taishan for 3.22/R-4.5 ?

Andres Wokaty (09:14:53) (in thread): > Yes!

Martin Grigorov (09:17:11) (in thread): > OK! I will work on it tomorrow!

2025-04-16

Martin Grigorov (02:48:12) (in thread): > @Andres Wokatytaishan is prepared for 3.22

2025-04-22

Andres Wokaty (16:54:59): > Hi@Martin GrigorovI added taishan to the 3.22 report. I also set release to build Mondays and Thursdays to allow slower machines to complete builds. Can you configure kunpeng to follow the slower cadence? Thank you.

2025-04-23

Martin Grigorov (00:06:23) (in thread): > Thanks! > Changed kunpeng2’s cron to10 19 * * 1,4!

Martin Grigorov (00:06:41) (in thread): > I will check taishan’s results later today!