#containers

2019-03-27

Vince Carey (11:02:46): > @Vince Carey has joined the channel

Vince Carey (11:02:46): > set the channel description: discuss container discipline and impacts on maintenance

Kasper D. Hansen (11:02:46): > @Kasper D. Hansen has joined the channel

Aaron Lun (11:02:46): > @Aaron Lun has joined the channel

Martin Morgan (11:02:46): > @Martin Morgan has joined the channel

BJ Stubbs (11:02:46): > @BJ Stubbs has joined the channel

Shweta Gopal (11:02:46): > @Shweta Gopal has joined the channel

Samuela Pollack (11:02:46): > @Samuela Pollack has joined the channel

Vince Carey (11:04:18): > The discussion of RBGL migrated to issues of maintenance in the presence > of dynamic linkage requirements. This leads quickly to container concepts. > > Are core developers using them much in real work? There are certain guarantees when we have a > working container. We don’t hear much about downsides outside of the general security risks. We can > get around the risk of dynamic linking failures by using containers. But the > implications for version control and version checking of running (but potentially > buggy) tools are not much discussed.

Sean Davis (11:09:22): > @Sean Davis has joined the channel

Sean Davis (11:10:27): > Just to be clear,@Vince Carey, are you talking about Docker, Singularity, and the like?

Vince Carey (11:10:52): > Yes, sorry – is there another connotation of ‘container’ in this context?

Vince Carey (11:14:41): > Despite years (now) of marginal acquaintance with the container concept, I am sure that my understanding of the concepts and vocabulary of the field is very superficial. But I hope it is worthwhile to have a channel devoted to the concept … maybe there is a better terminology insofar as we might think of our data classes as containers as well

Martin Morgan (11:16:43): > some flavors of container are anti-integrative – how is one going to use RBGL from container A with a standard Bioc installation? Presumably there is an answer to this, but it is likely technical and fragile

Martin Morgan (11:17:11): > Generally I think containers encourage bad practice – throw everything into a container without thinking about software engineering discipline

Vince Carey (11:27:57): > So this will be a short-lived channel.

Vince Carey (11:37:56): > But in case we are still breathing, a basic question is whether we can accept container benefits without losing integrity. For example, BiocManager::valid() can be run at any time, within a container, and for the versions of Bioc software in use in the container, valid() can tell the user that a stale package version is in use. Is that worthwhile?

2019-05-23

dave_sevenbridges (10:59:10): > @dave_sevenbridges has joined the channel

2019-05-24

Nicholas Knoblauch (13:10:13): > @Nicholas Knoblauch has joined the channel

2019-06-26

Mike Smith (03:28:07): > @Mike Smith has joined the channel

2019-06-28

Nitesh Turaga (14:21:34): > @Nitesh Turaga has joined the channel

2019-07-22

Marko Zecevic (08:47:23): > @Marko Zecevic has joined the channel

2019-07-23

Sehyun Oh (19:10:19): > @Sehyun Oh has joined the channel

2019-10-25

Lori Shepherd (11:06:17): > @Lori Shepherd has joined the channel

Martin Morgan (11:07:29): > #containers#generalWe’d like to hold a discussion about Bioconductor Docker containers on Tuesday November 5 at 11am Eastern US. The call will be athttps://bluejeans.com/434715813Please mark your calendars!

Nitesh Turaga (11:08:13): > set up a reminder about “Bioconductor docker meeting” in this channel at 11AM Tuesday, November 5th, India Standard Time.

Michael Lawrence (11:10:32): > @Michael Lawrence has joined the channel

Kayla Interdonato (11:16:44): > @Kayla Interdonato has joined the channel

Lorena Pantano (11:49:25): > @Lorena Pantano has joined the channel

Davide Risso (12:09:55): > @Davide Risso has joined the channel

Kevin Blighe (14:34:33): > @Kevin Blighe has joined the channel

Martin Morgan (14:38:53): > Actually had rather modest goals for our containers discussion on November 5 – we have several flavors of Bioconductor containers that the core team currently supports (devel_ and release_ base2 & core2, bioconductor_full, and a container for the#anvilproject), and we’d like to standardize things around the concept (though maybe not the name) of bioconductor_full (a container with system requirements but not necessarily packages) with branches and tagged releases maintained in git(hub) analogous to how Bioconductor packages are maintained. Of course wider ranging discussion is also possible…

Shraddha Pai (14:39:12): > @Shraddha Pai has joined the channel

Qian Liu (14:46:40): > @Qian Liu has joined the channel

Pratima Chennuri (15:01:07): > @Pratima Chennuri has joined the channel

Steve Lianoglou (15:31:07): > @Steve Lianoglou has joined the channel

Jayaram Kancherla (20:11:08): > @Jayaram Kancherla has joined the channel

2019-10-27

Peter Hickey (02:28:08): > @Peter Hickey has joined the channel

Dan Bunis (03:21:10): > @Dan Bunis has joined the channel

2019-10-28

Ruth Isserlin (10:55:37): > @Ruth Isserlin has joined the channel

2019-10-30

Michael Steinbaugh (14:48:15): > @Michael Steinbaugh has joined the channel

2019-11-04

Izaskun Mallona (07:58:46): > @Izaskun Mallona has joined the channel

2019-11-05

USLACKBOT (00:30:26): > Reminder: Bioconductor docker meeting.

Vince Carey (00:38:51): > I will be on a call most of the day and may miss this meeting.

Lori Shepherd (10:59:15): > Reminder of our docker meeting happening nowhttps://bluejeans.com/434715813/

Sean Davis (11:17:07): > From our phone discussion:https://seandavi.github.io/post/build-linux-r-binary-packages/ - Attachment (seandavi(s12)): Building R Binary Packages for Linux | seandavi(s12) > Background One of the challenges of producing a performant build environment for linux, such as what might be used to have developers test software in identical environments, is the need to compile R packages from source on linux. If, however, one had an identical set of installed libraries, kernel version, compiler, etc., we could use binary packages in linux as well. Docker provides just such a shareable and identical environment for linux.

Sean Davis (11:17:35): > https://seandavi.github.io/post/learning-github-actions/ - Attachment (seandavi(s12)): Experimenting with Github Actions | seandavi(s12) > GitHub actions allow flexible and potentially complicated actions that comprise workflows that respond to events on Github. Continuous integration, messaging Slack, greeting new contributors, deploying applications, and many other templates are ready for customization and integration into any repo.

Sean Davis (13:09:34) (in thread): > A major use case for docker is to have a reproducible, portable, shareable environment for building and testing software related to bioconductor.

Sean Davis (13:11:04) (in thread): > DockerHub maintains “download” stats to get a sense of how much something is used, at least for bioc-maintained images:https://hub.docker.com/u/bioconductor/

Sean Davis (13:12:31) (in thread): > All that said,@Kevin Bligheit would be great to hear about your experiences using docker for bioconductor (or more generally).

Vince Carey (17:19:54): > is there a timeframe for bioconductor/bioconductor_full:devel running R 4.0?

2019-11-06

Lori Shepherd (07:06:10): > I’m working on the new release / devel AMI’s today and will work on the dockers tomorrow (since we havent transitioned yet Bioconductor_full builds off the current bioconductionr_base2 so once that is updated it should be good to go) I expect by the weekend or early next week -

Vince Carey (09:30:22): > :+1:

2019-11-07

Nicholas Cooley (11:45:24): > @Nicholas Cooley has joined the channel

2019-11-27

Aedin Culhane (13:46:16): > @Aedin Culhane has joined the channel

2019-12-11

Christine Choirat (12:09:24): > @Christine Choirat has joined the channel

2020-01-02

Aaron Lun (11:17:22): > @Aaron Lun has left the channel

2020-01-03

Aaron Lun (18:57:11): > @Aaron Lun has joined the channel

Aaron Lun (18:57:53): > It would be a good idea to have a link to old versions of containers (https://hub.docker.com/r/bioconductor/release_core2/tags) on the BioC docker page. Took me a while to find it to point someone to it.

2020-01-05

Vince Carey (17:05:56): > Would it make sense to have a package, perhaps Containers4Bioc, that provides helper functions and documentation on container availability and use?@Lori Shepherd@Nitesh Turaga@Martin Morgan– note that Biocontainers already exists (biocontainers.pro) so BiocContainers is not a great choice for a package name.

2020-01-06

Nitesh Turaga (08:59:23): > Yes, I believe it might be helpful@Vince Carey.

Nitesh Turaga (10:26:08): > I agree that BiocContainers isn’t a great idea.

2020-01-15

olga tsiouri (16:29:33): > @olga tsiouri has joined the channel

2020-01-16

Nitesh Turaga (16:08:49): > Hi<!here>, Just wanted to keep people who are not following the bioc-devel mailing list emails about the deprecation of our previous set of containers in favor of thebioconductor_docker(https://github.com/bioconductor/bioconductor_docker) set of containers. > > More info on the emails:https://stat.ethz.ch/pipermail/bioc-devel/2020-January/016026.htmlhttps://stat.ethz.ch/pipermail/bioc-devel/2020-January/016037.html

Ruth Isserlin (16:22:33): > I am current user of bioconductor/release_core2:R3.6.0_Bioc3.9 and after going through the new documentation I can’t find a list of the new containers that are going to be available. It says they will be named “bioconductor/bioconductor_docker:RELEASE_X_Y” but is there a listing of the different releases that are available. Also, there were a bunch of packages that came with core2 (like knitr and biobase and biomartr). Do the new base containers come with any specific packages installed?

Nitesh Turaga (20:55:39): > Hi@Ruth Isserlin, > > The version release 3.9 is not available forbioconductor/bioconductor_dockerimage. We do have the release 3.10 though inbioconductor/bioconductor_docker:RELEASE_3_10. > > There are currently only two, i.edevelandRELEASE_3_10(also calledlatest).https://hub.docker.com/repository/docker/bioconductor/bioconductor_docker/tags?page=1. > > The new base containers DO NOT have any packages installed on them except forBiocManager. But they have all the system dependencies you’d need for all the Bioconductor packages, so you can install any package you want.

Ruth Isserlin (21:13:20): > With the latest version can I still use install2.r in my Dockerfile to add packages when I build my image?

Nitesh Turaga (23:23:00): > We recommend just usingBiocManager(). But, it should be extendable exactly as before if you choose to use the install2.r script by littler.https://github.com/bioconductor/bioconductor_docker#modify, take a look at example 1 and 2.

2020-01-17

Nitesh Turaga (11:14:08): > @Ruth IsserlinAnother word of advise about install2.r, you might want to use it like, > > install2.r --repos[https://bioconductor.org/packages/3.11/bioc](https://bioconductor.org/packages/3.11/bioc)BiocGenerics BiocParallel > > What setsBiocManagerapart from installinstall.packages()is the set of repositories it uses.http://bioconductor.org/install/#why-biocmanagerinstallTake a look that link which better explains why BiocManager should be used.

Ruth Isserlin (11:48:12): > Thanks@Nitesh Turaga! Good point about repos. I tend to use “http://www.bioconductor.org/packages/release/bioc” which will just get me into a mess is I am using an older version of bioc. I find in general images can take a really long time to build. Is there any speed advantage with BioCmanager vs. install2.r?

Nitesh Turaga (11:50:16): > There is no real speed advantage of “compiling packages” on the docker image. Depending on how you configure your container runtime (image below), there can be a speed up. > > BiocManager::install(c("package_name", "package_name"), Ncpus=4) > - File (PNG): Screen Shot 2020-01-17 at 11.49.17 AM.png

Nitesh Turaga (11:52:17): > Both install2.r and BiocManager compile the packages, so no, there is no speed advantage

Nitesh Turaga (11:54:29) (in thread): > Again, this should point to just using BiocManager because it’ll figure out the versions for you.BiocManager::valid()will tell you if there is incompatibility.

Martin Morgan (12:01:35): > just to clarify, you want to use the repositories returned fromBiocManager::repositories()on the image, so ‘3.11’ in the url if you’re using the devel image, but ‘3.10’ in release.

Nitesh Turaga (12:05:22): > Yes, it seems like the way to do it with install2.r is, > > install2.r --repos[https://bioconductor.org/packages/3.11/bioc](https://bioconductor.org/packages/3.11/bioc)\ > --repos[https://bioconductor.org/packages/3.11/data/annotation](https://bioconductor.org/packages/3.11/data/annotation)\ > --repos[https://bioconductor.org/packages/3.11/data/experiment](https://bioconductor.org/packages/3.11/data/experiment)\ > --repos[https://bioconductor.org/packages/3.11/workflows](https://bioconductor.org/packages/3.11/workflows)\ > --repos[https://cran.rstudio.com/](https://cran.rstudio.com/)\ > BiocGenerics GenomeInfoDbData ALL rnaseqGene tidyr > > There might be a smarter way too. > > Depending on the version of Bioconductor you are using, like@Martin Morgansuggested, usingBiocManager::repositories()is the correct way.

Ruth Isserlin (12:09:36): > @Martin Morganbut if I use BiocManager to install my packages I don’t need to worry about the repo version because BiocManager will handle that for me. Is that correct?

Nitesh Turaga (13:23:47): > Yes. That is correct.

2020-01-30

Nitesh Turaga (11:11:43): > Hi<!here>, has anyone used the new set ofbioconductor_dockerimages and faced issues similar to what is posted here,https://github.com/Bioconductor/bioconductor_docker/issues/3? It seems they are having trouble sharing volumes, and i’m unable to reproduce it, it works just fine on my end:confused:

Ruth Isserlin (11:12:35): > I have been struggling with this for the past 2 days:slightly_smiling_face:

Nitesh Turaga (11:13:11): > But i don’t try to set a new$USERor$USERID. That seems unnecessary

Nitesh Turaga (11:13:31): > Hi@Ruth IsserlinWhat exactly is the issue you are seeing?

Ruth Isserlin (11:14:06): > I changed over my existing containers to use the new bioc docker and I can launch them fine if I don’t specify the volume but the second I specify a volume the run stalls at > [cont-init.d] userconf: executing… > usermod: user ‘bioc’ already exists

Ruth Isserlin (11:14:13): > I tried with –user bioc

Ruth Isserlin (11:14:25): > I tried changing the path in the docker that I am mapping to.

Nitesh Turaga (11:14:43): > How big is the volume?

Nitesh Turaga (11:15:22): > > docker run --rm -d \ > -e PASSWORD=bioc -p 8888:8787 \ > -v ~/Documents:/home/bioc \ > -v ~/shared/devel/library-store/:/usr/local/lib/R/host-site-library \ > bioconductor/bioconductor_docker:devel >

Ruth Isserlin (11:16:28): > 4.2GB. currently it is set to -v “$(pwd)“:/home/bioc/rstudio/projects

Nitesh Turaga (11:16:50): > That’s what i’m using and it seems to work…..i can’t seem to figure out how to reproduce it. Although theusermod: user 'bioc' already existdoes show up because we are adding a new user for the image calledbiocand switching into it when RStudio starts up.

Nitesh Turaga (11:17:27): > I see, 4.2GB is not big at all. I’m surprised it doesn’t start up.

Ruth Isserlin (11:17:35): > when I get it to run without the volume specified I also see the same user already defined but when it stalls that is where it is stalling

Jayaram Kancherla (11:17:37): > are you using linux or macos ? I think on mac os you need to explicitly add volumes you wanna mount on docker (https://docs.docker.com/docker-for-mac/osxfs/#namespaces) - Attachment (Docker Documentation): File system sharing (osxfs) > osxfs is a new shared file system solution, exclusive to Docker Desktop for Mac. osxfs provides a close-to-native user experience for bind mounting macOS file system trees into Docker containers….

Ruth Isserlin (11:18:05): > Is this new? I haven’t had to do that in the past. I am on macos

Nitesh Turaga (11:18:10): > Can you try with--user rstudiohere in your command@Ruth Isserlinand see if the container stalls again?

Nitesh Turaga (11:18:50): > Maybe@Jayaram Kancherlahas a point too. You could try that as well.

Jayaram Kancherla (11:19:08): > I’m not sure if its new, the last time I had to get docker to mount properly on macos, you had share the volume with docker first. > > By default, you can share files in/Users/,/Volumes/,/private/, and/tmpdirectly. To add or remove directory trees that are exported to Docker, use theFile sharingtab in Docker preferences  ->Preferences->File sharing. (See Preferences.)

Ruth Isserlin (11:19:09): > I get permission denied with user rstudio

Nitesh Turaga (11:19:26): > Can you paste your command?

Ruth Isserlin (11:19:39): > the docker run command?

Nitesh Turaga (11:19:42): > yes

Ruth Isserlin (11:20:13): > docker run -v “$(pwd)“:/home/bioc/rstudio/projects -e PASSWORD=password -p 8787:8787 –user rstudio –add-host “localhost:###.###.###.###” risserlin/em_base_image:em_testing

Ruth Isserlin (11:23:24): > @Jayaram Kancherlathis file sharing thing looks new to me but I might have updated docker recently so maybe that is why the issue just started now. So an organizational question, do you keep all your volumes used for your dockers in the same directory. I tend to have my volumes all over the place as they are project based running in different containers but using the same base image.

Ruth Isserlin (11:25:54): > I tried adding the volumes but it still didn’t work.

Nitesh Turaga (11:27:59): > Let’s try removing the--user rstudio. Forget that entirely.

Nitesh Turaga (11:28:07): > We want the user to bebioc.

Nitesh Turaga (11:28:33): > can you change the"$(pwd)"with the full system path?

Ruth Isserlin (11:29:17): > I didn’t post the original question I am just having the exact same issue. Yes. I will try without the $pwd but I think I already tried that and it also stalled.

Nitesh Turaga (11:30:58): > ok, please try again and let me know. Because when I try your command without the $pwd it works on me end.

Ruth Isserlin (11:31:15): > docker run -v /Users/risserlin/gm_docker:/home/bioc -e PASSWORD=password -p 8787:8787 –add-host “localhost:###.###.###.###” risserlin/em_base_image:em_testing –> hangs at the exact same spot

Ruth Isserlin (11:31:36): > oh wait.

Ruth Isserlin (11:31:46): > it might have just taken a really long time

Nitesh Turaga (11:31:59): > ok, but it worked?

Ruth Isserlin (11:32:32): > Yes! that one worked. Is it maybe not hanging and just taking forever to load my 4.2 GB directory?

Ruth Isserlin (11:32:45): > I didn’t have this issue before moving to the new bioc docker though

Ruth Isserlin (11:33:02): > the directory I used for he second attempt was empty though

Nitesh Turaga (11:33:38): > hmm…

Nitesh Turaga (11:34:44): > Let me look into it a little more

Ruth Isserlin (11:36:31): > I went back and tried the same command with an empty directory and ($pwd) and it launched right away.

Ruth Isserlin (11:57:56): > So I just tried to hack it and launched the container with an empty volume and then tried copying the files into the empty directory after I ran the container and my system is saying it will take 15hours to copy over the files. This is very weird. I have never had this issue before.

Jayaram Kancherla (11:59:15) (in thread): > I keep all my docker volumes in a specific folder. some times even though you add the top level directory, you need to add the lower level directories for it to work. (do everything - restart docker & restart the machine) after adding the volume.

Ruth Isserlin (14:20:16): > some more info on the issue I was having. I went back and rebuilt my original image which was using bioconductor/release_core2:R3.6.0_Bioc3.9 and it was able to run with the directory no problem. I then came back to the new instance and instead of killing it after a minute I let it hang to see if something would happen. After about 4 or 5 minutes it starts spitting out hundreds of errors: > “chown: changing ownership of ‘/home/bioc/data/gsva_randomizations_GDC_subset_origoverlap/REGULATION OF MEMBRANE LIPID DISTRIBUTION%GOBP%GO_0097035.txt’: Input/output error” > One for every file in the mapped volume. (It is still going). Why is the new bioc container trying to change ownership of my files?

Nitesh Turaga (14:25:01): > Hi@Ruth IsserlinI’ll have to look into this more…but it’s because of these two blocks of code without getting into the details, > 1. https://github.com/rocker-org/rocker-versioned/blob/master/rstudio/userconf.sh#L64-L73(rocker/rstudio/userconf.sh) > 2. https://github.com/Bioconductor/bioconductor_docker/blob/master/Dockerfile#L161(bioconductor/bioconductor_docker) > I’ll have to figure out a solution to fix this.

Ruth Isserlin (15:05:48): > @Nitesh Turagado you want me to submit an issue to the github or is it fine to keep the report here?

Nitesh Turaga (15:10:25): > hi@Ruth Isserlin, File an issue to github if you can, that would help a lot. And please leave instructions on how I can reproduce the error you are getting with the exact commands.

2020-01-31

Nitesh Turaga (15:24:01): > Hi@Ruth IsserlinI just made changes to remove thebiocuser till i figure it out. Can you update yourbioconductor_dockerimages and try to see if you get the same error again?

2020-02-03

Nitesh Turaga (13:04:39): > @Ruth IsserlinLet me know if the updates worked for you.

Ruth Isserlin (13:05:44): > @Nitesh Turagasorry I haven’t gotten around to testing it yet. Will try to try it out today.

Nitesh Turaga (13:06:08): > Thank you.

Ruth Isserlin (15:46:44): > @Nitesh TuragaI rebuilt my container and it worked perfectly. (I was able to map my existing drive in seconds) and I used rstudio username to log in. Thanks!

2020-02-05

Aedin Culhane (22:14:22): > Great talk in Boston tonight

Aaron Lun (22:35:06): > Wasn’t that yesterday? In DC?

2020-02-08

Gabriele Sales (07:21:12): > @Gabriele Sales has joined the channel

2020-02-14

Andrew Skelton (05:04:21): > @Andrew Skelton has joined the channel

2020-03-04

Jialin Ma (15:02:11): > @Jialin Ma has joined the channel

2020-03-17

Catalina Vallejos (07:56:11): > @Catalina Vallejos has joined the channel

2020-03-18

Peter Hickey (19:27:34): > trying outhttp://bioconductor.org/help/docker/on ubuntu laptop so I can run R 4.0. Thanks! > quick question: instructions say go tohttps://localhost:8787but I get > > This site can’t provide a secure connectionlocalhost sent an invalid response. > ERR_SSL_PROTOCOL_ERROR >

Peter Hickey (19:28:11): > buthttp://localhost:8787(http) works

Peter Hickey (19:28:26): > that okay?

2020-03-19

Sean Davis (08:41:29): > That is normal,@Peter Hickey. https will require a certificate and likely a web proxy–more work. Since you are communicating with localhost, there shouldn’t be too much concern about encryption on the wire (which https gives you).

Nitesh Turaga (08:44:29): > Thanks for getting to this@Sean Davis!

Sean Davis (08:44:58) (in thread): > Team effort!

Peter Hickey (16:42:37): > thanks. instructions need updating then?

2020-03-24

Nitesh Turaga (09:04:46): - Attachment: Attachment > bioconductor/bioconductor_docker:devel

Nitesh Turaga (09:05:22): > @Shian SuThere is no tex installation on bioconductor/bioconductor_docker:devel. This is something you’ll have to build on your own.

Shian Su (09:05:29): > @Shian Su has joined the channel

Nitesh Turaga (09:06:37): > http://bioconductor.org/help/docker/#modifyExample #2 is what you want to build on your machine.

FelixErnst (10:14:59): > @FelixErnst has joined the channel

Sridhar N (10:28:58): > @Sridhar N has joined the channel

Robert Castelo (14:29:55): > @Robert Castelo has joined the channel

Robert Castelo (14:37:08): > hi all, when i open a bash shell in the bioc release container i land in the root directory instead of /home/rstudio:docker run -it --user rstudio bioconductor/bioconductor_docker:RELEASE_3_10 bash``rstudio@e1c4177fc5ff:/$ pwd``/``rstudio@e1c4177fc5ff:/$ ls ``bin  dev home lib lib64 media opt  root sbin sys usr``boot etc init lib32 libexec mnt proc run  srv  tmp var``rstudio@e1c4177fc5ff:/$ ls home``rstudio``rstudio@e1c4177fc5ff:/$i’m a newbie to the container world, so just out of curiosity, is there a reason for this?

Rena Yang (14:42:59): > @Rena Yang has joined the channel

FelixErnst (15:08:48) (in thread): > Hi Robert, I am not sure what your intention is using thebioconductor_dockerusually you would start it detached using-ditinstead of-itand then access rstudio from you browser by browsing tolocalhost:8787. However it is of course totally normal to startbashin the container as well and to have a look inside. > > Regarding therstudiouser: This user gets added to the container from underlyingrocker/rstudiodocker image (https://hub.docker.com/r/rocker/rstudio/dockerfile) during the install of the*.debfile. Which configuration this user gets depends entirely on RStudio, so I would not expect it to be your normal user. So the working directory might be expected to be/. > > I would also guess that by running the command you used, you will overwrite the default command/init, which probably will cause rstudio not to start.

FelixErnst (15:15:49) (in thread): > I would suggest you start the container using the default command with some slight changes > > docker run -dit \ > -e PASSWORD=bioc \ > -p 8787:8787 \ > bioconductor/bioconductor_docker:RELEASE_3_10 > > this way you can use the container. > > If you want to startbashadditionally, you can do so by running theexeccommand like this > > docker exec -it **youcontainername** bash > > (To get the container name rundocker psand have a look at the last column) > > To exit frombashtypeexitonce you are done. > > Stop the docker container by runningdocker stop **yourcontainername**Restart the container by runningdocker start **yourcontainername**

Peter Hickey (17:51:20): > in case useful to anyone: i’m using the docker containers to do package fixing/development for devel but I don’t want to (slash can’t) install R4.0 on my laptop. After some messing about I figured out how to mount(?) a directory on my laptop so it was visible within docker using: > > docker run -e PASSWORD=bioc -p 8787:8787 -v /home/peter/GitHub/MatrixGenerics:/home/rstudio/MatrixGenerics bioconductor/bioconductor_docker:devel >

Peter Hickey (17:51:42): > I wonder if this is a common enough use case to justify a place onhttp://bioconductor.org/help/docker/?

Sean Davis (19:48:10): > http://bioconductor.org/help/docker/#mounting

Peter Hickey (19:52:18): > forgot that was there@Sean Davisthanks! > that’s technically what i needed but the use case given in the example meant i didn’t realise it for quite a while (I was thinking, well I don’t want to point to my existing library because that’s R3.6.x whereas I need R4.0) > I guess what I was thinking more of was a ‘Frequent Use Cases’ section. E.g, “I want to update my package in devel”, “I want to reproduce an analysis from a specific version of BioC” etc.

Sean Davis (19:53:03): > Ah. Got it. Great point!

Peter Hickey (19:53:19): > i found the documentation there still fairly technical (but very clear!) - but something to ease use noobies into docker with common use cases would be a great addition

Chengyang Ji (20:43:46): > @Chengyang Ji has joined the channel

2020-03-25

Robert Castelo (03:38:20) (in thread): > Thanks Felix for your careful explanations, i was just poking around with docker and tried to enter the container with the user ‘rstudio’ to avoid entering as ‘root’ and this was the moment where i found it was odd to land in the / directory as a normal user, even entering as root, i’d expect to land in /root and not in /. anyway, thanks for the tips!!

FelixErnst (04:28:50) (in thread): > Yes thats usually the case. User control inside a container is not something to worry about or even consider as an end user. > > Docker starts container with root privileges or at least that how it is designed (https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface). The docker daemon runs as root. > > rootless mode is experimental for docker (https://docs.docker.com/engine/security/rootless/) and is build into other container engines such as podman (https://podman.io/), but now this is becoming deep dive so I will stop for now. - Attachment (Docker Documentation): Docker security > There are four major areas to consider when reviewing Docker security: the intrinsic security of the kernel and its support for namespaces and cgroups; the attack surface of the Docker… - Attachment (Docker Documentation): Run the Docker daemon as a non-root user (Rootless mode) > Rootless mode allows running the Docker daemon and containers as a non-root user, for the sake of mitigating potential vulnerabilities in the daemon and the container runtime. Rootless mode does… - Attachment (podman.io): Podman > Repository for podman.io website using GitHub Pages.

Nitesh Turaga (09:15:12): > Hi@Peter HickeyIf you would like to discuss these points, i’d be happy to work with you on this to improve the documentation page we have set up right now. We are always looking to improve the docs we have. Would you recommend something specifically for new comers to the docker world or something within the same document?

Peter Hickey (16:28:15): > I’m too swamped at the moment to do anything substantial but my idea is to brainstorm of common use cases for the Docker images BioC provide and then give step-by-step instructions that assume no familiarity with docker, perhaps avoid technical details in favour of getting people quickly to a situation where they have RStudio open using Docker and ready to solve their immediate problem (e.g., working on devel branch to fix a package)

2020-03-26

Sean Davis (10:26:17): > As a reminder to the community,bioconductor.orgcontent is hosted on github. Discuss via issues, contribute via pull-requests as usual.https://github.com/Bioconductor/bioconductor.org/blob/master/content/help/docker.md

Sean Davis (10:26:55): > No pressure,@Peter Hickey. This serves as a reminder to me, also!

Steve Lianoglou (16:43:06): > Can anyone help w/ a dumb docker issue I’m having. I’m trying to build a few images that inherit from the great and powerfulbioconductor/bioconductor_dockerimage. Just this morning everything worked fine, and I’ve been able to launch rstudio into the environment I setup

Steve Lianoglou (16:44:28): > now trying to relaunch my image, or even the basebioconductor_dockerimage, after logging into RStudio, I get an RStudio error window that says “Error occurred during transmission” … I’ve seen an issue or two on the rocker issue tracker, but … i’m at a bit of a loss. has anyone been hit by this here?

Mike Jiang (16:56:06): > @Mike Jiang has joined the channel

Mike Jiang (16:56:07): - Attachment: Attachment > dockerhub shows bioconductor/bioconductor_docker:RELEASE_3_10 was successfully built a day ago, but after I docker pull bioconductor/bioconductor_docker:RELEASE_3_10, and started the container, > oot@fbb5d60a2571:/# R --version > R version 3.6.2 (2019-12-12) -- "Dark and Stormy Night" > which is behind the current bioc night builds 3.6.3 (2020-02-29) -- "Holding the Windsock > So I couldn’t really use the docker image to reproduce the current error reported in bioc 3.10, the same thing applies to devel branch according to the last try from @Jake Wagner. > Not sure if this has been asked here before, Is it true that bioc docker image doesn’t necessarily use the exact same environment as bioc build server?

Jake Wagner (18:34:10): > @Jake Wagner has joined the channel

Steve Lianoglou (18:39:13): > figured out the issue to my “Error occurred during tansmission” error within RStudio .. it had to do with clogging up docker’s “virtual fileystem” (don’t know what the right word is)

Steve Lianoglou (18:40:14): > My machine has plenty of HD space, but there’s limit to something in the docker world, and running adocker image prunecleaned up enough space wherever those image caches live, where now RStudio is running all smooth now inside the image

Nitesh Turaga (20:15:16): > Hi@Mike Jiang, yes, the idea behind the bioc docker release imagesis notto reproduce the exact same environment as the build machines.

Nitesh Turaga (20:16:04): > It only promises that you’ll be able to install most bioconductor packages (about 99.9%) of the packages without having to deal with the system dependencies.

Mike Jiang (23:47:43) (in thread): > Then what good is it if it doesn’t guarantee the same package source from Bioc release can install/run properly on the container? I thought the very idea of docker is to offer user to reproducible bioc environment, at the very least, for the current release branch, no?

2020-03-27

Martin Morgan (04:41:45) (in thread): > the goals of (1) easy installation and use and (2) of reproducing the build environment are different and not compatible; the docker images aim for the former but often enable the later. I’ve had success ‘following along from home’ on several mailing list questions that seek to reproduce build system errors, and I’ll try to post something about best practices for doing so. > > The RELEASE_3_10 version of the docker images is really meant for users, and it builds on an underlying rocker container; we won’t try to make it useful for developers (e.g., we won’t set build-system-specific environment variables, and we won’t try to work harder than the rocker container to keep the instance current with patched versions of R). > > We are trying to make the devel version of the docker container more useful to developers. We’re working on having weekly updates to R (ironically, the docker will then often be ahead of the build system R). We also thought we had a solution to set environment variables like on the builders, but our current solution isn’t good enough and we’re iterating on that. We will NOT support vignettes (LaTeX in particular), so generally vignette-related problems would need to be debugged indirectly, e.g,R CMD Stangle vignette.Rnw; R -f vignette.R

Martin Morgan (05:42:00) (in thread): > One example of container use, not quite ‘best practices’ (I would have created a persistent task-specific library as athttp://bioconductor.org/help/docker/#mounting) is athttps://stat.ethz.ch/pipermail/bioc-devel/2020-March/016440.html

Mike Jiang (12:11:51) (in thread): > Thanks for the detailed clarifications. It is worth to add these notes to the docker pages to avoid similar questions in future.

2020-03-29

Milan Malfait (04:43:31): > @Milan Malfait has joined the channel

2020-03-31

Levi Waldron (09:30:59): > @Levi Waldron has joined the channel

2020-04-06

Nitesh Turaga (12:36:26): > Hello Community, > > A brief update on the “bioconductor/bioconductor_docker:devel” image which is produced by the Bioconductor team. > > The image will now be updated “weekly” on Friday evening around 6pm EST. This means that the image will have the latest version of R-devel by the week, and pushed to docker hub with a new version number located in the Dockerfile. > > The version number will be located in the LABEL field of the Dockerfile, which is available to be inspected in the docker image, with the following command > > $ docker inspect bioconductor/bioconductor_docker:devel > > This will produce a JSON file which shows the following “Labels”. Please note that the “version” in this section refers to the update, i.e “3.11.3”. Next week it will be “3.11.4” and so on, till there is a new Bioconductor devel version number, which will reset the count. The version number in the format “x.y.z”, refers to the bioconductor version number in the “x.y” coordinates, and the “z” coordinate indicates the version of the Dockerfile / Docker image being updated. > > "Labels": { > "description": "Bioconductor docker image with system dependencies to install most packages.", > "license": "Artistic-2.0", > "maintainer": "[maintainer@bioconductor.org](mailto:maintainer@bioconductor.org)", > "name": "bioconductor/bioconductor_docker", > "org.label-schema.license": "GPL-2.0", > "org.label-schema.vcs-url": "[https://github.com/rocker-org/rocker-versioned](https://github.com/rocker-org/rocker-versioned)", > "org.label-schema.vendor": "Rocker Project", > "url": "[https://github.com/Bioconductor/bioconductor_docker](https://github.com/Bioconductor/bioconductor_docker)", > "vendor": "Bioconductor Project", > "version": "3.11.3" > } > > If you have any questions, please feel free to reply to this thread.

2020-04-13

Stevie Pederson (10:20:12): > @Stevie Pederson has joined the channel

2020-04-19

Charlotte Soneson (11:30:07): > @Charlotte Soneson has joined the channel

2020-04-30

Robert Castelo (14:19:03): > hi, the message above says that the bioconductor_docker:devel image will be updated tomorrow Friday. Will be also available the image for the new release version 3.11? thanks!

Martin Morgan (17:39:21): > @Robert Castelothe images rely on rocker, and rocker is not yet building 4.0.0 releases, so unfortunately the ‘devel’ image will use the wrong R (R-devel, rather than R-4.0.0)…@Nitesh Turagacan correct me if I’ve got the details wrong…

2020-05-01

Kevin Rue-Albrecht (03:59:15): > @Kevin Rue-Albrecht has joined the channel

Kevin Rue-Albrecht (04:00:51): > DockerHub is complaining: > > Step 6/11 : RUN Rscript -e "BiocManager::install('iSEE', version = 'devel')" > ---> Running in 63c4727c2f14 > [91mError: To use Bioconductor version '3.12', first upgrade 5 packages with > BiocManager::install(version = '3.12') > Execution halted > [0m > Removing intermediate container 63c4727c2f14 > The command '/bin/sh -c Rscript -e "BiocManager::install('iSEE', version = 'devel')"' returned a non-zero code: 1 > > I’d rather not have a lineBiocManager::install(version = '3.12')to update every 6 months. > Is it just a matter of waiting forbioconductor/bioconductor_docker:develto be brought up to date? (I haven’t checked yet where it’s at)https://hub.docker.com/repository/registry-1.docker.io/iseedevelopers/isee/builds/056eab00-c8b1-4a7d-b526-63a4e2555657

Kevin Rue-Albrecht (04:01:48): > Looks like the last build was 14 days ago:https://hub.docker.com/r/bioconductor/bioconductor_docker/builds

Nitesh Turaga (09:11:03): > Hi@Kevin Rue-Albrechtcouple of things, > 1. Yes it is a matter of waiting for bioconductor_docker:devel to be brought up to date to R-4.0. Then you will not have to specify anything. > 2. So, the way we update ourdevelimage is a little different. Since we build devel with the “latest” version of the svn repo, we push the “devel” image manually to dockerhub. You can see that the devel image is updated every 7 days.https://hub.docker.com/r/bioconductor/bioconductor_docker/tags

Nitesh Turaga (09:12:03): > I understand that the builds on dockerhub can be a little confusing.

Nitesh Turaga (09:12:23): > We are waiting for rocker to update and then we’ll update the docker images.

Kevin Rue-Albrecht (09:12:47): > Awesome. Thanks for the explanation!

Nicholas Knoblauch (10:02:55): > @Nicholas Knoblauch has left the channel

Sean Davis (10:12:21): > @Nitesh Turagado you happen to have an ETA for the rocker update to R-4.0? I ask only to know if it is worth trying to build a container in the meantime (on my own), not to suggest that there is a way to hurry the process on the rocker side….

Nitesh Turaga (10:24:07): > Hi@Sean Davis, I just asked the rocker folks again at (https://github.com/rocker-org/rocker-versioned/issues/208) this issue. Based on the reply, we can judge if it’s worth it for you to make a temp image.

Nitesh Turaga (10:34:16): > But it might be the case that@BJ Stubbsput some work in building a R-4.0 image already.

Nitesh Turaga (10:34:29): > @BJ Stubbscan you make that image public by any chance?

Mark Robinson (10:49:25): > @Mark Robinson has joined the channel

Mark Robinson (10:54:23): > @Sean Davis@Nitesh Turaganice! just also to say that i’m running a course that starts next thursday and would also like to access a rocker/rstudio image with R-4.0.0. Or, if@BJ Stubbs’s is public, that could be a solution. I’ll keep an eye on the issue above. Thanks all!

BJ Stubbs (11:35:04): > My image doesn’t work. I am not sure why. I might have tk build it from r-base and work though the pipeline

Nitesh Turaga (12:26:29): > Ok all, seems we got a reply.

Nitesh Turaga (12:26:49): > docker pull rockerdev/rstudio:4.0.0this might be the image I take a look at before they release the official images.

Mark Robinson (12:30:20): > testing this now.

Nitesh Turaga (12:58:05): > If you are trying to build the bioconductor image on top of this, just be warned that it’ll break a lot and you’ll have to manage the system dependencies from it.

Nitesh Turaga (13:03:33): > For people who want to take a look at rockerdev,https://hub.docker.com/u/rockerdev

Mark Robinson (13:50:26): > in case it’s useful to others .. > > (base) vpn-10-21-0-251:~ mark$ docker run -it rockerdev/rstudio:4.0.0 /bin/bash > root@1757e2801f05:/# R > > R version 4.0.0 (2020-04-24) -- "Arbor Day" > Copyright (C) 2020 The R Foundation for Statistical Computing > Platform: x86_64-pc-linux-gnu (64-bit) > [snip] > > > install.packages("png") > Installing package into '/usr/local/lib/R/site-library' > (as 'lib' is unspecified) > trying URL '[https://cran.r-project.org/src/contrib/png_0.1-7.tar.gz](https://cran.r-project.org/src/contrib/png_0.1-7.tar.gz)' > Content type 'application/x-gzip' length 24990 bytes (24 KB) > ================================================== > downloaded 24 KB > > * installing **source** package 'png' ... > **** package 'png' successfully unpacked and MD5 sums checked > **** using staged installation > **** libs > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include `libpng-config --cflags` -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c read.c -o read.o > /bin/bash: libpng-config: command not found > read.c:3:10: fatal error: png.h: No such file or directory > 3 | #include <png.h> > | ^~~~~~~~~~~~~ > compilation terminated. > make: ***** [/usr/local/lib/R/etc/Makeconf:167: read.o] Error 1 > ERROR: compilation failed for package 'png' > * removing '/usr/local/lib/R/site-library/png' > > The downloaded source packages are in > '/tmp/RtmpoocaiN/downloaded_packages' > Warning message: > In install.packages("png") : > installation of package 'png' had non-zero exit status > > >

Mark Robinson (13:54:32): > all fine after aapt-get install libpng-dev

Mark Robinson (14:26:02): > ok, yeah, obviously this is already covered inhttps://github.com/Bioconductor/bioconductor_docker/blob/e84d41cc83990051e584df25ef4a080a47648fa0/Dockerfile#L41

Mark Robinson (14:29:35): > @Nitesh Turagaare you building (or planning to build) this:https://github.com/Bioconductor/bioconductor_docker/blob/master/Dockerfile? Or are there other issues lurking?

Nitesh Turaga (14:33:24): > There are a lot of issues when we try to replace the inheritance of thebioconductor_docker:develimage to inherit fromrockerdev/rstudio:4.0.0. Mostly system dependencies which are not the same because of the shift fromdebiantoubuntuin the rockerdev stack. > > There seem to be a few issues lurking because of this shift that need to be fixed. > > But,@Mark Robinson, i’ll be working on this soon enough and we’ll try to get the RELEASE_3_11 and the devel images up.

Mark Robinson (14:34:53): > right. let me know if i can help.

Michael Lawrence (15:26:48): > @Michael Lawrence has left the channel

Sean Davis (16:53:58): > I tried out the rockerdev/geospatial and it did the trick for me. For those without esoteric needs, it seems that rockerdev/rstudio followed by installing BiocManager will get you a long way.

Mark Robinson (17:26:26): > I took a different path. Because we previously built an image on top of the bioconductor ones (for a course), i ended up stepping through the earlier biocDockerfileand fixed a few of the changed package names, removed python 2 (I think it was only meant to be maintained until 2020 anyways), commented out a couple things. Some stuff I didn’t manage to fix (xvfb, s6-overlay.., which are not needed for my course, I think). If it’s useful, I put details in a gitlab here:https://renkulab.io/gitlab/mark.robinson/bio334_spring2020/blob/master/README.md#how-the-images-were-built

Aaron Lun (17:26:44): > @Aaron Lun has left the channel

2020-05-02

Leonardo Collado Torres (01:57:17): > @Leonardo Collado Torres has joined the channel

2020-05-03

Nitesh Turaga (09:40:44): > Just to share some information about how i’m going about this, to be closer to the build system in linux we are building thebioconductor_docker:develimage on therockerdev/rstudio:4.0.0-ubuntu18.04. The linux build machine runs on Ubuntu 18.04.

2020-05-04

Michael Stadler (04:44:48): > @Michael Stadler has joined the channel

2020-05-05

Nitesh Turaga (13:37:30): > Is anyone else seeing the same issuehttps://github.com/rocker-org/rocker-versioned/issues/208#issuecomment-624118588?@Mark Robinsonor@Sean Davisif you guys are using the image?

Sean Davis (13:39:26): > I haven’t tried mounting a volume in yet,@Nitesh Turaga, so I probably wouldn’t have noticed any issue.

Mark Robinson (14:06:09): > sorry@Nitesh Turaga, also not mounting a volume here ..

Dan Bunis (14:08:33): > @Dan Bunis has left the channel

2020-05-08

Nitesh Turaga (08:33:41): > Hi<!here>, The new containers for RELEASE_3_11 and devel based on rockerdev/rstudio:R.4.0.0_ubuntu18.04 are now available on dockerhub.

Nitesh Turaga (08:33:52): > Please use them and let me know if you see any issues.

Leonardo Collado Torres (10:14:59): > YAY! woo:smiley:Thanks Nitesh!!!

Leonardo Collado Torres (19:50:47): > Nitesh, do you know if thegitversion changed from atbioconductor/bioconductor_docker:develfrom what it used to be? I see that the latest stablegitseems to be 2.17.1 for Ubuntu 18.04. > > On GitHub Actions from older logs I was getting version 2.20.1 like athttps://github.com/lcolladotor/derfinder/runs/622654306?check_suite_focus=true#step:3:27.

2020-05-09

Leonardo Collado Torres (10:52:36): > Fromhttps://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-18-04I got the instructions for compiling a newergitwhich takes about 2.5 minutes to run. I had to adapt the code a little bit to make sure it worked properly. > > sudo apt-get update -y > sudo apt install make libssl-dev libghc-zlib-dev libcurl4-gnutls-dev libexpat1-dev gettext unzip -y > wget[https://github.com/git/git/archive/v2.26.2.zip](https://github.com/git/git/archive/v2.26.2.zip)-O git.zip > unzip git.zip > cd git-* > make prefix=/usr all > sudo make prefix=/usr install > cd - > git --version > - Attachment (DigitalOcean): How To Install and Configure Git on Ubuntu 18.04 | DigitalOcean > With version control systems for your software development projects, you can share and collaborate on code. In this guide, we will install and configure the popular version control system Git on an Ubuntu 18.04 server.

2020-05-10

Sangram Keshari Sahu (09:29:14): > @Sangram Keshari Sahu has joined the channel

2020-05-11

Nitesh Turaga (08:08:59): > Hi@Leonardo Collado Torres, I didn’t check thegitversion. It’s what we get from rocker directly I guess. But whatgitissues did you see on the new devel image?

2020-05-14

Leonardo Collado Torres (12:04:06): > Hi Nitesh. GitHub actions (particularlyactions/checkout) has two types of behavior depending on whether git version 2.18 (or newer) is available or not. When it’s not available, it only downloads the files, while with newer versions it clones the repo you are working with.pkgdownneeds to have a functional repo in order to update thegh-pagesbranch with the new output. That’s why I had to figure out how to install a newer version ofgit(newer than the new default of2.17.1). It takes 2-3 minutes, so it’s not bad. Though I guess that if you installed a newergitwhen making the containers, it would be nice:smiley:(yet it goes away fromsudo apt-getso maybe you don’t like/want that)

Nitesh Turaga (16:52:30): > Hi@Leonardo Collado Torres, in that case, we will not make any changes to the docker container.

2020-05-15

Leonardo Collado Torres (09:30:27): > Ohh ok. Is it because it’s not thegitversion provided byapt-getfor Ubuntu 18.04?

2020-05-18

Huipeng Li (11:03:58): > @Huipeng Li has joined the channel

2020-05-19

Leonardo Collado Torres (11:26:42): > If anyone else is running into installation issues with the containers because ofhttps://packagemanager.rstudio.com/all/linux/bionic/latest/checkhttps://github.com/rocker-org/rocker-versioned2/issues/50andhttps://github.com/r-lib/remotes/issues/501. Basically, at Rocker they recently (6 days ago) changed the environment variable forCRANwhich changes thereposfor it.

2020-06-11

Kozo Nishida (03:13:12): > @Kozo Nishida has joined the channel

2020-07-15

Spencer Nystrom (08:49:58): > @Spencer Nystrom has joined the channel

2020-07-22

Aedin Culhane (16:49:16): > Hi, I was playing with docker desktop but it seems to have limited functionality, it doesn’t include a search of dockerhub, GUI options to pull or run an image. Is there another option ?

2020-07-27

Leonardo Collado Torres (13:25:12): - File (PNG): Capture.PNG - File (PNG): Capture2.PNG

Leonardo Collado Torres (13:26:05): > just tried Docker on my Windows laptop and hm…. I have the latest Windows home according to Windows update but not according to the Docker Desktop installation software =(

Leonardo Collado Torres (13:28:12): > tryinghttps://www.microsoft.com/en-us/software-download/windows10right now

Nitesh Turaga (14:02:51): > Do you have windows profressional?

FelixErnst (14:12:04): > Apparently not. This will probably not work, since Docker relies on Hyper-V as a Hypervisor, which you only get on Pro

FelixErnst (14:12:59): > I guess technically it might rely on Windows Subsystem for Linux 2, which then works only with Hyper-V enabled.

FelixErnst (14:13:25): > Keep in mind, that Hyper-V and VirtualBox don’t work on the same machine.

FelixErnst (14:15:05): > An alternative solution would be to install ubuntu in VirtualBox running docker. I know, this is a bit indirect, but it should work on Windows Home

Nitesh Turaga (15:23:16): > I think the easiest way is to get Windows 10 professional.

Nitesh Turaga (15:23:46): > It works just fine on my Windows machine (which has windows 10 pro)

FelixErnst (15:24:10): > of course it does

FelixErnst (15:24:48): > but you have to have pro.

FelixErnst (15:25:21): > and you cannot install Linux natively alongside Windows due to Hypervisor. At least I never got it to work

FelixErnst (15:26:41): > My point is that there are pros and cons to turning Hyper-V on and in the above situation there are alternative solutions than getting pro

Nitesh Turaga (15:40:48): > @FelixErnstI’ll take your word for it. I do not have too much experience with windows

Leonardo Collado Torres (16:59:54): - File (PNG): image.png

Leonardo Collado Torres (16:59:58): > I’m getting farther now, still using Windows 10 Home

Leonardo Collado Torres (17:26:44): > success ^^ - File (PNG): Capture3.PNG

Leonardo Collado Torres (17:27:47): > Though I would like to be able to install Docker on a particular disk and it seems like that involves a workaround as detailed inhttps://forums.docker.com/t/docker-windows-installer-doesnt-let-you-choose-drive/11205/6 - Attachment (Docker Forums): Docker windows installer doesn’t let you choose drive > Found a work around of sorts. start an Admin command prompt (cmd.exe) and change directory to wherever you’ve saved the installer. change the following environment variables to start with the drive letter you want ( e.g: set=ProgramFile=F:Files ) Execute the installer by running from this cmd window. Haven’t fully tested, but it looks like it installed to the right spot now and not defaulted to C: drive.

Shian Su (18:49:46): > @Shian Su has left the channel

2020-07-28

FelixErnst (00:26:45): > @Leonardo Collado Torreswow. I didn’t know that was now possible on Home. Learned something:slightly_smiling_face:

Sean Davis (11:12:41): > It might be worth considering something like this:https://github.com/GoogleContainerTools/container-structure-testfor continuous integration testing of the Bioc containers. That particular toolset might not be perfect, but I toss it out as an example. Given the edge cases discovered during workshop building and issues with upstream changes, it seems like this is a good time….

Nitesh Turaga (11:13:42): > Thanks Sean! I’ll take a look

2020-07-31

Saulius Lukauskas (03:14:39): > @Saulius Lukauskas has joined the channel

Sunil Nahata (09:10:48): > @Sunil Nahata has joined the channel

Erick Cuevas (14:12:30): > @Erick Cuevas has joined the channel

Dr Awala Fortune O. (16:22:08): > @Dr Awala Fortune O. has joined the channel

Leopoldo Valiente (16:41:14): > @Leopoldo Valiente has joined the channel

2020-08-05

Hans-Rudolf Hotz (07:38:58): > @Hans-Rudolf Hotz has joined the channel

2020-08-08

Alan O’C (10:21:41): > @Alan O’C has joined the channel

2020-08-09

Sean Davis (13:32:19): > Hi,@Nitesh Turaga. Does this work as expected for you to generate a docker-based Rstudio without login? > > docker run --rm \ > -p 8788:8787 \ > -e DISABLE_AUTH=true \ > bioconductor/bioconductor_docker:devel > > All I ever get is a screen that never connects to Rstudio. TheDISABLE_AUTHis the variable that leads to this behavior. The normal specification of password works fine. - File (PNG): image.png

FelixErnst (15:33:04) (in thread): > Hi@Sean Davis, I had to start a container as well, did a pull and didn’t have a problem. Is the output in the console looking ok?

Sean Davis (15:37:28) (in thread): > Thx for the reply. Were you usingDISABLE_AUTH,@FelixErnst? And what container did you use, just so I can compare apples to apples?

FelixErnst (15:40:15) (in thread): > yes I did useDISABLE_AUTH(basically did a copy paste from your snippet for the test). The label of the images returns3.12.15 > > docker inspect --format="{{.Config.Labels.version}}" bioconductor/bioconductor_docker:devel >

Nitesh Turaga (19:31:49) (in thread): > Hi@Sean DavisIt works fine for me as well. This might be non consequential in all rights…but can you trydocker system prunebefore trying again. Full disclosure, it will get rid of any orphan images you are building.

2020-08-10

Nitesh Turaga (10:04:39): > Did that work@Sean Davis?

Nitesh Turaga (15:57:16): > Great!

2020-08-13

Frederick Tan (16:38:39): > @Frederick Tan has joined the channel

Geet (16:47:22): > @Geet has joined the channel

Dirk Eddelbuettel (17:14:24): > @Dirk Eddelbuettel has joined the channel

Geet (18:40:08): > Hi,@Nitesh Turaga,@Sean DavisIs it recommended to mention R libraries’ version to install and run R packages in the docker image?

Sean Davis (19:11:13): > You really can really install only the current versions of packages in R.

Dirk Eddelbuettel (19:15:23) (in thread): > As@Sean Davissaid, thebase Rfunctionality ininstall.packages()has no notion of versions. > > One can work around that and layer but that gets nonstandard and fragile quickly. MRAN did it, RSPM tries it to, you can hand-manage. > > But in short: no. We prefer to work on a repo status where “everything current works with everything else”. CRAN works very hard to ensure that, and BioC does too via the biannual releases.

Geet (19:30:23): > Thank you@Sean Davisand@Dirk EddelbuettelI am new to Docker, so still in the exploring mode.

Dirk Eddelbuettel (20:11:06): > Sure thing. It all seems pretty mysterious at times, but I find it helps to think of it as “just shell scripts” that allow you to put together executable bundles. But the key is “inside” it is just a copy operation (as Sean showed you) or aninstall.packages()(however disguised). Which works both ways: if you can do it in the shell you can probably get it done for Docker too.

Pariksheet Nanda (20:44:14): > @Pariksheet Nanda has joined the channel

Geet (20:48:41): > @Dirk EddelbuettelThank you for simplifying it, and surely thinking of Docker as a shell script boosts my confidence. I was all over the place otherwise.

Daniela Cassol (22:51:46): > @Daniela Cassol has joined the channel

2020-08-14

Nitesh Turaga (08:36:20): > Thanks@Dirk Eddelbuetteland@Sean Davisfor getting to this question.

2020-08-17

Charlie George (14:36:43): > @Charlie George has joined the channel

2020-08-18

Geet (18:29:20): > I am getting the following error while running the docker image “Error in grid.newpage() : Could not open file ‘output/Plot.png’ Calls: ggsave … grid.draw.ggplot -> print -> print.ggplot -> grid.newpage” . I keep getting this error, and the execution halted even after I removed the ggsave option from R script. Any idea how to fix it? Thank you!

Alan O’C (18:32:52): > Is it a ggplot object or a set of combined plots (eg, survminer output)?

Geet (18:33:14): > yes

Alan O’C (18:34:09): > That was an either/or question not a yes/no question

Geet (18:34:46): > sorry, it is a ggplot object

Geet (18:35:59): > but even I removed the ggsave, still got the same error

Alan O’C (18:37:01): > Alright, if it was a grid object I thought I had seen the issue before. It seems like it may just be a file permission issue (“could not open file”). Does the docker image have write permission in the Rstudio working directory?

Geet (18:43:13): > I didn’t add any permission . How to handle it differently if it would be a survminer or ggforest?

Geet (18:44:28): > I just checked and yes, the docker image have write permission

Alan O’C (18:44:49): > Fair enough, I’m all out of relevant ideas.

2020-08-19

Ruth Isserlin (11:44:23): > Often when I run through a whole pipeline that is set up as a RNotebook in my bioc docker rstudio part way through I get a message saying that R had to restart and I lose the entire R session and have to restart. It happens when I “run all” and also when I step through the different steps in my notebook. (For the most part when I render the notebook from the command line instead of in Rstudio through the web browser it works. Definitely doesn’t crash as often). Has anyone had similar issues? Is there a way to stop R from crashing and restarting?

Nitesh Turaga (11:45:10): > Where are you running your docker image?

Nitesh Turaga (11:45:14): > on your local machine?

Ruth Isserlin (11:56:18): > Yes. on my local machine (I have the same behaviour on my desktop computer and my laptop. I thought it might a resources issue but my desktop is very powerful)

Nitesh Turaga (12:03:21): > as a follow up question, does the docker container you are working with, have access to those resources ?? > > If you run command below, it’ll show what percentage of the resources available your container is using, > > $ docker stats > > for example, when I run a docker image on my machine and check usage, > > CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS > a8e3018b0278 modest_gagarin 0.00% 952KiB / 11.71GiB 0.01% 976B / 0B 0B / 0B 1 > > Notice the 11.71GiB limit on memory usage. I’ve assigned about 12GB on memory on my local machine for the container to run.

Nitesh Turaga (12:05:08): > Since I do most of my R/Bioc work on the docker image, I allocate almost 75% of my resources on my local machine to docker.

Nitesh Turaga (12:05:20): - File (PNG): Screen Shot 2020-08-19 at 12.04.35 PM.png

Nitesh Turaga (12:07:46): > Also, it depends on the size of the “job” you are running….if your “run all” exceeds the limits of your docker containers capabilities …it’s likely that it could have issues. Running RStudio has more memory requirements on the docker container than using it via command line. You can cross check using thedocker statfunction and checking one container with RStudio running and one container without RStudio running.

Nitesh Turaga (12:13:09): > For example,vibrant_hooverhas RStudio running andmodest_gagarindoes not have an RStudio server running. > > ~ ❯❯❯ docker ps ✘ 130 > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES > ba915820edea bioconductor/bioconductor_docker:RELEASE_3_11 "/init" 40 seconds ago Up 39 seconds 0.0.0.0:8787->8787/tcp vibrant_hoover > a8e3018b0278 bioconductor/bioconductor_docker:RELEASE_3_11 "bash" 11 minutes ago Up 11 minutes 8787/tcp modest_gagarin > > When I check memory requirements > > CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS > ba915820edea vibrant_hoover 0.46% 70.34MiB / 11.71GiB 0.59% 118kB / 2.71MB 0B / 0B 12 > a8e3018b0278 modest_gagarin 0.00% 952KiB / 11.71GiB 0.01% 1.23kB / 0B 0B / 0B 1 > > You can see thatvibrant_hooverhas some amount of memory being used here. I’m not running any R code in these containers. These are merely starting the containers and showing you.

Ruth Isserlin (12:15:53): > Wow. Thanks! exactly the sort of answer I was looking for. I will check out my memory limits. I think that is the issue but I couldn’t figure out how to control allocations.

Ruth Isserlin (12:17:46): > I had only 2GB memory allocated to the docker instance (definitely not enough!)

Nitesh Turaga (12:23:16): > No problem!

2020-08-27

Spencer Nystrom (12:04:36): > I’m looking to build a docker container to use for CI tests over gh-actions for a package. The short version is: I want to run my tests across multiple different versions of a specific software. Are there any guides, or design conventions for building something like this? I’ve dug through howr-libmakes some of theirCMD CHECKcontainers, and it’s a total web, so worried I’m missing some key design principles.

Sean Davis (12:20:49) (in thread): > What do you mean by “multiple different versions of a specific software”?

Spencer Nystrom (12:23:53) (in thread): > I have a package that talks to “software v1”. As the software updates I’ll want to support “v1” and “v2” if possible, so I want to have the ability to easily make a second container, eg “myContainer:v1” and “myContainer:v2”.

Spencer Nystrom (12:24:14) (in thread): > And runR CMD CHECKinside those.

Spencer Nystrom (12:25:48) (in thread): > I think what I’m after is the ability to have a gh-action with aruns-onmatrix.

Sean Davis (12:28:31) (in thread): > I’d invert things a bit. Run R CMD CHECK on the code (likely placed into separate branches on github). Then, after passing checks, build the container with appropriate tags. Set GH-actions to run on “push” for any branch. Then, pushes to V2 will test and build V2 while V1 remains unchanged. V1 works the same way. If you happen to push changes to both branches (bug fix, for example), your gh-actions runs for both branches and generates both docker containers.

Sean Davis (12:29:50) (in thread): > See here:https://github.com/seandavi/BuildABiocWorkshop2020for a repo that does the installation of dependencies, runs R CMD CHECK, builds a docker image, and then pushes the resulting docker image to dockerhub.

Sean Davis (12:31:24) (in thread): > That repo also deploys a pkgdown site, but you can simply remove that piece (https://github.com/seandavi/BuildABiocWorkshop2020/blob/master/.github/workflows/basic_checks.yaml#L68-L73) from the gh-actions workflow.

Spencer Nystrom (12:33:22) (in thread): > So the idea is first run the unit tests that don’t require “software:vX”, then pull each docker container with “vX” installed, and run the unit tests in those?

Sean Davis (12:41:31) (in thread): > Not quite. The code for vX is checked out from github into the github actions workspace. Dependencies are installed into R in the github actions workspace. Then, R CMD CHECK is run on the code in the repo. If it passes, that code is used to build a docker container. No need to test code in the container since we know the code works after check/testing. I am assuming that “code” here is equivalent to a repo. A container will never be created for code that does not pass R CMD CHECK first.

Spencer Nystrom (12:43:25) (in thread): > oooh, ok, gotcha. This is a great idea, and actually would technically eliminate my need to even build a container. But doing so enables shipping them so people can try the package out. Perfect!

Sean Davis (12:45:45) (in thread): > Github actions are very flexible, so you can also do online doc building, updating website with version info, etc. If you place those pieces after your checking/testing, you sort of guarantee that nothing gets changed unless the code works.

Spencer Nystrom (12:47:47) (in thread): > Yeah, I’ve just started playing with them and have them running smoothly on a less intense package of mine, so now scaling it up for the workhorse package. I was already digging around in your BiocWorkshop builder for examples as well, so thanks for the advice, really helps frame how to get the most out of the tool!

Sean Davis (12:49:54) (in thread): > It may be helpful to actually go through the motions outlined:https://seandavi.github.io/BuildABiocWorkshop2020/articles/HOWTO_BUILD_WORKSHOP.html. That will give you a working version, complete with your credentials, to test out.

2020-08-31

Spencer Nystrom (12:15:04): > I’m having a weird issue with singularity where if I dosingularity pulldocker://bioconductor_docker:develand run singularity from that image the Rstudio session ends itself after ~30seconds, but if I dosingularity pullshub://bioconductor_docker:latest(which is built from thedeveldocker image) my job runs fine. So the issue appears to be that .sif images that I build myself from docker fail, but .sif files from singularity hub built from the same image work? Anyone seen this issue before? > > Also, can someone confirm that theshubimages are truly up to date to rule out the possibility that this is a regression in a newer version of the docker image?

Spencer Nystrom (12:23:46) (in thread): > Ah, ok theshubbuilds are from Jan 2, 2020 so a bit out of date suggesting a regression.

Sean Davis (12:25:46) (in thread): > @Nitesh Turagacan confirm, but I do not think the singularity images are getting regular updates like the docker images and support for singularity is sort of ad hoc for the time being.

Spencer Nystrom (12:27:10) (in thread): > Yeah, just found that. Such a weird problem. Tracking it down a little more now, but my guess is this is really a problem withrocker.

Martin Morgan (13:23:26) (in thread): > A colleague had problems with recent RStudio / singularity > > 10 Aug 2020 20:41:32 [rserver] ERROR system error 11 (Resource temporarily unavailable) [description: Could not acquire revocation list file lock]; OCCURRED AT rstudio::core::Error rstudio::server::auth::handler::initialize() src/cpp/server/auth/ServerAuthHandler.cpp:570; LOGGED FROM: int main(int, char* const*) src/cpp/server/ServerMain.cpp:674 > > this was with > > singularity pull --name rstudio.simg docker://rocker/rstudio:4.0.2 > > and “Looks like the version that works is RStudio 1.2.5042” > > Could be a complete red herring…

Spencer Nystrom (13:24:22) (in thread): > I just tracked this down to arockerissue. Things work fine with3.6.3but any build afterwards, including3.6.3-ubuntu18.04fails.

Spencer Nystrom (13:26:31) (in thread): > Interesting it could be an rstudio issue, I’m trying to compare the dockerfiles between working / broken versions now. Some chatter about similar failures atthis issue

2020-09-09

Nicholas Cooley (13:08:27): > Hi I’m having a funny issue where if I build a container from R-Base and then pull in packages afterwards I get this error from BiocGenerics: > > Error: package or namespace load failed for 'BiocGenerics': > package 'BiocGenerics' was installed before R 4.0.0: please re-install it > Error: package 'BiocGenerics' could not be loaded > > My session info looks fine, am i just missing version specification when i install my packages? > > R version 4.0.2 (2020-06-22) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Debian GNU/Linux 10 (buster) > > Matrix products: default > BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0 > LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0 > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] parallel stats graphics grDevices utils datasets methods > [8] base > > loaded via a namespace (and not attached): > [1] compiler_4.0.2 >

Dirk Eddelbuettel (13:27:47): > Can you runapt-cache r-bioc-genericsin the container so that we can see where this package comes up?

Nicholas Cooley (13:30:44): > Sure, I’m rebuilding stuff now, unfortunately it’s a pretty big container. While looking it seems as though > > RUN Rscript -e "install.packages(c('remotes', 'docopt', 'BiocManager', 'igraph'))" > > is pulling in the incorrect version of BiocManager, so I’m adding in: > > RUN Rscript -e "BiocManager::install(version = '3.11')" > > Afterwards to correct it? I’ll update with results when it completes!

Nicholas Cooley (14:15:28): > Ok so an intermediate update;apt-cache r-bioc-genericswon’t run in the container > > currently R-base is R 4.02 so: > > FROM r-base > > RUN apt-get update > RUN apt-get -y install libgmp-dev > > RUN Rscript -e "install.packages(c('remotes', 'docopt', 'BiocManager', 'igraph'))" > RUN Rscript -e "BiocManager::install(version = '3.11')" > > will error out with: > > Step 5/20 : RUN Rscript -e "BiocManager::install(version = '3.11')" > ---> Running in 453fd3410d24 > Error: Bioconductor version '3.11' requires R version '4.0'; see[https://bioconductor.org/install](https://bioconductor.org/install)Execution halted > > however i can specify: > > FROM r-base:4.0.0 > > So we’re seeing how that goes.

Nitesh Turaga (14:16:27): > Which version of the container do you have?

Nicholas Cooley (14:19:08): > previously it was just pulling latest from R-base, it’s now specifying 4.0.0, and has passed the versioning of BiocManager without errors.

Nitesh Turaga (14:21:40): > Ok, maybe try this and send me the output,docker inspect --format '{{ index .Config.Labels.version}}' bioconductor/bioconductor_docker:develsend me the version number

Nicholas Cooley (14:26:38): > I get back: > > npc19$ docker inspect --format '{{ index .Config.Labels.version}}' bioconductor/bioconductor_docker:devel > 3.11.1 >

Nitesh Turaga (14:27:49): > The current version of the image is, > > $ docker inspect --format '{{ index .Config.Labels.version}}' bioconductor/bioconductor_docker:devel > 3.12.19 >

Nicholas Cooley (14:28:46): > But I’m not building off the bioconductor build, I’m pulling directly from R-base?

Nitesh Turaga (14:29:48): > I’m not following here. So you are using the rocker/r-ver image directly and installing everything on top of that yourself?

Nicholas Cooley (14:37:45): > Yes, so far: > > FROM r-base > > RUN apt-get update > RUN apt-get -y install libgmp-dev > > RUN Rscript -e "install.packages(c('remotes', 'docopt', 'BiocManager', 'igraph'))" > RUN Rscript -e "BiocManager::install(version = '3.11')" > RUN Rscript -e "BiocManager::install('DECIPHER')" > RUN Rscript -e "install.packages('dendextend')" > > well throw an error because BiocManager won’t like versioning to 3.11 in R 4.0.2 and: > > FROM r-base > > RUN apt-get update > RUN apt-get -y install libgmp-dev > > RUN Rscript -e "install.packages(c('remotes', 'docopt', 'BiocManager', 'igraph'))" > RUN Rscript -e "BiocManager::install('DECIPHER')" > RUN Rscript -e "install.packages('dendextend')" > > Will complete the build just fine but will install everything with BiocManager 3.10.

Nicholas Cooley (14:38:11): > Which will then not load and complain.

Nicholas Cooley (14:40:24): > While, this is finally successful: > > FROM r-base:4.0.0 > > RUN apt-get update > RUN apt-get -y install libgmp-dev > > RUN Rscript -e "install.packages(c('remotes', 'docopt', 'BiocManager', 'igraph'))" > RUN Rscript -e "BiocManager::install(version = '3.11')" > RUN Rscript -e "BiocManager::install('DECIPHER')" > RUN Rscript -e "install.packages('dendextend')" > > Which seems like a bit of rigamarole for getting versioning correct.

Dirk Eddelbuettel (14:47:30): - File (Plain Text): Untitled

Dirk Eddelbuettel (14:48:03): > Minor rewrite suggestion above. The containers with the newestlittlerinstalled also have a helperinstallBioc.r.

Nicholas Cooley (15:49:20): > Hi Dirk, can you help me understand something about your suggestion above? If i don’t specify the version for r-base, it pulls latest, which i’m pretty sure is the same as 4.0.2 currently? > > When i don’t specify the version, theRUN Rscript -e "BiocManager::install(version = '3.11')"will throw an error during build. But, specifying the version prevents the error and the container builds just fine. Is there something I’m missing here? IsFROM r-base:4.0.2not equivalent toFROM r-basewhen latest == 4.0.2 ?

Dirk Eddelbuettel (16:12:24): > @Nicholas CooleyCurrently busy for work, sorry. Maybe try to work with@Nitesh Turaga. But yes, 4.0.2 is latest and good. 4.0.0 is not latest and known buggy (in minor aspect at the base R level).

Dirk Eddelbuettel (16:13:30): > As for the base R and BioC version matches, that is more common pitfall but we have many many experienced BioCers here who can probably help you….

Nitesh Turaga (16:36:52): > Hi@Nicholas CooleyCan I ask first why you are trying to build fromr-base:4.0.2? Specifically why this image as opposed to something the rocker/r-ver:4.0.2 gives you?

Nitesh Turaga (16:37:44): > What is the use case of building a new image when you can just use the bioconductor/bioconductor_docker:devel or RELEASE_3_11 image and install the package DECIPHER?

Sean Davis (16:41:11) (in thread): > Thelatesttag is NOT equivalent to a specific version. Instead,latestrefers to the most recent successfully pulled image available on your system. If you want to get thecurrentlatesttag from a repo, you must clear any cached versions of the image locally first. Then, docker will pull a new image. This is REALLY confusing and surprising to many (myself included), so using a specific tagged version is a best practice and leads to fewer surprises.

Sean Davis (16:43:17) (in thread): > Just a note that in docker terms,latestwill get you the latest successfully pulled image if there is one already available, regardless of thenewestversion available on dockerhub. Besides good practice, this is another good reason to use a version tag in a dockerfile. It can save some headache later.

Nicholas Cooley (16:46:48): > I’m passing this container around on the open science grid and one of the suggestions i was given originally was to keep the container as small as possible. So since I don’t need any of the python stuff or RStudio, I just built from scratch off r-base.

Dirk Eddelbuettel (16:47:53) (in thread): > Very good point but recall that we were talking about a Dockerfile. And if and when you build athub.docker.com, you do want the latestand get it. And that is afeaturetoo. I have maybe a dozen Dockerfiles deriving/refining r-base and I wouldhateto have to explicitly roll them forward each R release. Two sides to every coin. > > But you are entirely correct thatlocallyit may give you an older version that happens to be thelatestyou as your local admin have set it up. Or you may indeed be a pull or two behind.

Nicholas Cooley (16:55:55): > Also I guess i didn’t describe the problem quite that well above, but if myFROM r-basecommand does not include a version, even though that command should build fromr-base:latest/r-base:4.0.2, versioning withRUN Rscript -e "BiocManager::install(version = '3.11')"will error during build, but just changing toFROM r-base:4.0.2allows everything to install correctly. > > So my question is, does not including the version inFROMnot work the way that i thought it did, or is something else happening that I should know about?

Sean Davis (17:44:46): > r-base:latestisnot necessarily the same asr-base:4.0.2. See my response in the thread above or follow along here:https://medium.com/@mccode/the-misunderstood-docker-tag-latest-af3babfd6375 - Attachment (Medium): The misunderstood Docker tag: latest > Docker images have a tag named latest which doesn’t work as most people expect. I’ve listened to numerous speakers and Docker 101 talks…

Nicholas Cooley (18:45:44): > Oh i missed your response previously, sorry about that! Thanks!

Nicholas Cooley (22:16:56) (in thread): > So if you don’t mind explaining a little further, i was under the impression that--no-cachewould override whatever version exists locally?

2020-09-14

Geet (12:43:14): > I am trying to automate copying the container’s results to the host system using the following commands in a bash script,but it is not working. Has anybody done anything similar: > container_id = $(docker ps -q -l) docker cp $container_id :/work/data/proc/. data/proc > docker cp $container_id :/work/output/. output

Sean Davis (13:53:05) (in thread): > Is there a reason not to use a mounted volume to avoid having to copy data at all?

Geet (19:27:28) (in thread): > @Sean Davisthere is no reason

Sean Davis (19:54:26) (in thread): > I’d give the mounted volume approach (-v) a try, then.

2020-09-15

FelixErnst (02:10:52) (in thread): > And in addition the format of directories involved in copying are wrong, IMO. It is either a directory/files without any:, which assumes that the path are the same on the host and the container or it ispath:path. I think depending on your setup, explicit paths must be used on both sides: This.:pathshould not work, but this should$(pwd):path.

2020-09-24

FelixErnst (13:09:18): > Regarding the system dependencies of R packages as touched on by@Dirk Eddelbuettel@Martin Morgan@Nitesh Turagaon todays developers-forum: How do you/can I convert system dependencies into package names depending on the OS used? For example ubuntu/debian package names differ from fedora/centOS/RHEL. Can you share some best practices for this task or point me in the direction of some resources? Thanks in advance

Dirk Eddelbuettel (13:13:35): > In general, cross-distro cataloging of dependencies is a problem. Gabor has a database for that in RHub “somewhere” derived form SystemRequirements: and, I believe, underlying their (loose) resolution in the RSPM support. Search at RHub, it’s there somewhere (and a proper REST service). > > What we have done when building deb package is to simply nail it down for one distro; Inaki does the for CRAN “Copr” repo for Fedora. BTW the 1st blog post, I can show more later:http://dirk.eddelbuettel.com/blog/2020/08/26#029_introducing_bspm - Attachment (dirk.eddelbuettel.com): Thinking inside the box > Dirk Eddelbuettel, R, C++, Rcpp

Dirk Eddelbuettel (13:25:59): > (Also, mappinginstall.packages(c("data.table", "digest"))tosudo apt install r-cran-data.table r-cran-digestis of course not limited to containers, but nice to try in containers. I built two “narrow” sets that have the mechanics enable:rocker/r-rspmfor Ubuntu 18.04 + 20.04, androcker/r-bspmfor the same two plus Debian, and I recently added first drafts of command-line tools tolittlernamelyinstallRSPM.randinstallBSPM.rwhich then do it one call at a time. That’s is in littler’s repo on GH, but not yet on CRAN.)

FelixErnst (13:33:12): > Thanks!

2020-09-25

Nitesh Turaga (13:53:06): > https://azure.microsoft.com/en-us/blog/accelerating-genomics-workflows-and-data-analysis-on-azure/ - Attachment (Microsoft Azure): Accelerating genomics workflows and data analysis on Azure > Microsoft Genomics has released several open source projects on GitHub, including Cromwell on Azure, Genomics Notebooks and Bioconductor support for Azure. We have also made available a growing list of genomics public datasets on the Azure Open Dataset platform

2020-09-27

Leonardo Collado Torres (16:09:11): > Hi, I’m having an issue with thedeveldocker that you can reproduce with: > > > httr::GET("[https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz](https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz)") > Error in curl::curl_fetch_memory(url, handle = handle) : > error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature type >

Leonardo Collado Torres (16:10:01): > googling lead me tohttps://stackoverflow.com/questions/58342699/php-curl-curl-error-35-error1414d172ssl-routinestls12-check-peer-sigalgwrand in particular the last answerhttps://stackoverflow.com/a/62914628/9374370which mentions upgradingopenssl - Attachment (Stack Overflow): PHP CURL - cURL error 35: error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature type > I want to make a curl request in PHP 7.3.90 curl -V curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1… - Attachment (Stack Overflow): PHP CURL - cURL error 35: error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature type > I want to make a curl request in PHP 7.3.90 curl -V curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1…

Dirk Eddelbuettel (16:20:40): > FWIW that reproduces in a basic rockerr-basecontainer just addingcurlandhttrand their depends and also happens outside of R usingwget:

Dirk Eddelbuettel (16:21:00): - File (Shell): Untitled

Dirk Eddelbuettel (16:21:07): > So could be “them” no us.

Leonardo Collado Torres (16:28:21): > do you mean something with the host?

Leonardo Collado Torres (16:28:30): > idies.jhu.eduin this case

Dirk Eddelbuettel (16:29:20) (in thread): > Yes. That’s why I showed you the simplest case, a directwgetcall on the shell.

Leonardo Collado Torres (16:30:33) (in thread): > ok, thanks. I don’t get the error outside the container though, like on my macOS > > that’s why I thought that it was a container issue

Dirk Eddelbuettel (16:31:11) (in thread): > Which is a valid starting point and why I tried to help with a simpler container. Basic debugging -> stripping off layer by layer.

Dirk Eddelbuettel (16:32:41) (in thread): > Good point though trying outside of Docker. When I do that … DNS hangs at first (rarely see that) and then thesame error happens. So surprised macOS behaves for you.

Leonardo Collado Torres (16:47:44): > If I upgrade openssl to version 1.1.1g on bioc-devel docker following the instructions athttps://askubuntu.com/a/1102966, then I can run outside of R: > > # openssl version > OpenSSL 1.1.1g 21 Apr 2020 > > and in R: > > > httr::GET("[https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz](https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz)") > Response [[https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz](https://idies.jhu.edu/recount3/data/human/data_sources/sra/metadata/66/ERP110066/sra.sra.ERP110066.MD.gz)] > Date: 2020-09-27 20:46 > Status: 200 > Content-Type: application/x-gzip > Size: 125 kB > <BINARY BODY> > - Attachment (Ask Ubuntu): How to upgrade OpenSSL 1.1.0 to 1.1.1 in Ubuntu 18.04? > I have been running a production server with Ubuntu 18 installed. Recently, I found that my web application was not allowed on some of the firewalls installed at the customer location. I found th…

Dirk Eddelbuettel (16:50:04) (in thread): > ~I would be careful here. What I just showed you is that it also fails on plain Ubuntu 20.04 so random upgrades may not fix it.~Scratch that. Let me just say that we should find a careful way to get the newer package.

Dirk Eddelbuettel (16:51:20) (in thread): > ~~~What I can tell you is that there are (IIRC) ~~three~~~ different ssl libraries and you may have to try linking curl/wget/… with the different ones. There is the openssl and a gnutsl one and if memory serves another one.~~~~Scratch that too.

Dirk Eddelbuettel (16:52:42) (in thread): > But it may well be it. I am indeed at ‘f’ (and failing) not ‘g’.

Dirk Eddelbuettel (16:56:23) (in thread): > And in plain Rockerr-base(using Debian testing) we have ‘g’ and it works.

Leonardo Collado Torres (16:57:02) (in thread): > ohh interesting, thanks

Leonardo Collado Torres (16:57:14) (in thread): > I’m looking athttps://launchpad.net/ubuntu/+source/opensslto see if1.1.1gis somewhere there - Attachment (Launchpad): openssl package : Ubuntu

Dirk Eddelbuettel (16:57:41) (in thread): > Getting itsbinarypackages for CRAN’scurlandhttrstill fails withhttrthough.

Dirk Eddelbuettel (16:58:23) (in thread): > That is a good PPA, if they have a package for your distro (bionic, 18.04, right?) it may well help.

Dirk Eddelbuettel (16:59:22) (in thread): > I gotta work on work on something else. Good sleuthing on your part, and sorry for bad earlier suggestions. Interesting to see that is indeed the newer software that helps. I tend to like being on Debian testing or curent Ubuntu LTS for these reasons….

Leonardo Collado Torres (17:00:18) (in thread): > thanks Dirk! > > And from the output below, bioc-devel is not on ubuntu 20.04 (it was on 18.04 earlier this year when we last chatted) > > # cat /etc/os-release > NAME="Ubuntu" > VERSION="20.04.1 LTS (Focal Fossa)" > ID=ubuntu > ID_LIKE=debian > PRETTY_NAME="Ubuntu 20.04.1 LTS" > VERSION_ID="20.04" > HOME_URL="[https://www.ubuntu.com/](https://www.ubuntu.com/)" > SUPPORT_URL="[https://help.ubuntu.com/](https://help.ubuntu.com/)" > BUG_REPORT_URL="[https://bugs.launchpad.net/ubuntu/](https://bugs.launchpad.net/ubuntu/)" > PRIVACY_POLICY_URL="[https://www.ubuntu.com/legal/terms-and-policies/privacy-policy](https://www.ubuntu.com/legal/terms-and-policies/privacy-policy)" > VERSION_CODENAME=focal > UBUNTU_CODENAME=focal >

Leonardo Collado Torres (17:21:01): > Actually, I can dodge all of this by usinghttpinstead ofhttps^^

Dirk Eddelbuettel (17:39:12) (in thread): > I also run 20.04 here, but as seen, still hadwgetfail.

2020-09-28

Hervé Pagès (13:14:06): > @Hervé Pagès has joined the channel

Hervé Pagès (13:19:39): > @Leonardo Collado Torres@Dirk EddelbuettelFWIW the problem is specific to Ubuntu 20.04 and is also affecting our new Linux builder nebbiolo1. Seems to be a combination of server misconfiguration + increased security level in Ubuntu 20.04 + a bug in OpenSSL 1.1.1. Seehttps://github.com/Ensembl/ensembl-rest/issues/427for the details andhttps://askubuntu.com/questions/1233186/ubuntu-20-04-how-to-set-lower-ssl-security-levelfor a fix that doesn’t require upgrading OpenSSL (still requires sudo access though). But yeah, in the mean time, the easy fix is to ditch HTTPS for good ol’ HTTP. - Attachment (Ask Ubuntu): Ubuntu 20.04 - how to set lower SSL security level? > I’d like to ask if there’s a way to lower SSL security level to 1 on Ubuntu 20.04, since I’m receiving: 141A318A:SSL routines:tls_process_ske_dhe:dh key too small when trying to curl the website….

Dirk Eddelbuettel (13:20:24) (in thread): > Thanks for the sleuthing!

Hervé Pagès (13:21:55) (in thread): > no problem, these are the joys of taking care of the build machines

Leonardo Collado Torres (13:23:09) (in thread): > thanks Hervé! those are great links and the thread on Ensembl was very clear to follow

2020-10-01

Martin Morgan (06:17:02) (in thread): > @Hervé Pagèswhat’s the advice to, e.g.,https://stat.ethz.ch/pipermail/bioc-devel/2020-September/017261.html? Will the build system get some sort of patch, with the rationale that ubuntu 20.04 users will do something similar on their own systems ?

Geet (16:55:31): > Hi, What is an efficient way to increase the container size? I tried to improve it using docker desktop and using the –memory option in build command, but nothing worked so far. Thanks.

Hervé Pagès (18:09:23) (in thread): > A fix at the system level requires sudo access which makes it a no-go for some users. Hopefully Mike’s (@Mike Smith) suggestion works for them.

2020-10-02

Sean Davis (09:24:42) (in thread): > The amount of memory given to a container is not something that you change at “build time.” Instead, the amount of memory is controlled by the run environment. On a laptop/desktop, you can control that in the docker desktop settings. If you are running your docker container in another environment (HPC, cloud), there will be different ways to set the possible memory usage. > > Could you be more specific about what you mean when you say “nothing worked so far”? What problem are you seeing?

Nitesh Turaga (09:26:40) (in thread): > and also be more specific about “container size” maybe..

Nitesh Turaga (09:27:16) (in thread): > My first reaction when I saw “container size” was the size of the “image”….but I don’t think you want to “increase” this….

2020-10-04

Geet (13:07:31) (in thread): > Thanks@Sean Davisand@Nitesh Turaga. I am still getting the error of “no space left on device” even I updated the memory in the docker desktop.

Sean Davis (14:23:53) (in thread): > Seehttps://net2.com/how-to-fix-docker-error-no-space-left-on-device/ - Attachment (net2): How to fix Docker error: no space left on device > You will find below how to solve the Docker error : no space left on device . There are two similar solutions. The first is the long winded whereas the

Sean Davis (14:24:20) (in thread): > The error is due to the docker file system being full and doesn’t have anything to do with the container itself.

2020-10-23

Jianhai Zhang (12:55:39): > @Jianhai Zhang has joined the channel

Jianhai Zhang (12:56:07): > Is the Bioconductor Docker image only available for linux builder, not windows?

Martin Morgan (13:03:40): > only linuxhttp://bioconductor.org/help/docker/; also the primary goal of the image is to enable easy use of bioc packages, rather than to closely emulate the build system.

2020-11-05

Leonardo Collado Torres (13:03:32): > Hi! Is this the best channel to talk abouthttps://github.com/Bioconductor/bioconductor.org/commit/9c7cabcb00170eb9d24a9597b6dd4c5ede588268? I was just made aware of this change, which would affect packages that use a GitHub Actions workflow, such as those made bybiocthis

Nitesh Turaga (13:04:13): > hmm…doesn’t seem like it…

Nitesh Turaga (13:04:28): > what am I missing, this is the containers channel ….

Nitesh Turaga (13:04:53): > your question seems to be able the branches on git

Leonardo Collado Torres (13:07:54): > ahh,#bioc_gitthen, right?

Nitesh Turaga (13:10:12): > Probably

Leonardo Collado Torres (13:10:20): > kk

2020-11-11

watanabe_st (05:08:02): > @watanabe_st has joined the channel

2020-11-30

Roy Storey (04:55:16): > @Roy Storey has joined the channel

2020-12-02

Konstantinos Geles (Constantinos Yeles) (05:42:38): > @Konstantinos Geles (Constantinos Yeles) has joined the channel

2020-12-12

Huipeng Li (00:40:35): > @Huipeng Li has joined the channel

Sean Davis (03:35:47): > See this thread. If this turns out to be a reproducible problem, is there any way that Bioconductor Docker images could undergo regular testing to help catch issues before they make it into live containers? - Attachment: Attachment > Hi again, if I just want to use the latest release, is it enough to change the docker image (container:) to bioconductor/bioconductor_docker:RELEASE_3_12 in the workflow file?

Martin Morgan (06:18:40): > We’re working on regular testing, but is that the right solution here? For instance we were thinking of weekly testing when the image is automatically rebuilt, but this problem was introduced some time on Thursday and fixed on Friday. The docker image wasn’t built broken, but built last Friday and became broken because of a change in CRAN. This problem is different from the Bioc2020 problem, where igraph failed to install because of a change in the underlying rocker image – probably testing would have caught that. > > As we learn from experience, it really seems like workshops should adopt a last-known-good approach to the docker image and package installation procedures, rather than ‘whatever is currently available’. This is facilitated by the RStudio Package Manager, including now for Bioconductor packages (I think – I’m not sure how devel works on the RSPM).

Martin Morgan (06:26:17): > I’ll mention that this also has consequence for other uses of the container, e.g., in the AnVIL where the jupyter notebooks and RStudio release deployments are now out-of-date with respect to installing usethis. > > It seems like usethis would be a developer tool and should be in theSuggests:field, but I see 170 packages with it as required dependency. I see the following Bioconductor packages with usethis as required dependency > > [1] "biocthis" "BiocWorkflowTools" > [3] "BUMHMM" "cogena" > [5] "dasper" "GCSscore" > [7] "INDEED" "megadepth" > [9] "MetID" "oppti" > [11] "PathoStat" "Rmmquant" > [13] "srnadiff" "wpm" > [15] "BloodCancerMultiOmics2017" "spatialLIBD" > [17] "RNAseq123" "rnaseqDTU" >

Sean Davis (12:38:56): > Kudos to@Nitesh Turagaand the Bioconductor core team for so quickly generating fixes! This rapid response is a primary reason for the success of the docker images, in my opinion. > I totally agree that folks using tagged versions of docker containers should be aware of interdependencies that the changing ecosystem expose. > I’d definitely be interested in learning about the “last known working version” approach.

Sean Davis (12:44:42): > It sounds like I should be developing a more controlled approach to building workshops than we have been using with GHActions.

Martin Morgan (13:43:56) (in thread): > I was thinking ofhttps://community-bioc.slack.com/archives/CJDMYKG2U/p1607714996015600as an implementation of last known good – a derived docker container that fixes the CRAN (and maybe Bioconductor) snapshot date, and only increments when the derived container knows (by its own criteria…) that it is OK to do so. > > In terms of testing the images that the core team produces, it seems like a straight-forward best practice would be to test-load (or better build and check) a subset of packages in the current weekly build GitHub action athttps://github.com/Bioconductor/bioconductor_docker/blob/master/.github/workflows/weekly-devel-builder.yml(what packages? GenomicRanges, devtools, dplyr, to get some basic coverage of core and common [including implied dependencies, like Rcpp], plus packages that have caused problems in the past and not included in this dependency graph (igraph?). > > A second workflow might run a once-daily script to just test these packages on release and devel images. > > This does not seem particularly hard to implement (he says, with limited GitHub action experience) and a pull request would be a great starting point - Attachment: Attachment > This is because of a change in the dependency graph that introduces the package ‘gert’ into the usethis dependency graph. It’s a recent change (published 2020-12-10). > > To avoid this problem in the future, maybe the workshop should have a custom image and peg to a particular date via the RStudio package manager CRAN mirror, options(repos = c(CRAN = "[https://packagemanager.rstudio.com/all/161958](https://packagemanager.rstudio.com/all/161958)")) (for 2020-12-09 snapshot…). > > From the command line with the current container > $ docker run -it --rm bioconductor/bioconductor_docker:devel R > > BiocManager::install("usethis") #fails > ... > > options(repos= c(CRAN="[https://packagemanager.rstudio.com/all/161958](https://packagemanager.rstudio.com/all/161958)")) > > BiocManager::install("usethis") # succeeds > This URL is from the <https://packagemanager.rstudio.com/client/#/repos/1/overview|RSPM setup page> fro Ubuntu 20.04.

Sean Davis (16:05:15) (in thread): > @Nitesh Turagahttps://github.com/Bioconductor/bioconductor_docker/pull/20

Sean Davis (16:15:32): > Here is a PR for matrix testing “canary” package installs of dockerdevelandRELEASEtags run every 6 hours using the “current” docker images. Using the “matrix” approach can enable further “variables” such as base repositories (CRAN, RSPM, etc.) and a selection of packages to be tested independently. - Attachment: Attachment > @Nitesh Turaga > https://github.com/Bioconductor/bioconductor_docker/pull/20

2020-12-13

Federico Agostinis (14:53:44): > @Federico Agostinis has joined the channel

2020-12-14

Thomas Naake (08:55:44): > @Thomas Naake has joined the channel

2020-12-15

Francesc Català (05:41:33): > @Francesc Català has joined the channel

Martin Morgan (12:34:13): > Not sure that this is the right channel:wink:@Sean Davisis there a ‘really good’ workshop from a teaching perspective? Something that can be launched athttp://app.orchestra.cancerdatasci.org/and will immediately present the user with whatever is needed? For instance, if I choose theSPEAQeasy workshop, I end up with an RStudio session, but I need to know that I should look in vignettes and then find an Rmd file, and then ignore a lot of material about git and so on….

Leonardo Collado Torres (13:05:46) (in thread): > err, I was just working on that workshop:stuck_out_tongue:It wasn’t complete when you looked at it

Leonardo Collado Torres (13:06:10) (in thread): > but yes, we include some info about how to find the information (since it’s not evident where to look at)

Kevin Rue-Albrecht (13:12:08) (in thread): > Very good point Martin. > I also noticed that blank workspace effect. > I think it’s possible to preconfigure the image so that RStudio opens certain files on startup, e.g. the vignette, but I didn’t try to implement this yet.

Sean Davis (16:18:27) (in thread): > The standalone workshop concept (not guided) still needs some development. There are no particular rules that we need to follow. I’ve started playing with hosting a learnr-based approach that presents a set of learnr tutorials running on a shiny server. Jupyter notebooks are another possibility. For a more traditional approach, using a custom R message in the console can go a long way.

Sean Davis (16:20:48) (in thread): > Essentially, whatever runs in a container and exposes a web interface is fair game.

2020-12-16

Sean Davis (11:31:11) (in thread): > Here is an ipython-based starter: - File (PNG): image.png

Sean Davis (11:33:20) (in thread): - File (PNG): image.png

Sean Davis (11:38:19) (in thread): > And the example Dockerfile:FROM bioconductor/bioconductor_docker:devel``RUN apt-get update && apt-get install -y python3 python3-dev python3-pip``RUN pip3 install jupyter -U && pip3 install jupyterlab``ENV CRAN=https://cran.rstudio.comRUN Rscript -e 'options("repos"=c(CRAN="https://cran.rstudio.com"));install.packages("IRkernel")'``RUN Rscript -e 'options("repos"=c(CRAN="https://cran.rstudio.com"));IRkernel::installspec(user=FALSE)'``RUN Rscript -e 'BiocManager::install(c("GEOquery", "maftools", "GenomicDataCommons"))'``RUN Rscript -e 'BiocManager::install(c("ggplot2"))'``EXPOSE 8888``WORKDIR /home/rstudio``COPY . .``ENTRYPOINT ["jupyter", "lab","--ip=0.0.0.0","--allow-root","--NotebookApp.token=''"]

Martin Morgan (11:56:57) (in thread): > Thanks, its useful to think a little outside (my) box; the jupyter notebooks are quite convenient as a learning on-ramp. FWIWAnVILPublish::as_notebook()tries to create a ipynb from Rmd; it has pre-requisites outlined in the vignette, doesn’t handle figures, etc… but might be a start.

Sean Davis (12:37:30) (in thread): > Nice! I’ll give it a try on one of the simpler workshops.

Sean Davis (12:38:39) (in thread): > It would be great to compare notes. The technical parts seem to be roughly working, so it frees us up to think about the actual educational materials and how to make them more accessible.

2020-12-17

Manojkumar Selvaraju (11:11:05): > @Manojkumar Selvaraju has joined the channel

2021-01-02

Vince Carey (09:17:08): > Adverse interaction in container context in VPN. If I am using my school’s VPN and I try to launch R from the bioconductor_docker:devel container, I have a long delay and then > > During startup - Warning messages: > 1: In file(con, "r") : > URL '[https://bioconductor.org/config.yaml](https://bioconductor.org/config.yaml)': status was 'Couldn't resolve host name' > 2: In file(con, "r") : > URL '[http://bioconductor.org/config.yaml](http://bioconductor.org/config.yaml)': status was 'Couldn't resolve host name' >

Vince Carey (09:23:19): > Why this happens from the docker container is the curiosity.

Dirk Eddelbuettel (09:26:53) (in thread): > I wouldsuspectbut cannot prove that this may be between your docker host and your setup. Someone is unkind and does not forward DNS requests. In a much more minimal setup (that is Debian-based) it works just fine:

Dirk Eddelbuettel (09:27:09) (in thread): - File (Plain Text): Untitled

2021-01-04

Fred Boehm (15:11:28): > @Fred Boehm has joined the channel

2021-01-19

Pablo Rodriguez (04:56:05): > @Pablo Rodriguez has joined the channel

2021-01-22

Annajiat Alim Rasel (15:43:25): > @Annajiat Alim Rasel has joined the channel

2021-01-29

Dario Righelli (03:50:04): > @Dario Righelli has joined the channel

2021-04-05

Maya Reed McDaniel (11:22:59): > @Maya Reed McDaniel has joined the channel

2021-04-12

Naohisa Goto (00:29:07): > @Naohisa Goto has joined the channel

2021-05-12

Megha Lal (09:59:57): > @Megha Lal has joined the channel

2021-05-20

Aedin Culhane (17:41:08) (in thread): > Are you using a chromebook?

2021-05-21

Pablo Rodriguez (06:53:43): > Is there any estimated date for the updated Bioconductor docker images?

Lori Shepherd (07:23:29): > @Nitesh Turagawill begin working on the updates and should have them soon.

Nitesh Turaga (13:24:25): > I’ll be working on updates today, and sending out an email as soon as they are ready

Alan O’C (13:36:05): > @Alan O’C has left the channel

2021-05-26

Shraddha Pai (11:27:40): > Hi all, > My Rstudio docker instance is giving me apermission deniederror when I try to write anywhere except tempdir(). My docker container is run asrootbut in rstudio my user name isrstudio. > > Any advice on how to fix this issue? Thank you.

Dirk Eddelbuettel (12:16:12) (in thread): > Are you trying to write to the host file system? Maybe you need to create a (scratch ?) directory with appropriate permissions for the (RStudio inside the containre) user to write to./tmpis generally open for all which is why you can write there.

2021-05-27

Shraddha Pai (09:22:54) (in thread): > Great, thank you@Dirk Eddelbuettel, changing the write permission within the container to allowrstudioto write fixed it.

2021-06-08

Lorenzo Bonaguro (08:50:39): > @Lorenzo Bonaguro has joined the channel

2021-07-19

Leo Lahti (17:01:07): > @Leo Lahti has joined the channel

2021-07-23

Batool Almarzouq (15:52:28): > @Batool Almarzouq has joined the channel

2021-08-04

Rene Welch (12:16:55): > @Rene Welch has joined the channel

2021-08-14

Sean Davis (12:20:14): - Attachment: Attachment > I’ve been using bioconductor/bioconductor_docker:devel, and its Dockerfile doesn’t seem to have qpdf. Adding it as suggested, helped (thanks, @Dirk Eddelbuettel). I’m not sure if I should use bioconductor/bioconductor_docker:devel or nturaga/bioconductor_build_docker:master

2021-08-17

Nicholas Cooley (08:41:31): > Not sure if this question belongs here or in random or general; but does anyone use the NCBI EDirect command line tools inside a container? Docker thinks they install just fine, but only a few of the executables seem to set up correctly.

Nitesh Turaga (10:14:29): > Not sure about NCBI EDirect, but are the executables in your$PATH?

Nitesh Turaga (10:14:41): > Also, this may or may not be a container issue.

Nicholas Cooley (11:43:33): > The executables are in the path, itseemslike a container issue, but it’s difficult to figure out: A really simple Dockerfile example looks like: > > FROM r-base:4.1.0 > > # 'docker build --no-cache -t npcooley/test2:latest .' > # 'docker run -i -t --rm npcooley/test2:latest sh' > > # Dependencies > RUN apt-get update && \ > apt-get -y install build-essential && \ > apt-get -y install libgmp-dev && \ > apt-get -y install libcurl4-openssl-dev && \ > apt-get -y install libssl-dev && \ > apt-get -y install openmpi-common && \ > apt-get -y install curl > > # some generic R stuff > RUN install.r remotes \ > httr > > # install edirect: NCBI instructions > RUN wget[ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/install-edirect.sh](ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/install-edirect.sh)&& \ > . ./install-edirect.sh > > # it's not clear to me why this lands in 'root' > ENV PATH=$PATH:/root/edirect > > spinning up the container and asking if esearch exists gives a normal response: > > # esearch -h | head -n 5 > esearch 15.5 > > Query Specification > > -db Database name > > but asking for anything else throws a complaint: > > # xtract -h > > Unable to locate xtract executable. Please execute the following: > > nquire -dwn ftp.ncbi.nlm.nih.gov entrez/entrezdirect xtract.Linux.gz > gunzip -f xtract.Linux.gz > chmod +x xtract.Linux > > calling edirect’s setup.sh seems to help interactively, but that’s not what i need, and I can’t seem to include that in my Dockerfile and it to be successful.

Nitesh Turaga (11:45:42): > How are you spinning up the container ?

Nicholas Cooley (11:52:26): > just withdocker run -i -t --rm npcooley/test2:latest sh

Nitesh Turaga (13:13:58): > Yeah, i’m not sure actually. > > You are installing everything in the root user, so there shouldn’t be any issues there.

Nicholas Cooley (13:27:02): > Yeah, the real puzzle is that running setup interactively gets them running correctly, but trying to include setup in the dockerfile doesn’t.

Nicholas Cooley (13:32:38): > i.e. > > Nicholass-MBP:Test2 nicholascooley$ docker run -i -t --rm npcooley/test2:latest sh > # esearch -db assembly -query 'kitasatospora[organism] AND "complete genome"[filter] AND "refseq has annotation"[properties] AND "latest refseq"[filter] NOT anomalous[filter]' | esummary | xtract -pattern DocumentSummary -element FtpPath_RefSeq > > Unable to locate xtract executable. Please execute the following: > > nquire -dwn ftp.ncbi.nlm.nih.gov entrez/entrezdirect xtract.Linux.gz > gunzip -f xtract.Linux.gz > chmod +x xtract.Linux > > running setup in the container though: > > Nicholass-MBP:Test2 nicholascooley$ docker run -i -t --rm npcooley/test2:latest sh > # /root/edirect/setup.sh > > Entrez Direct has been successfully downloaded and installed. > > In order to complete the configuration process, please execute the following: > > echo "export PATH=\${PATH}:/root/edirect" >> $HOME/.bashrc > > or manually edit the PATH variable assignment in your .bashrc file. > > Would you like to do that automatically now? [y/N] > n > Holding off, then. > # esearch -db assembly -query 'kitasatospora[organism] AND "complete genome"[filter] AND "refseq has annotation"[properties] AND "latest refseq"[filter] NOT anomalous[filter]' | esummary | xtract -pattern DocumentSummary -element FtpPath_RefSeq[ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/013/364/235/GCF_013364235.1_ASM1336423v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/013/364/235/GCF_013364235.1_ASM1336423v1)[ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/943/525/GCF_002943525.1_ASM294352v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/943/525/GCF_002943525.1_ASM294352v1)[ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/082/605/GCF_002082605.1_ASM208260v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/082/605/GCF_002082605.1_ASM208260v1)[ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/082/585/GCF_002082585.1_ASM208258v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/082/585/GCF_002082585.1_ASM208258v1)[ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/269/985/GCF_000269985.1_ASM26998v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/269/985/GCF_000269985.1_ASM26998v1) > > seems to run just fine

Nicholas Cooley (13:35:17): > andingRUN /root/edirect/setup.shto the dockerfile doesn’t fix anything

Nicholas Cooley (14:02:44): > Alright, sorry for all the spam, I’ve gotten it working. The following dockerfile seems to run my normal edirect queries without complaint: > > FROM r-base:4.1.0 > > # 'docker build --no-cache -t npcooley/test2:latest .' > # 'docker run -i -t --rm npcooley/test2:latest' > > # Dependencies > RUN apt-get update && \ > apt-get -y install build-essential && \ > apt-get -y install libgmp-dev && \ > apt-get -y install libcurl4-openssl-dev && \ > apt-get -y install libssl-dev && \ > apt-get -y install openmpi-common && \ > apt-get -y install curl > > # some generic R stuff > RUN install.r remotes \ > httr > > # install edirect: NCBI instructions > RUN wget[ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/install-edirect.sh](ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/install-edirect.sh)&& \ > . ./install-edirect.sh > > ENV PATH=$PATH:/root/edirect > > # not clear why these run incorrectly in setup > RUN wget ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/xtract.Linux.gz && \ > gunzip -f xtract.Linux.gz && \ > chmod +x xtract.Linux && \ > mv xtract.Linux /root/edirect/ > > RUN wget ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/transmute.Linux.gz && \ > gunzip -f transmute.Linux.gz && \ > chmod +x transmute.Linux && \ > mv transmute.Linux /root/edirect/ > > RUN wget ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/rchive.Linux.gz && \ > gunzip -f rchive.Linux.gz && \ > chmod +x rchive.Linux && \ > mv rchive.Linux /root/edirect/ > > Thewgetstatements just mimic somenquirestatements in edirect’s setup script that just don’t seem to run correctly during build.

2021-08-18

Liz Ing-Simmons (04:24:43): > @Liz Ing-Simmons has joined the channel

Liz Ing-Simmons (04:36:40): > I’m not sure if this is an appropriate place to ask this, happy to be directed somewhere else! > > I’m using the Bioconductor release docker container with Singularity in order to run RStudio Server on my lab’s computing cluster (as users don’t have root access, we have to run Singularity in user namespace mode). It works well, except that often when reconnecting to an RStudio Server session I’ll get errors like this: > > 18 Aug 2021 08:19:22 [rserver] ERROR system error 2 (No such file or directory) [path: /proc/277799/fd]; OCCURRED AT rstudio::core::Error rstudio::core::system::{anonymous}::scanDir(const string&, std::vector<std::__cxx11::basic_string<char> >**) src/cpp/core/system/PosixFileScanner.cpp:74; LOGGED FROM: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType**) src/cpp/core/system/PosixSystem.cpp:2109 > > And, indeed, the times that I’ve managed to start up the RStudio Server session anyway and check, that file is missing and so is the whole/proc/directory in the container’s filesystem. It’s impossible to actually use the container/session (just get the same error repeatedly when doing anything) after this happens so I have to shut it down and restart. > > I don’t think this is an issue with the Bioc docker container itself, but I can’t tell if this is an underlying issue with RStudio Server, Singularity, how Singularity is set up on our system, or something else, and I don’t really know enough about any of the above to know where to start trying to debug this. Has anyone else had this issue or has any suggestions?

Spencer Nystrom (07:59:23) (in thread): > Hey Liz, I think this GitHub issue will be helpful to you.https://github.com/rocker-org/rocker-versioned/issues/213#issuecomment-730503117

Spencer Nystrom (08:00:33) (in thread): > Near the bottom we worked out a set of additional flags you need to pass to get the latest rstudio to boot under singularity.

Liz Ing-Simmons (08:38:41) (in thread): > Hey, thanks for the suggestion! However I’m not sure that it’s the same issue, because I can get RStudio server to boot, and can use it - it’s only if I leave Singularity running and try to then re-login after some time (e.g. the next day) that I get this kind of error. > > The version of RStudio server in the image I’m using is 1.4.1717. I am already binding my own/tmpand/rundirectories and setting--server-data-dir, which were the suggestions I saw in that issue. Here’s the full command I use for completeness (this was based on recommendations from a colleague who runs RStudio Server on a different compute cluster at the same institute):SINGTMP="/tmp/${USER}_rstudio_tmp";rm -rf $SINGTMP; mkdir -p $SINGTMP/tmp;mkdir -p $SINGTMP/run;mkdir -p $SINGTMP/lib;PASSWORD='password' /bin/singularity exec --bind $SINGTMP/tmp:/tmp --bind $SINGTMP/run:/run --bind $SINGTMP/lib:/var/lib/rstudio-server --bind /home/research/ --userns /home/research/vaquerizas/singularity/bioconductor_docker_RELEASE_3_13.sif rserver --auth-none=0 --auth-pam-helper-path=pam-helper --www-port 8600 --server-data-dir /tmp

Spencer Nystrom (08:41:29) (in thread): > Try setting--auth-timeout-minutes=0 --auth-stay-signed-in-days=30That should give you 30 days between timeouts. Which obviously doesn’t solve your issue, but may be an effective workaround.

Liz Ing-Simmons (08:51:49) (in thread): > Okay, I’ll give that a go. I can typically log in again fine after a timeout as long as it’s on the same day I started the session. So my thinking was that it’s to do with the length of time rather than the timeout itself. But maybe it’s the combination. > > If this works it would definitely be an effective workaround for us. Might take a while to tell, since this issue is a bit unpredictable:crossed_fingers:

2021-08-19

Liz Ing-Simmons (02:35:58) (in thread): > Unfortunately this didn’t help. > > Opening my browser this morning, in the RStudio console I get the same error as before, repeated with timestamps roughly every ten seconds for half an hour yesterday evening, then stopping: > > 18 Aug 2021 18:29:50 [rsession-lingsim] ERROR system error 2 (No such file or directory) [path: /proc/meminfo]; OCCURRED AT rstudio::core::Error rstudio::core::FilePath::openForRead(std::shared_ptr<std::basic_istream<char> >&) const src/cpp/shared_core/FilePath.cpp:1427; LOGGED FROM: void rstudio::session::modules::system_resources::{anonymous}::emitMemoryChangedEvent() src/cpp/session/modules/SessionSystemResources.cpp:63 > > I can’t do anything in the RStudio Server session, I just get the messageError: Unable to connect to service. > > I also have errors in the login session on the cluster where I’m running Singularity, which doesn’t always happen: > > 18 Aug 2021 16:08:29 [rsession-lingsim] ERROR system error 37 (No locks available) [lock-file: /home/lingsim/.local/share/rstudio/sources/s-210d75ec/lock_file]; OCCURRED AT virtual rstudio::core::Error rstudio::core::AdvisoryFileLock::acquire(const rstudio::core::FilePath&) src/cpp/core/file_lock/AdvisoryFileLock.cpp:129; LOGGED FROM: rstudio::core::Error rstudio::session::source_database::supervisor::attachToSourceDatabase() src/cpp/session/SessionSourceDatabaseSupervisor.cpp:424 > 18 Aug 2021 16:08:29 [rsession-lingsim] ERROR system error 37 (No locks available) [lock-file: /home/lingsim/.local/share/rstudio/sources/s-210d75ec/lock_file]; OCCURRED AT virtual rstudio::core::Error rstudio::core::AdvisoryFileLock::acquire(const rstudio::core::FilePath&) src/cpp/core/file_lock/AdvisoryFileLock.cpp:129; LOGGED FROM: rstudio::core::Error rstudio::session::source_database::supervisor::{anonymous}::createSessionDir() src/cpp/session/SessionSourceDatabaseSupervisor.cpp:249 > 19 Aug 2021 06:13:03 [rserver] ERROR system error 2 (No such file or directory) [path: /proc/270482/fd]; OCCURRED AT rstudio::core::Error rstudio::core::system::{anonymous}::scanDir(const string&, std::vector<std::__cxx11::basic_string<char> >**) src/cpp/core/system/PosixFileScanner.cpp:74; LOGGED FROM: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType**) src/cpp/core/system/PosixSystem.cpp:2109 > > The last one is from this morning and is triggered every time I try to do anything in the Rstudio session that is still open in my browser. (The time is off by 2 hours on the errors in the login session, btw, so theNo locks availableprobably happened shortly before the errors in the Rstudio console). > > Does anyone have any idea if this is likely to be an issue with RStudio Server, Singularity, the intersection of the two, or something else?

Spencer Nystrom (09:46:24) (in thread): > This almost looks like a server issue where files a disappearing from the filesystem (ie unrelated to the container or singularity)? Is the tempdir getting cleared while the container is still running?

Liz Ing-Simmons (10:08:29) (in thread): > Hmm, could be. One of the times I was able to do stuff in the session after getting this kind of error I was able to verify that the entire/proc/directory was missing, but there were other files/directories present under/- I don’t remember exactly what any more. > > As far as I know there’s not any automatic deletion of files in/tmp/, and the disk isn’t anywhere near full, but I will check with our sysadmin in case there’s something set up that I’m not aware of. > > While checking that, I did notice that as well as the designated/tmp/lingsim_rstudio_tmp/, there’s also a/tmp/rootfs-048868045/directory which was created at the same time and seems to contain files relating to the container: inside/tmp/rootfs-048868045/root/there’sbin/,rocker_scripts/,var/,proc/, etc. This seems to map to/inside the container, but weirdly not all of the files have the same last modification times or owner if I look at this from inside the container.

Liz Ing-Simmons (10:25:17) (in thread): > I’ve marked the files with different owners in blue and different modification times in red, for reference. The first screenshot is from inside the container, the second from the compute node - File (PNG): Screenshot 2021-08-19 at 16.14.49.png - File (PNG): Screenshot 2021-08-19 at 16.20.01.png

Spencer Nystrom (10:34:39) (in thread): > Weird… This does seem to be on the right track, though… A bit over my head at this point, but maybe double check that you’ve set all the run flags in your singularity call to include some newly suggested ones from here:https://www.rocker-project.org/use/singularity/. It could be that some of those files aren’t getting mounted appropriately? Kinda grasping at straws. I’ve been running this container under singularity for months without issue, but I only ever run ~10 hour jobs at a time with pretty much continuous usage & spin the image up daily, so maybe I’ve just got lucky.

Liz Ing-Simmons (10:44:00) (in thread): > Yeah… I’m at a loss as well! > > We’ve not been bindingdatabase.confso I’ll definitely give that a try and double-check that the others are as advised by the rocker project. > > My colleague also runs this container without this issue coming up as often as it does for me, although he’s running on another cluster and probably has a different version of singularity. I thought he’d never seen it but he just told me he also gets it occasionally! It could well be a quirk of how our sysadmin has set up these systems that doesn’t play well with singularity. In the end spinning up a new session every day isn’t too annoying, I guess.

Liz Ing-Simmons (10:44:11) (in thread): > Thanks for the help!

Spencer Nystrom (10:46:44) (in thread): > Yeah since I switched to usingtargetsspinning up a new session is much less painful since things are cached… But still, very annoying vs long running forever session. > > Good luck, sorry I couldn’t solve it.:man-shrugging:

Liz Ing-Simmons (10:47:59) (in thread): > I should look intotargetsthen, that was on my to do list already :)

Spencer Nystrom (10:51:27) (in thread): > I’ve been working on the most unreliable cluster for the last couple months andtargets+renvhas saved me an immense amount of pain, really can’t suggest them more highly. And it’s so easy to set up! I do my interactive dev in a notebook, then once I’ve settled on what I want the r object to be, I just copy that line into a targets rule, delete it from the notebook, &tar_loadthe object into the environment.:exploding_head:

2021-09-09

Julien Roux (01:57:31): > @Julien Roux has joined the channel

2021-09-16

Henry Miller (18:35:11): > @Henry Miller has joined the channel

2021-10-14

Andres Wokaty (13:25:42): > @Andres Wokaty has joined the channel

2021-10-18

John Kerl (09:15:47): > @John Kerl has joined the channel

2021-10-31

Marcel Ramos Pérez (01:21:36): > @Marcel Ramos Pérez has joined the channel

Kozo Nishida (02:04:23): - Attachment: Attachment > Hi, When do you think we will be able to pull bioconductor_docker:RELEASE_3_14? > PS C:\Users\kozo2> docker pull bioconductor/bioconductor_docker:RELEASE_3_14 > Error response from daemon: manifest for bioconductor/bioconductor_docker:RELEASE_3_14 not found: manifest unknown: manifest unknown > (I asked this because it is related to the update of the BioC Asia workshop materials.)

Nitesh Turaga (13:45:14): > @Kozo NishidaNow..

Nitesh Turaga (13:45:17): > it’s available

Kozo Nishida (20:53:48): > Thanks@Nitesh Turaga. > Now I can pullbioconductor/bioconductor_docker:RELEASE_3_14

2021-11-03

Alan Murphy (03:39:51): > @Alan Murphy has joined the channel

2021-11-23

Federico Marini (07:54:28): > @Federico Marini has joined the channel

2021-11-26

Francesc Català (06:43:36): > @Francesc Català has left the channel

2021-12-07

Jenny Smith (15:20:37): > @Jenny Smith has joined the channel

2021-12-14

Megha Lal (08:24:54): > @Megha Lal has left the channel

2022-01-10

Ruth Isserlin (10:57:00): > Has anyone been able to use the bioconductory/bioconductory_docker:RELEASE_3_14 on the new M1 arm64 ? I tried building my image with –platform linux/arm64 but still having issues getting the container to run.

Nitesh Turaga (10:57:45): > Hi@Ruth Isserlin, Unfortunately i’ve not tried it since i don’t have access to an M1 arm64 mac.

Nitesh Turaga (10:58:16): > Is it possible for me to test this somewhere online?

Ruth Isserlin (10:58:21): > I don’t either but a few of my students have them and I am just trying to help them debug it.

Nitesh Turaga (10:58:31): > I see.

Ruth Isserlin (11:00:59): > I am wonding if using an earlier release of bioc docker would help because it seems like Rstudio is the problem with regards to the arm64 compatability

Kasper D. Hansen (11:17:23): > Is this necessary? It may very well be with docker, but in general, R supposedly works well under Rosetta and the goal of building it under arm64 is just to get even more speed

Nitesh Turaga (11:18:12): > I’m not sure at all about this, sorry. > > Maybe@Vince Careyor@Martin Morgancan chip in with some advice?

Ruth Isserlin (11:29:28): > When they try and run the docker compiled for amd64 rstudio won’t launch. I thought that the whole point of docker was to avoid these kinds of issues. So I was very surprised.https://gitmemory.cn/repo/rocker-org/rocker-versioned2/issues/287 - Attachment (gitmemory.cn): Operation Not Permitted Error from Rocker RStudio Image on M1 MacBook - gitmemory > Operation Not Permitted Error from Rocker RStudio Image on M1 MacBook

Ruth Isserlin (11:30:08): > but after compiling with platform tag specific to arm64 it still doesn’t work.

Dirk Eddelbuettel (11:34:40): > I will say that I do not fully understand the if/but/when of this (as I don’t have a mac, either i7 or arm) but we created an Rocker RStudioderivedcontainer for my course (for use in the wicked test / grading system developed at U of I, and which is in use on a few other campuses) and that image somehow runs on my TAs macBook with an M1 chip. The hard constraintmaybe related just to the login or auth mechanism.

Vince Carey (12:00:58): > I have an m1. I am a bit concerned about the spelling inbioconductory/bioconductory_docker:RELEASE_3_14is this a typo or is there a repo called bioconductory?

Vince Carey (12:01:42): > I will test with the correct spelling and get back to you.

Ruth Isserlin (12:24:43): > that was a typo, sorry.

Ruth Isserlin (12:24:56): > I didn’t cut and copy. I just free typed it. sorry.

Ruth Isserlin (12:26:29): > @Dirk Eddelbuettelis the image based on RELEASE_3_14? I am wondering if I won’t have this issue if I go back and use an older release? It looks like it might be a later R version issue as some of the previous R versions are already arm64 compatible.

Vince Carey (12:27:34): > > Status: Downloaded newer image for bioconductor/bioconductor_docker:RELEASE_3_14 > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested > > Command line R works fine.

Ruth Isserlin (12:29:01): > I think it is just an issue with Rstudio access.

Vince Carey (12:30:27): > So is the issue that you get a login splash page with Rstudio but can’t log in? I am currently encountering that – it will not accept the assigned password.

Ruth Isserlin (12:32:40): > Yes. It says the operation is not permitted.

Ruth Isserlin (12:33:01): > [rserver] ERROR system error 1 (Operation not permitted); OCCURRED AT rstudio::core::Error rstudio::core::system::posix::{anonymous}::restorePrivilegesImpl(uid_t) src/cpp/shared_core/system/PosixSystem.cpp:66; LOGGED FROM: void rstudio::server::pam_auth::{anonymous}::assumeRootPriv() src/cpp/server/ServerPAMAuth.cpp:57

Vince Carey (12:36:10): > Those messages are from the docker run process? I do not see them, I am simply rejected at the splash page.

Dirk Eddelbuettel (12:36:40) (in thread): > I never said it had any BioConductor. It is an R image with RStudio, and it provides an existence proof of RStudio via Docker on M1.

Ruth Isserlin (12:39:22): > unfortunately they aren’t my error message but I believe they are coming from the command line where the docker container was launched from.

Ruth Isserlin (12:41:20): - File (PNG): image001.png

Ruth Isserlin (12:43:09) (in thread): > Which tagged version of R (rocker) was it?

Dirk Eddelbuettel (12:46:57) (in thread): > Good question. Whatever was current when we needed it circa late August. The container is public: prairielearn/workspace-rstudio. When Iexeca command such asRscript --versionI get 4.1.1 back which seems right.

Vince Carey (12:50:49): > I am not sure why I am not seeing those messages too. In any event our assumptions about docker on M1 seem incorrect. Perhaps if an intel-mac version of docker desktop were invoked under a rosetta emulating terminal things would turn out differently.

Vince Carey (12:51:24): > It seems clear there is no direct support for M1 mac for Rstudio server at this time.

Ruth Isserlin (12:59:43) (in thread): > perfect. I will give that version a try and see if it fixes the problem. Thanks.

Dirk Eddelbuettel (13:01:09) (in thread): > It is a bit of a special case build in that it turns of password and what not – we run it in labs on short lived machines.Butif it shows that RStudio can in fact run on M1 it may help some progress there…

Dirk Eddelbuettel (13:02:55) (in thread): > See my thread to Ruth above. We appear to have an existence in that my TA runs our course-container with RStudio on his M1 box. That’s all I know. I had the fellow who helped us build it report back to the Rocker repo thread too. YMMV.

Liz Ing-Simmons (13:39:32): > FWIW I can reproduce the “Operation not permitted” error on my M1 mac with'bioconductor/bioconductor_docker:RELEASE_3_14'. RStudio Server accepts the password but hangs and then gives an error message. > > Weirdly enough withbioconductor/bioconductor_docker:develI don’t get any error in the terminal, but I get the same error message in the browser in both cases: > > Cannot connect to R Session > Could not connect to the R session on RStudio Server. > Unable to connect to service (1) >

Vince Carey (13:41:10) (in thread): > OK – the link to #144 was broken in the originalgitmemory.cnlink – here is the thread of interesthttps://github.com/rocker-org/rocker-versioned2/issues/144

Vince Carey (13:51:44): > Here’s my 2c on this problem. 1) Rstudio seems to have a path towards direct support in the next couple of months. 2) Dirk has given a path towards a workable instance of Rstudio in docker for M1 in which you may have to install BiocManager and additional Bioc packages. I think our core dev team is maxxed out and we would like to have community input and data on the viability of use of the container defined athttps://github.com/PrairieLearn/PrairieLearn/tree/master/workspaces/rstudio… if this works out we may be able to put some effort into making an M1-capable container for general use in the next month or two.

Nitesh Turaga (13:55:55): > Thank you for your inputs@Vince Careyand@Dirk Eddelbuettel. I appreciate it.

Nitesh Turaga (14:06:15): > @Ruth IsserlinIf it’s for the sake of 1 student, is it possible to provide a deployed docker instance to that student?

Ruth Isserlin (14:08:31): > I am not sure how many students have the issue. So far two have reported the issue. But yes, I can rebuild the container for just them. I am currently building the container using Bioconductor 3_13 which uses R 4.1 with the platform arm64 tag to see if that helps. But they can always use R from the command line as it sounds like that works.

Vince Carey (15:08:57): > I have to confess that I am not sure how graphics will work from the command line. I think it can be done with proper display variable handling but I do not know the details at this time.

2022-01-11

Dirk Eddelbuettel (17:59:52): > Playing messenger here, updates inhttps://github.com/rocker-org/rocker-versioned2/issues/287– part of the story seems to be that (because of howwedeploy this RStudio image in PrairieLearn) we run some parts in the startup as a different user. That appars to allow RStudio to start on machines with M1 chips. As for what@Vince Careybrought up: there is a (recent-ish) ‘networked’ graphics device used by e.g. VS Code and other ‘non-graphical’ environments that dispatches to a browser. It’s clever and may help.https://cran.r-project.org/package=httpgd

2022-01-12

Vince Carey (06:16:32): > @Nitesh TuragaI have made progress with the prarielearning container noted at the end of issue 287 referenced above. Rstudio starts up nicely and BiocManager and various packages from CRAN and Bioc install cleanly. However installation of XVector failed with > > io_utils.c:16:10: fatal error: zlib.h: No such file or directory > 16 | #include <zlib.h> > | ^~~~~~~~~~~~~~ > compilation terminated. > make: ***** [/usr/local/lib/R/etc/Makeconf:168: io_utils.o] Error 1 > ERROR: compilation failed for package ‘XVector’ > * removing ‘/usr/local/lib/R/site-library/XVector’ > > despite zlibbioc installing fine. If you have a chance, see if there is a tweak to the dockerfile that will help …?

Vince Carey (06:27:56): > My guess is that there would be an addition here:https://github.com/PrairieLearn/PrairieLearn/blob/master/images/plbase/plbase-install.sh

Martin Morgan (06:58:08) (in thread): > zlibbioc is a placeholder for installing zlib on Windows; it does nothing on Linux / macOS since these have always had (an easy way to) install libz-devel. I believe that the ‘new’ Windows toolchain will also install zlib easily, so zlibbioc is redundant.

2022-01-18

Sean Davis (18:32:33): > Just in case it helps anyone, thebioconductor:bioconductor_docker:RELEASE_3_14container is available onOrchestra. For teaching purposes, it may be enough to have an 8-hour dedicated instance. Once you login to Orchestra, just search for 3.14 (version, not Pi).

Sean Davis (18:33:38): - File (PNG): image.png

Sean Davis (18:38:05) (in thread): > cc:@Ruth Isserlin

2022-01-19

Ruth Isserlin (10:01:46) (in thread): > Thanks!

2022-07-22

Gary Sieling (14:49:37): > @Gary Sieling has joined the channel

2022-07-28

Mervin Fansler (17:19:27): > @Mervin Fansler has joined the channel

2022-08-15

Michael Kaufman (13:14:27): > @Michael Kaufman has joined the channel

2022-08-26

Nicholas Cooley (13:23:22): > I have a docker container that i would like to automatically pull the dev version of a package tarball so i can build that package locally, if i do not know that package version number, what would be the correct way to do this? I assume that as soon as i know the package number, i can just wget the tarball with the appropriately constructed link, but i don’t know how to grab the version number if i don’t know it ahead of time.

Aidan Lakshman (13:23:51): > @Aidan Lakshman has joined the channel

Martin Morgan (13:57:27) (in thread): > If you mean the tar ball on the package landing page, you can download this with > > download.packages("<your-package">, ".", repos = BiocManager::repositories()) > > This assumes you’re using the ‘devel’ image andBiocManager::version()returns the devel version.

Nicholas Cooley (14:41:03) (in thread): > I can set to devel with:BiocManager::install(version='devel')correct?

Martin Morgan (14:56:24) (in thread): > yes, provided you’re using the correct version of R for devel – currently R 4.2.*. There is also the Bioconductor docker containerbioconductor/bioconductor_docker:develbut I realize it might not be appropriate for everything… > > Actually a little behind-the-scenes but using"repos = "https://bioconductor.org/packages/devel/bioc"will resolve to the current devel version. But really if you’re installing Bioconductor packages they should really all come from the same release, and for that you should be usingBiocManager::install(...)consistently…

Nicholas Cooley (15:34:04) (in thread): > Thanks that is good to know! We’re just compiling from source to implement multithreading correctly. In some cases we’ll want to pull from dev, and in others we’ll want to pull from release, in either case not hard coding the version number seemed like the correct approach.

2022-09-04

Gurpreet Kaur (15:00:04): > @Gurpreet Kaur has joined the channel

2022-09-27

Jennifer Holmes (16:14:49): > @Jennifer Holmes has joined the channel

2022-10-28

Brian Schilder (08:30:32): > @Brian Schilder has joined the channel

2022-11-08

James MacDonald (10:15:24): > @James MacDonald has joined the channel

2022-11-30

Chris Vanderaa (14:45:50): > @Chris Vanderaa has joined the channel

2022-12-09

Jianhai Zhang (16:45:05): > I want to reproduce errors in my package on Linux (Ubuntu 22.04.1 LTS) (http://bioconductor.org/checkResults/devel/bioc-LATEST/spatialHeatmap/) using the container athttps://www.bioconductor.org/help/docker/#current. Is there a specific container image for Linux (Ubuntu 22.04.1 LTS)?

Andres Wokaty (21:51:03) (in thread): > The current bioconductor_docker:devel is running 22.04.1: > > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES > 26a0137d573c bioconductor/bioconductor_docker:devel "/init" 6 seconds ago Up 5 seconds 0.0.0.0:8787->8787/tcp, :::8787->8787/tcp vigilant_cannon > ~/Work/Bioconductor/bioconductor_salt$ docker exec -it vigilant_cannon bash > root@26a0137d573c:/# cat /etc/os-release > PRETTY_NAME="Ubuntu 22.04.1 LTS" > NAME="Ubuntu" > VERSION_ID="22.04" > VERSION="22.04.1 LTS (Jammy Jellyfish)" > VERSION_CODENAME=jammy > ID=ubuntu > ID_LIKE=debian > HOME_URL="[https://www.ubuntu.com/](https://www.ubuntu.com/)" > SUPPORT_URL="[https://help.ubuntu.com/](https://help.ubuntu.com/)" > BUG_REPORT_URL="[https://bugs.launchpad.net/ubuntu/](https://bugs.launchpad.net/ubuntu/)" > PRIVACY_POLICY_URL="[https://www.ubuntu.com/legal/terms-and-policies/privacy-policy](https://www.ubuntu.com/legal/terms-and-policies/privacy-policy)" > UBUNTU_CODENAME=jammy >

2022-12-10

Jianhai Zhang (03:45:25) (in thread): > Got it.

2022-12-12

Umran (17:57:36): > @Umran has joined the channel

2023-01-16

Dario Righelli (05:46:22): > Is there a windows-based bioconductor docker image somewhere? I’m trying to reproduce an error I’m getting with SpatialExperiment on this platform and I don’t have any windows machine to test it

2023-01-17

Andres Wokaty (13:59:47) (in thread): > Not yet. You could try using the WinBuilder:https://blog.r-hub.io/2020/04/01/win-builder/. - Attachment (blog.r-hub.io): Everything you should know about WinBuilder - R-hub blog > If you’ve tried submitting a package on CRAN, you might have heard of WinBuilder, as it is mentioned in CRAN submission checklist. In this post inspired by reading R-package-devel archive, we shall go through important questions around WinBuilder. > What is WinBuilder and why use it? As mentioned in its public-facing web page, WinBuilder “provides services for building and checking R source packages for Windows.”. It offers checking on Windows for several R versions.

2023-01-19

Dario Righelli (03:54:19) (in thread): > sounds useful! Thanks!

2023-01-27

kent riemondy (10:50:32): > @kent riemondy has joined the channel

2023-02-03

Ciro Ramírez-Suástegui (07:02:12): > @Ciro Ramírez-Suástegui has joined the channel

2023-03-15

Jiefei Wang (11:17:12): > @Jiefei Wang has joined the channel

Mike Smith (11:44:53): > There seems to be some consternation about Docker removing their Free Team Organization tier e.g.https://blog.alexellis.io/docker-is-deleting-open-source-images/andhttps://web.docker.com/rs/790-SSB-375/images/privatereposfaq.pdfIs this something that might affect the Bioconductor Docker images? - Attachment (Alex Ellis’ Blog): Docker is deleting Open Source organisations - what you need to know > This controversial decision coupled with poor messaging has created anxiety the Open Source community. Learn what’s happening and how we can move forward.

Leonardo Collado Torres (12:55:49): > ping@Nick Eagles

Nick Eagles (12:55:51): > @Nick Eagles has joined the channel

Leonardo Collado Torres (12:56:08): > Nick and I were also discussing this with respect tohttps://hub.docker.com/orgs/libddocker/

Leonardo Collado Torres (12:56:41): > > Hmm, Nick, I’m not sure we actually benefit from any of the paid account benefits. I thinkhttps://hub.docker.com/orgs/libddocker/memberswould be automatically deleted, which would be a bummer buthttps://hub.docker.com/orgs/libddocker/billing/plan/updatestarts at $300 annually. However, with a personal free account, my understanding is that would be enough for ushttps://hub.docker.com/billing/plan/update. We might just need to make an account (libddocker if allowed or libddocker_individual) and simply share the password. A free personal account would allow us to have unlimited public repos, which is what we use anyways. They would limit the number of image pulls, but that’s ok since our docker images don’t have lots of pulls anyways. > > > > > > Or am I missing something? > > > > Best, > > Leo

Leonardo Collado Torres (12:57:00): > For BioC though, the part about the limited number of image pulls is probably the biggest restriction

2023-03-16

Marcel Ramos Pérez (10:05:59) (in thread): > @Alex MahmoudI think the alternative is the GitHub Container Registry. Is there a transition planned to that service or any other service? I guess pull commands would be affected.

Alex Mahmoud (10:06:19): > @Alex Mahmoud has joined the channel

Alex Mahmoud (12:29:01) (in thread): > Afaik we have the open source “paid” (but free) version of a Team account, so it shouldn’t affect us. We do have GHCR and GCR as registries too if it starts being an issue though

Alex Mahmoud (12:32:44) (in thread): > Got an email aboutcloudvebeing affected because it’s not paid, but bioc and galaxy accounts are already technically on a paid subscription so I am hoping we don’t need to do anything - File (PNG): image.png

Mike Smith (12:53:17) (in thread): > Excellent, that’s nice to know. Thanks for sharing the info.

2023-03-24

Leonardo Collado Torres (17:47:30): > yay! > > On March 14, 2023, we emailed you about your Free Team subscription, outlining our intention to sunset that plan. After listening to the concerns of the community, we’ve decided to reverse course, and are no longer sunsetting the Free Team plan.

2023-03-29

Alex Mahmoud (16:02:23): > Pasting question from the core channel to here for wider visibility to get everyone’s opinion: While not breaking any backward compatibility, I am proposing we start keeping containers per R version as rocker does, as well as add a new tag for each release which drops theRELEASE_which I personally find annoying to type. In concrete terms: the current 3.16 container is on R 4.2.2, and I am about to change it to 4.2.3. The new image would have 3 tags pointing to the same image:RELEASE_3_16,3.16, and3.16_R_4.2.3, and the current image would remain only as tag3.16_R_4.2.2and never rebuilt going forward. The change is a very low hanging fruit in that it is relatively trivial and the effort is minimal, so it’s more of an aesthetic/administrative than technical decision. Any opinions/objections regarding the new tags?

Himel Mallick (16:57:11): > @Himel Mallick has joined the channel

2023-03-31

Ilaria Billato (08:56:44): > @Ilaria Billato has joined the channel

2023-04-04

Jacques SERIZAY (05:02:42): > @Jacques SERIZAY has joined the channel

2023-04-28

James W. MacDonald (09:35:36): > @James W. MacDonald has joined the channel

2023-05-01

Ning Liu (21:35:20): > @Ning Liu has joined the channel

Ning Liu (21:41:20): > Hi all, I’m trying to build and push a Docker image of my workflow using theBuildABiocWorkshoptemplate with the default github workflow script (https://github.com/DavisLaboratory/GeoMXAnalysisWorkflow/blob/master/.github/workflows/basic_checks.yaml), but I keep getting the following error saying some packages are not available of this version of R, any ideas or suggestions? > > #10 2378.9 Warning message: > #10 2378.9 packages 'speckle', 'scater', 'standR', 'SpatialExperiment', 'edgeR' are not available for this version of R > #10 2378.9 > #10 2378.9 Versions of these packages for your version of R might be available elsewhere, > #10 2378.9 see the ideas at > #10 2378.9[https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages](https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages)#10 2378.9 Execution halted > ------ > Dockerfile:9 > -------------------- > 7 | RUN Rscript -e "options(repos = c(CRAN = '[https://cran.r-project.org](https://cran.r-project.org)')); BiocManager::install(ask=FALSE)" > 8 | > 9 | >>> RUN Rscript -e "options(repos = c(CRAN = '[https://cran.r-project.org](https://cran.r-project.org)')); devtools::install('.', dependencies=TRUE, build_vignettes=TRUE, repos = BiocManager::repositories())" > 10 | > -------------------- > ERROR: failed to solve: process "/bin/sh -c Rscript -e \"options(repos = c(CRAN = '[https://cran.r-project.org](https://cran.r-project.org)')); devtools::install('.', dependencies=TRUE, build_vignettes=TRUE, repos = BiocManager::repositories())\"" did not complete successfully: exit code: 1 > Error: buildx call failed with: ERROR: failed to solve: process "/bin/sh -c Rscript -e \"options(repos = c(CRAN = '[https://cran.r-project.org](https://cran.r-project.org)')); devtools::install('.', dependencies=TRUE, build_vignettes=TRUE, repos = BiocManager::repositories())\"" did not complete successfully: exit code: 1 >

2023-05-02

Alex Mahmoud (00:04:03): > I don’t think this is container related, but rather a more general issue with those packages in devel. It seemsedgeRis failing in devel and that error is cascading to other packages. I will bring it up internally to make sure. In the meantime, changing to the 3.17 container might work

Alex Mahmoud (10:25:36): > @Ning LiuI can confirm this is an issue with the packages in devel and not related to the containers. Relaying Lori’s response from#developers-forumto a similar report: > “packages can be found on the builder because we do an install step before build/check – but if the package can’t be propagated and found by the general use because of failed propagation of a dependency, while it will show a clean build report it will not propagate until its dependency can be available” > > I believe in your specific case,edgeRfailing is the root, and other packages are subsequently failing due to having failed dependencies

Ning Liu (22:01:56): > @Alex Mahmoudthanks, yeah changing to 3.17 container did address the problem.

2023-05-16

Nitesh Turaga (15:25:38): > Hello Bioc team! > > I hope you are all doing well:slightly_smiling_face:I’m trying to install some CRAN packages usingBiocManager::install()on the new RELEASE_3_17 container, and it seems like these packages are available in thePACKAGES.gzfile in the CRAN style repo index but not available to download from the google bucket, andRisn’t passing down the list inBiocManager::repositories()if it fails? > > Please correct me if i’m wrong here. For reference, i’ve added theBiocManager::repositories()output, and the install trials from bothinstall.packages()andBiocManager::install(). > > > BiocManager::repositories() > 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for details. > Replacement repositories: > CRAN:[https://packagemanager.posit.co/cran/__linux__/jammy/latest](https://packagemanager.posit.co/cran/__linux__/jammy/latest)BioCcontainers BioCsoft > "[https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker](https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker)" "[https://bioconductor.org/packages/3.17/bioc](https://bioconductor.org/packages/3.17/bioc)" > BioCann BioCexp > "[https://bioconductor.org/packages/3.17/data/annotation](https://bioconductor.org/packages/3.17/data/annotation)" "[https://bioconductor.org/packages/3.17/data/experiment](https://bioconductor.org/packages/3.17/data/experiment)" > BioCworkflows BioCbooks > "[https://bioconductor.org/packages/3.17/workflows](https://bioconductor.org/packages/3.17/workflows)" "[https://bioconductor.org/packages/3.17/books](https://bioconductor.org/packages/3.17/books)" > CRAN > "[https://packagemanager.posit.co/cran/__linux__/jammy/latest](https://packagemanager.posit.co/cran/__linux__/jammy/latest)" > > > > > install.packages("gargle") > Installing package into '/usr/local/lib/R/host-site-library' > (as 'lib' is unspecified) > trying URL '[https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/gargle_1.4.0.tar.gz](https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/gargle_1.4.0.tar.gz)' > Content type 'binary/octet-stream' length 720409 bytes (703 KB) > ================================================== > downloaded 703 KB > > * installing **binary** package 'gargle' ... > * DONE (gargle) > > The downloaded source packages are in > '/tmp/Rtmp8IqI74/downloaded_packages' > > > > > BiocManager::install("gargle") > 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for details. > Replacement repositories: > CRAN:[https://packagemanager.posit.co/cran/__linux__/jammy/latest](https://packagemanager.posit.co/cran/__linux__/jammy/latest)Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21) > Installing package(s) 'gargle' > trying URL '[https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz](https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz)' > Error in download.file(url, destfile, method, mode = "wb", ...) : > cannot open URL '[https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz](https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz)' > In addition: Warning messages: > 1: In download.file(url, destfile, method, mode = "wb", ...) : > downloaded length 0 != reported length 0 > 2: In download.file(url, destfile, method, mode = "wb", ...) : > cannot open URL '[https://storage.googleapis.com/bioconductor-packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz](https://storage.googleapis.com/bioconductor-packages/3.17/container-binaries/bioconductor_docker/src/contrib/gargle_1.4.0_R_x86_64-pc-linux-gnu.tar.gz)': HTTP status was '404 Not Found' > Warning in download.packages(pkgs, destdir = tmpd, available = available, : > download of package 'gargle' failed >

Alex Mahmoud (15:30:41): > I just copied over the new ones, binaries are still transferring to match the index… I was hoping it would be faster and the time gap wouldn’t be an issue but I guess for next pass I’ll have to do it differently

Alex Mahmoud (15:41:29): > gargle 1.4.0 transfer happened at this point fwiw

Nitesh Turaga (15:43:41): > Thank you!

Alex Mahmoud (17:08:49): > I’ve also updated the script to exclude the PACKAGES files from the dev bucket and just rewrite them for prod bucket after the transfer, so it will better combine old+new versions and there shouldn’t be any more time gaps in the future

2023-05-17

Samuel Gamboa (09:50:59): > @Samuel Gamboa has joined the channel

Samuel Gamboa (10:00:09): > Hello. From theBioc Website: “NOTE: If you forget to add the tag devel or RELEASE_X_Y while using the bioconductor/bioconductor_docker image, it will automatically use the latest tag which points to the latest RELEASE version of Bioconductor” > > However, I get the 3.16 release if I omit RELEASE_3_17. The same happens if I use the tag bioconductor_docker:latest > > docker run -e PASSWORD=<password> \ > -p 8787:8787 \ > bioconductor/bioconductor_docker:latest > > Or > > docker run -e PASSWORD=<password> \ > -p 8787:8787 \ > bioconductor/bioconductor_docker >

Marcel Ramos Pérez (10:45:00) (in thread): > Have you done adocker pull?

Samuel Gamboa (10:53:26) (in thread): > @Marcel Ramos Pérez, I didn’t to that before. I just did but got the same version (3.16). I don’t know it this is related, but the last push of the ‘latest’ tag was done 4 months ago while the others a few days ago:https://hub.docker.com/r/bioconductor/bioconductor_docker/tags

Marcel Ramos Pérez (10:56:28) (in thread): > @Alex MahmoudIt looks like thelatesttag wasn’t updated by the biocdockerbot

Alex Mahmoud (11:25:53) (in thread): > latestdoesn’t get rebuilt automatically anymore since I stopped the docker builds in favor of GHA. I will sync it manually for now, and then we should discuss internally what tag we want it to follow. I think it was traditionally the latest release but can’t remember

Alex Mahmoud (11:36:20) (in thread): > Synced with latest3.17. Will deal with setting up auto sync later today

Samuel Gamboa (11:57:51) (in thread): > Thanks!

2023-05-18

Sarvesh Nikumbh (03:38:17): > @Sarvesh Nikumbh has joined the channel

Oluwafemi Oyedele (05:53:19): > @Oluwafemi Oyedele has joined the channel

2023-05-20

Franck RICHARD (06:15:13): > @Franck RICHARD has joined the channel

2023-06-07

Alyssa Obermayer (18:28:50): > @Alyssa Obermayer has joined the channel

2023-07-25

Nitesh Turaga (17:18:47): - Attachment: Attachment > The BioCcontainers repo doesn’t want to show up as available for RELEASE_3_17 branch. What am I doing wrong? > > ➜ Documents docker run -it bioconductor/bioconductor_docker:RELEASE_3_17 bash > root@30573ea624d5:/# cd > root@30573ea624d5:~# R > > R version 4.3.0 (2023-04-21) -- "Already Tomorrow" > Copyright (C) 2023 The R Foundation for Statistical Computing > Platform: aarch64-unknown-linux-gnu (64-bit) > > R is free software and comes with ABSOLUTELY NO WARRANTY. > You are welcome to redistribute it under certain conditions. > Type 'license()' or 'licence()' for distribution details. > > Natural language support but running in an English locale > > R is a collaborative project with many contributors. > Type 'contributors()' for more information and > 'citation()' on how to cite R or R packages in publications. > > Type 'demo()' for some demos, 'help()' for on-line help, or > 'help.start()' for an HTML browser interface to help. > Type 'q()' to quit R. > > > BiocManager::repositories() > 'getOption("repos")' replaces Bioconductor standard repositories, see > 'help("repositories", package = "BiocManager")' for details. > Replacement repositories: > CRAN: [https://packagemanager.posit.co/cran/latest](https://packagemanager.posit.co/cran/latest) > BioCsoft > "[https://bioconductor.org/packages/3.17/bioc](https://bioconductor.org/packages/3.17/bioc)" > BioCann > "[https://bioconductor.org/packages/3.17/data/annotation](https://bioconductor.org/packages/3.17/data/annotation)" > BioCexp > "[https://bioconductor.org/packages/3.17/data/experiment](https://bioconductor.org/packages/3.17/data/experiment)" > BioCworkflows > "[https://bioconductor.org/packages/3.17/workflows](https://bioconductor.org/packages/3.17/workflows)" > BioCbooks > "[https://bioconductor.org/packages/3.17/books](https://bioconductor.org/packages/3.17/books)" > CRAN > "[https://packagemanager.posit.co/cran/latest](https://packagemanager.posit.co/cran/latest)"

2023-07-26

Marcel Ramos Pérez (09:30:29): > This works for me. Are you using a derivative container? Check that the environment variable is available > > Sys.getenv("BIOCONDUCTOR_DOCKER_VERSION") > [1] "3.17.34" > > which should populatecontainerRepository()

Alex Mahmoud (14:38:13): > Hey@Nitesh Turaga! Can also confirm it works as expected on my end. > > docker run --rm -it --platform linux/amd64 bioconductor/bioconductor_docker:RELEASE_3_17 Rscript -e "BiocManager::repositories()" > 'getOption("repos")' replaces Bioconductor standard repositories, see > 'help("repositories", package = "BiocManager")' for details. > Replacement repositories: > CRAN:[https://packagemanager.posit.co/cran/__linux__/jammy/latest](https://packagemanager.posit.co/cran/__linux__/jammy/latest)BioCcontainers > "[https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker](https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker)" > BioCsoft > "[https://bioconductor.org/packages/3.17/bioc](https://bioconductor.org/packages/3.17/bioc)" > BioCann > "[https://bioconductor.org/packages/3.17/data/annotation](https://bioconductor.org/packages/3.17/data/annotation)" > BioCexp > "[https://bioconductor.org/packages/3.17/data/experiment](https://bioconductor.org/packages/3.17/data/experiment)" > BioCworkflows > "[https://bioconductor.org/packages/3.17/workflows](https://bioconductor.org/packages/3.17/workflows)" > BioCbooks > "[https://bioconductor.org/packages/3.17/books](https://bioconductor.org/packages/3.17/books)" > CRAN > "[https://packagemanager.posit.co/cran/__linux__/jammy/latest](https://packagemanager.posit.co/cran/__linux__/jammy/latest)" >

Alex Mahmoud (14:39:10): > One thing to note though is that now we havearm64containers as well as the defaultamd64, and thearm64containers have binaries disabled (because they are incompatible). We do have arm64 binaries built and will hopefully get them into production for 3.18. In the meantime, you might have to add--platform linux/amd64to your docker run command to make sure you are using the right container if you are on anarm64machine (eg M chip macs). Note however that it will use an emulator if you are on anarm64and explicitly request thelinux/amd64container, and that can slow down your computation

Nitesh Turaga (14:59:32): > Thanks@Alex Mahmoudand@Marcel Ramos Pérez! That helps a lot.

2023-07-28

Konstantinos Daniilidis (13:47:26): > @Konstantinos Daniilidis has joined the channel

2023-08-03

Tyrone Lee (10:03:54): > @Tyrone Lee has joined the channel

Abiud Cantu (10:59:43): > @Abiud Cantu has joined the channel

Ludwig Geistlinger (11:04:56): > @Ludwig Geistlinger has joined the channel

2023-08-18

Alexander Bender (04:25:56): > @Alexander Bender has joined the channel

2023-08-24

Lachlan Baer (01:20:34): > @Lachlan Baer has joined the channel

2023-08-30

Nitesh Turaga (16:01:25): > Hello! Is there an R 4.3.1 version available for the bioconductor docker images?

Hervé Pagès (16:18:36): > @Hervé Pagès has left the channel

Marcel Ramos Pérez (16:37:02): > Thebioconductor/bioconductor_docker:develimage is using R 4.3.1

Nitesh Turaga (17:37:39): > Got it!! thanks

2023-09-07

Alexander Bender (12:35:16): > Hello. Has it ever been discussed to change the base image for the Bioconductor container? Currently it is rocker/rstudio which is quite nice and I use it daily. Still, rocker/binder would provide RStudio as well, plus comes with many other handy features such as JupyterNotebook, dedicated terminal window, VScode… > Wouldn’t that be a more versatile choice compared to rocker/rstudio?

Vince Carey (13:31:32): > Thanks for the comment. I am going to have a look.

2023-09-11

Jacques SERIZAY (04:59:12): > FYI there seems to be an issue in the Dockerfile when downloadingRenviron.sitethrough unsecured HTTP. I’ve opened a PR for thedevelDocker image (https://github.com/Bioconductor/bioconductor_docker/pull/85), although it wasn’t clear to me whether direct contributions were welcome or not? - Attachment: #85 fix: update curl to secured http link > Not sure whether PRs are welcome for this repo, feel free to close it if not. > > The Dockerfile attempts to fetch Renviron.bioc hosted by Bioconductor, but it looks like unsecured HTTP links are not available anymore for [bioconductor.org](http://bioconductor.org) domain. > > > $ curl [http://bioconductor.org/checkResults/devel/bioc-LATEST/Renviron.bioc](http://bioconductor.org/checkResults/devel/bioc-LATEST/Renviron.bioc) | head > > <html> > <head><title>301 Moved Permanently</title></head> > <body> > <center><h1>301 Moved Permanently</h1></center> > <hr><center>CloudFront</center> > </body> > </html> > > $ curl https//bioconductor.org/checkResults/devel/bioc-LATEST/Renviron.bioc | head > > # ==================================================================== > # Environment variables used on the Bioconductor build machines to > # control the behavior of R 4.3 for the BioC 3.18 builds > # ==================================================================== > # > # BIOCONDUCTOR PACKAGE DEVELOPERS/MAINTAINERS: Please use the settings > # below on your machine when working on the devel branch of your > # package. Also make sure to use a recent version of R 4.3. This > # should allow you to reproduce any error or warning you see on the > # Bioconductor build reports. The easiest way to use the settings > > > When in the devel docker container, this results in installation of system dependencies to fail when using this command (from neurogenomics/rworkflows): > > > $ sysreqs=$(Rscript -e 'cat("apt-get update -y && apt-get install -y", paste(gsub("apt-get install -y ", "", remotes::system_requirements("ubuntu", "20.04")), collapse = " "))') > > $ echo $sysreqs > File /usr/local/lib/R/etc//Renviron.site contains invalid line(s) <html> <head><title>301 Moved Permanently</title></head> <body> <center><h1>301 Moved Permanently</h1></center> <hr><center>CloudFront</center> </body> </html> They were ignored apt-get update -y && apt-get install -y libcurl4-openssl-dev libssl-dev pandoc make libicu-dev libxml2-dev > > > And more generally an invalid [Renviron.site](http://Renviron.site) file: > > > $ R > > File /usr/local/lib/R/etc//Renviron.site contains invalid line(s) > <html> > <head><title>301 Moved Permanently</title></head> > <body> > <center><h1>301 Moved Permanently</h1></center> > <hr><center>CloudFront</center> > </body> > </html> > They were ignored > > > R version 4.3.1 (2023-06-16) -- "Beagle Scouts" > Copyright (C) 2023 The R Foundation for Statistical Computing > Platform: x86_64-pc-linux-gnu (64-bit) >

2023-10-11

Romane Libouban (05:41:42): > @Romane Libouban has joined the channel

2023-11-08

Nitesh Turaga (12:12:29): > Hello, it seems like some binary packages on RELEASE_3_17 have not built e.gGenomeInfoDb,GenomicRanges–> forcing them to install from source. > > Is there a plan to rebuild the binaries for RELEASE_3_17 to support the packages that are missing a binary?

Alex Mahmoud (13:22:12): > Hey Nitesh, thanks for the report! Yes, I’ll be working on fixing the binary build process for 3.17 and building for 3.18 soon. I haven’t touched the binary building in a bit, focus has been onhttps://github.com/Bioconductor/bioc2u(in case of interest, bioc2u and r2u can provide binaries + sysdeps viaaptif you’re in an ubuntu system) and other projects, but I noticed that due to a change with Github action disk capacity (building entirely on github runners atm), some packages are not building anymore. Working on PR for the anvil jupyter container then fixing binary building will be my main focus next

Nitesh Turaga (14:27:57): > Hi, Alex! This is good to know. Just wondering about the few packages that are missing on 3_17. Thank you for all your work!

2023-11-20

Samuel Gamboa (23:46:05) (in thread): > I think latest has not been updated:https://hub.docker.com/r/bioconductor/bioconductor_docker/tags.

2023-12-19

Vince Carey (09:52:01): > @Andres Wokaty@Alex Mahmoudcan you pin information about the salt container here? How to pull the latest version for devel branch e.g.

Vince Carey (09:52:22): > or bbs container

Vince Carey (09:58:49): > Well, seehttps://github.com/Bioconductor/bioconductor_salt

Andres Wokaty (11:41:30) (in thread): > These containers use configurations similar to the Linux build machines. They don’t have all dependencies (such as dotnet for rmspc) or might need additional, manual configuration for some packages (such as credentials for immuneSpaceR) but we hope to get very close. They might be helpful for troubleshooting issues you encounter with the Nebbiolos. > {#devel} > > docker run -it ghcr.io/bioconductor/bioconductor_salt:devel-jammy > > # release > > docker run -it ghcr.io/bioconductor/bioconductor_salt:jammy > > Packages:https://github.com/Bioconductor/bioconductor_salt/pkgs/container/bioconductor_saltREADME:https://github.com/Bioconductor/bioconductor_salt?tab=readme-ov-file#simulating-the-bbs-ubuntu-environment-in-a-container

2023-12-27

Cindy Reichel (14:38:26): > @Cindy Reichel has joined the channel

2024-01-02

Dan Bunis (17:19:44): > @Dan Bunis has joined the channel

2024-01-12

Davide Risso (04:43:16): > Hi all, I’m getting the following error within the latest Bioc-devel container when trying to install a package: > > > BiocManager::install("S4Arrays") > [...] > Installing package(s) 'S4Arrays' > trying URL '[https://bioconductor.org/packages/3.19/bioc/src/contrib/S4Arrays_1.3.1.tar.gz](https://bioconductor.org/packages/3.19/bioc/src/contrib/S4Arrays_1.3.1.tar.gz)' > Content type 'application/x-gzip' length 281076 bytes (274 KB) > ================================================== > downloaded 274 KB > > * installing **source** package 'S4Arrays' ... > **** using staged installation > **** libs > using C compiler: 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0' > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I'/usr/local/lib/R/site-library/S4Vectors/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c R_init_S4Arrays.c -o R_init_S4Arrays.o > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I'/usr/local/lib/R/site-library/S4Vectors/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c S4Vectors_stubs.c -o S4Vectors_stubs.o > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I'/usr/local/lib/R/site-library/S4Vectors/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c abind.c -o abind.o > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I'/usr/local/lib/R/site-library/S4Vectors/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c array_selection.c -o array_selection.o > array_selection.c: In function 'C_Lindex2Mindex': > array_selection.c:352:17: error: format not a string literal and no format arguments [-Werror=format-security] > 352 | error(errmsg_buf()); > | ^~~~~~~~~ > array_selection.c: In function 'C_Mindex2Lindex': > array_selection.c:416:17: error: format not a string literal and no format arguments [-Werror=format-security] > 416 | error(errmsg_buf()); > | ^~~~~~~~~ > cc1: some warnings being treated as errors > make: ***** [/usr/local/lib/R/etc/Makeconf:191: array_selection.o] Error 1 > ERROR: compilation failed for package 'S4Arrays' > * removing '/usr/local/lib/R/site-library/S4Arrays' >

Davide Risso (04:44:04): > (Note that S4Arrays dependencies can be installed with no issues)

Davide Risso (04:45:29): > I was under the impression that Bioc containers should install binary packages (or is this just for RELEASE_X_XX?). More importantly, why am I getting this compilation error? Note that S4Arrays builds fine in the build system:https://bioconductor.org/checkResults/devel/bioc-LATEST/S4Arrays/

Davide Risso (04:45:58): > This is the image I’m using: > > $ docker images > REPOSITORY TAG IMAGE ID CREATED SIZE > bioconductor/bioconductor_docker devel 8aa39b3ae73e 6 days ago 4.13GB >

Davide Risso (04:46:16): > Any help is highly appreciated!

Mike Smith (04:58:22): > Hi@Davide RissoThere was some discussion on this issue here:https://community-bioc.slack.com/archives/CLUJWDQF4/p1702970799150809I think the problems withS4Arrayshave now been patched, but it might take a day or so for the new version to propagate. > > I think the container binaries are only for the RELEASE_X_Y images, but maybe someone else can confirm that. - Attachment: Attachment > Hi all, > I’m having issues with the current Bioconductor docker image (from Friday 15.12.). rlang failed to update: > > BiocManager::install("rlang") > 'getOption("repos")' replaces Bioconductor standard repositories, see > 'help("repositories", package = "BiocManager")' for details. > Replacement repositories: > CRAN: [https://cloud.r-project.org](https://cloud.r-project.org) > Bioconductor version 3.19 (BiocManager 1.30.22), R Under development (unstable) > (2023-12-13 r85679) > Installing package(s) 'rlang' > trying URL '[https://cloud.r-project.org/src/contrib/rlang_1.1.2.tar.gz](https://cloud.r-project.org/src/contrib/rlang_1.1.2.tar.gz)' > Content type 'application/x-gzip' length 763521 bytes (745 KB) > ================================================== > downloaded 745 KB > > * installing **source** package 'rlang' ... > **** package 'rlang' successfully unpacked and MD5 sums checked > **** using staged installation > **** libs > using C compiler: 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0' > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I./rlang/ -I/usr/local/include -fvisibility=hidden -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c capture.c -o capture.o > gcc -I"/usr/local/lib/R/include" -DNDEBUG -I./rlang/ -I/usr/local/include -fvisibility=hidden -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c internal.c -o internal.o > In file included from internal/internal.c:32, > from internal.c:1: > internal/tests.c: In function 'ffi_test_Rf_warning': > internal/tests.c:104:3: error: format not a string literal and no format arguments [-Werror=format-security] > 104 | Rf_warning(r_chr_get_c_string(msg, 0)); > | ^~~~~~~~~~~~~~~~~~ > internal/tests.c: In function 'ffi_test_Rf_error': > internal/tests.c:108:3: error: format not a string literal and no format arguments [-Werror=format-security] > 108 | Rf_error(r_chr_get_c_string(msg, 0)); > | ^~~~~~~~~~~~~~ > internal/tests.c: In function 'ffi_test_Rf_warningcall': > internal/tests.c:113:3: error: format not a string literal and no format arguments [-Werror=format-security] > 113 | Rf_warningcall(call, r_chr_get_c_string(msg, 0)); > | ^~~~~~~~~~~~~~~~~~~~~~~~~~ > internal/tests.c: In function 'ffi_test_Rf_errorcall': > internal/tests.c:117:3: error: format not a string literal and no format arguments [-Werror=format-security] > 117 | Rf_errorcall(call, r_chr_get_c_string(msg, 0)); > | ^~~~~~~~~~~~~~~~~~~~~~ > cc1: some warnings being treated as errors > make: ***** [/usr/local/lib/R/etc/Makeconf:191: internal.o] Error 1 > ERROR: compilation failed for package 'rlang' > * removing '/usr/local/lib/R/host-site-library/rlang' > don’t know if that’s an issue of rlang or of the docker? anybody having a solution to this? > > sessionInfo from the docker: > > sessionInfo() > R Under development (unstable) (2023-12-13 r85679) > Platform: x86_64-pc-linux-gnu > Running under: Ubuntu 22.04.3 LTS > > Matrix products: default > BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 > LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so; LAPACK version 3.10.0 > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > time zone: Etc/UTC > tzcode source: system (glibc) > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > loaded via a namespace (and not attached): > [1] compiler_4.4.0

Davide Risso (05:06:47): > Thanks@Mike Smith! Sorry, I didn’t think of looking at the#developers-forumchannel! Good to know that things have been fixed and I’ll wait for them to propagate!

Vince Carey (10:23:04): > after misreading the bioc-devel message on this i posted that you can install from source in the container

2024-01-27

atongsa miyamoto (04:52:35): > @atongsa miyamoto has joined the channel

2024-03-11

Melysssa Minto (10:14:02): > @Melysssa Minto has joined the channel

2024-03-15

Peace Sandy (08:54:41): > @Peace Sandy has joined the channel

2024-04-04

Alexandru Mizeranschi (09:37:17): > @Alexandru Mizeranschi has joined the channel

2024-04-28

Danielle Callan (08:28:44): > @Danielle Callan has joined the channel

2024-05-06

Samuel Gamboa (14:37:01) (in thread): > @Alex Mahmoud, it seems that this note on the website is no longer valid? Not providing any tag keeps pointing to 3.17.https://www.bioconductor.org/help/docker/ - Attachment (bioconductor.org): Bioconductor - Docker for Bioconductor > The Bioconductor project aims to develop and share open source software for precise and repeatable analysis of biological data. We foster an inclusive and collaborative community of developers and data scientists. - File (PNG): image.png

Alex Mahmoud (15:02:02) (in thread): > Updated manually and made a quick workflow to auto-update:https://github.com/Bioconductor/bioconductor_docker/pull/102will test and merge soon. Thanks for pointing it out, it had fallen off my radar!! - Attachment: #102 Create update_latest.yaml

2024-05-08

Artur Sannikov (12:22:46): > @Artur Sannikov has joined the channel

2024-06-05

Adrian Hirt (05:47:38): > @Adrian Hirt has joined the channel

2024-07-04

Sounkou Mahamane Toure (15:28:14): > @Sounkou Mahamane Toure has joined the channel

2024-07-05

Antonin Thiébaut (04:38:00): > @Antonin Thiébaut has joined the channel

2024-08-09

Charlotte Soneson (05:15:01): > At the moment, all my GitHub Actions workflows that use Bioconductor Docker devel container fail with the messageError in setwd(libdir) : cannot change working directory(in theR CMD checkpart). An example is here:https://github.com/csoneson/iCOBRA/actions/runs/10316573094/job/28559082070. Could there be an issue in the latest image, or is there something I should change on my side? For comparison, last Friday morning it worked fine (https://github.com/csoneson/iCOBRA/actions/runs/10212910913/job/28258667304). Thanks!

Marcel Ramos Pérez (09:52:20) (in thread): > TheR_LIBS_USERlooks different (/github/home/R/x86_64-pc-linux-gnu-library/4.4). It might be something to do with the container configuration. cc:@Alex Mahmoud

Alex Mahmoud (10:02:32) (in thread): > Idon’t thinkanything has changed from last week so this is strange, but will look into it

Charlotte Soneson (10:04:13) (in thread): > Thanks both!

Alex Mahmoud (10:58:30) (in thread): > That env variable is not set in the container, but I was able to track it down to the r-lib action and then this PRhttps://github.com/r-lib/actions/commit/9677f2e32657bbce94feb8bc3786495c443d5d60which added logic to setR_LIBS_USERwhen unset. Not entirely sure yet what easiest way around it would be, will test some options. I think setting that env variable explicitly might be enough in the interim to pass, but will try to find something less manual

Charlotte Soneson (11:01:18) (in thread): > Oh, good catch! That makes sense, I just now tried running an action with an older container and I see the same thing.

Alex Mahmoud (12:44:44) (in thread): > Discussed this briefly at the team meeting and I think there are a few things that could happen in parallel: > 1. For a fix on their side: We should probably open an issue onr-lib/actionsto report that this change caused unintended consequences, and potentially PR a quick change that makes this conditional (eg: a flag to the action to enable/disable setting the env variable). > 2. For a fix on our side, especially if there are issues accomplishing the above on their side, we could start setting this by default in the container image, to a path that is known to work within the container. This could have unintended consequences though, so we would have to do it in devel and monitor to ensure nothing is breaking, but I think it’d mostly be affecting to folks who have hardcoded a libpath based on current default which should ideally not be too many > Anyone has any thoughts/considerations/preference for either? And@Charlotte Sonesonwould you be able to open the issue to r-lib/actions or would you prefer I do it?

Charlotte Soneson (13:45:43) (in thread): > Thanks - I can give it a go with the issue and ask if they would be willing to add a flag (I likely won’t get to it until tomorrow evening or Sunday though, as I am traveling for vacation).

Alex Mahmoud (13:46:25) (in thread): > No worries, enjoy your vacation and don’t worry about it, I can do it!

2024-08-10

Charlotte Soneson (01:38:49) (in thread): > Ok - if you get to it before, obviously feel free! Thanks:pray:

2024-08-14

Charlotte Soneson (02:08:55) (in thread): > Issue opened here:https://github.com/r-lib/actions/issues/912. - Attachment: #912 Conditional setting of R_LIBS_USER > Hello! > > We have noticed that a recent change to setup-r-dependencies, setting R_LIBS_USER if unset, had downstream effects on actions run in the Bioconductor Docker container. For an example, see here, the explicit error is > > > Error in setwd(libdir) : cannot change working directory > > > > due to the change in R_LIBS_USER. > > We are considering options (such as setting the R_LIBS_USER in the container, which may also have an impact on users relying on the current default), but wanted to ask if you would be open to adding a flag to the action, to make it possible for the user to choose whether R_LIBS_USER should be set if missing, or not. > > cc @almahmoud > > Thanks in advance!

Charlotte Soneson (03:50:44) (in thread): > @Alex Mahmoud,@Marcel Ramos PérezWould one of you with more insights into the containers be willing to answer Gabor’s question in the issue?:pray:

2024-08-19

Rema Gesaka (09:41:03): > @Rema Gesaka has joined the channel

2024-08-27

Bharat Mishra (11:36:39): > @Bharat Mishra has joined the channel

2024-10-02

Eva Hamrud (19:05:10): > @Eva Hamrud has joined the channel

2025-01-09

Ammar Sabir Cheema (11:30:58): > @Ammar Sabir Cheema has joined the channel

2025-02-17

Davide Risso (02:54:30): > Hi all, I hope this is the right channel. Mysconepackage is failing in devel and I’m trying to reproduce the error in the bioconductor devel Docker. However, when trying to installingbeachmat(an indirect dependency ofscone). I get the following error: > > > BiocManager::install("beachmat") > 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for > details. > Replacement repositories: > CRAN:[https://cloud.r-project.org](https://cloud.r-project.org)Bioconductor version 3.21 (BiocManager 1.30.25), R Under development (unstable) (2025-02-12 r87715) > Installing package(s) 'beachmat' > trying URL '[https://bioconductor.org/packages/3.21/bioc/src/contrib/beachmat_2.23.6.tar.gz](https://bioconductor.org/packages/3.21/bioc/src/contrib/beachmat_2.23.6.tar.gz)' > Content type 'application/x-gzip' length 377367 bytes (368 KB) > ================================================== > downloaded 368 KB > > * installing **source** package 'beachmat' ... > **** this is package 'beachmat' version '2.23.6' > **** using staged installation > **** libs > using C++ compiler: 'g++ (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0' > using C++17 > g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I../inst/include -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/host-site-library/assorthead/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o > g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I../inst/include -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/host-site-library/assorthead/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c constant_matrix.cpp -o constant_matrix.o > g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I../inst/include -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/host-site-library/assorthead/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c delayed_isometric_binary.cpp -o delayed_isometric_binary.o > g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I../inst/include -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/host-site-library/assorthead/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c delayed_isometric_math.cpp -o delayed_isometric_math.o > g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I../inst/include -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/host-site-library/assorthead/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c delayed_isometric_unary.cpp -o delayed_isometric_unary.o > g++: fatal error: Killed signal terminated program cc1plus > compilation terminated. > make: ***** [/usr/local/lib/R/etc/Makeconf:210: delayed_isometric_unary.o] Error 1 > ERROR: compilation failed for package 'beachmat' > * removing '/usr/local/lib/R/host-site-library/beachmat' > > Any idea of what’s going on here? Any help is much appreciated!

Davide Risso (02:56:30): > For completeness, this is the image I’m using: > > $ docker images > REPOSITORY TAG IMAGE ID CREATED SIZE > bioconductor/bioconductor_docker devel da64f1f7013d 2 days ago 4.93GB >

Mike Smith (03:34:09): > Hi@Davide RissoI don’t see that error when I try to run the same operation in the same container version. However, if I constrain the memory available to Docker then I can reproduce the error e.g.docker run -it --rm --memory 1gghcr.io/bioconductor/bioconductor_docker:develwill lead to that failure. Do you have anything on your system limiting the memory available to the container?

Davide Risso (08:22:09): > ha! Good point, will try to increase the memory size!

Davide Risso (08:23:09): > I think it’s at 2GB now

Davide Risso (09:30:43): > I can confirm that increasing the RAM memory available to the container solved the problem. Thanks@Mike Smith!

Davide Risso (09:32:17) (in thread): > For future reference (in case someone has the same issue): Docker on Mac limits RAM memory available to containers to 2GB by default (at least on my system). You can increase it by opening Docker and going to Preferences -> Resources

2025-02-19

Kasper D. Hansen (09:38:11) (in thread): > How much did you have to increase it to?

Davide Risso (09:46:46) (in thread): > well… I increased it to 16GB, but didn’t try any intermediate value… I can try a more systematic benchmark if it’s of interest

Kasper D. Hansen (09:57:40) (in thread): > I am thinking this would be a pretty good addition to our documents on using docker.

Kasper D. Hansen (09:58:04) (in thread): > Especially because the error does not make it clear it is memory related

2025-03-10

Pariksheet Nanda (13:48:56): > @Pariksheet Nanda has joined the channel

2025-03-12

Thomaz Bastiaanssen (04:01:16): > @Thomaz Bastiaanssen has joined the channel

Thomaz Bastiaanssen (04:05:06): > Hi all, I have a question you may know off the top of your head: > > When including new S4 classes in a Bioconductor submission, what is the stance towards having some of the slots be for non-S4Vector (etc) classes? > > e.g.,slots = c(my_slot = data.frame(... >vsslots = c(my_slot = DataFrame(... >I wasn’t able to find a clear statement, would love to know your thoughts

Marcel Ramos Pérez (08:30:49) (in thread): > Hi Thomaz, I think it would be better to ask in the#biocclasseschannel (edit: or#developers-forum)

Thomaz Bastiaanssen (08:46:45) (in thread): > Cheers Marcel!

2025-03-14

Kasper D. Hansen (15:52:04): > I am trying to run the bioconductor devel container on macOS on Apple Silicon. I have installed Docker Desktop.

Kasper D. Hansen (15:52:13): > In my terminal, I write for example,

Kasper D. Hansen (15:52:32): > > % docker pull bioconductor/bioconductor_docker:devel > Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found >

Kasper D. Hansen (15:53:26): > with this, it is perhaps not surprising that the quickstart fails similarly

Kasper D. Hansen (15:53:44): > > % docker run \ > -e PASSWORD=bioc \ > -p 8787:8787 \ > bioconductor/bioconductor_docker:devel > Unable to find image 'bioconductor/bioconductor_docker:devel' locally > docker: Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found. > See 'docker run --help'. >

Kasper D. Hansen (15:54:05): > I am trying to follow the description onhttps://bioconductor.org/help/docker/

Kasper D. Hansen (15:58:32): > I am kind of inexperience with docker. When I go tohttps://hub.docker.com/r/bioconductor/bioconductor_docker/tagsI can see that the devel tag only listsamd64under architecture and notarm64. Is this the issue?

Dirk Eddelbuettel (16:00:29) (in thread): > I was about to bring this up, having fairly recently started to build arm64 containers (for #r2u) and running into that. I think you can (thanks to Rosetta) run the amd64 image by using the ‘–platform …’ argument.

Kasper D. Hansen (16:03:05) (in thread): > Thanks. If I add--platform=amd64in thedocker pullcommand I had listed above, something is being downloaded. That’s a good start

Dirk Eddelbuettel (16:03:27) (in thread): > Same for thedocker run ...

Marcel Ramos Pérez (16:04:04) (in thread): > It hasn’t been built recently but it’s therehttps://hub.docker.com/layers/bioconductor/bioconductor_docker/devel-arm64/images/sh[…]ba36b9d2e95e6a5cc6c735632124876fd58e50c72366aa57b0d6f6a405c9d

Kasper D. Hansen (16:07:28) (in thread): > But with a different name. That’s kind of confusing to newbies like me

Kasper D. Hansen (16:09:17) (in thread): > Kind of weird with docker run > > % docker run --platform amd64 -it --user rstudio bioconductor/bioconductor_docker:devel bash > > Unable to find image 'bioconductor/bioconductor_docker:devel' locally > devel: Pulling from bioconductor/bioconductor_docker > Digest: sha256:77f56e9960ff09c5d913b9e9a8ba93c56148348f2cf5265e7fac30055905219a > Status: Image is up to date for bioconductor/bioconductor_docker:devel > docker: Error response from daemon: No such image: bioconductor/bioconductor_docker:devel. > See 'docker run --help'. > > % docker run -it --user rstudio bioconductor/bioconductor_docker:devel bash > > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested >

Kasper D. Hansen (16:10:06) (in thread): > Basically, if I give--platformit cannot find the image. If I don’t give the platform it finds the image, start and complains about architecture mismatch

Kasper D. Hansen (16:39:51) (in thread): > If I don’t use--platformI can install R packages, build and check package sources

Kasper D. Hansen (16:39:54) (in thread): > Thanks

2025-03-16

Vince Carey (20:15:09) (in thread): > i think the syntax is--platform=linux/amd64

2025-03-21

Kasper D. Hansen (15:33:32) (in thread): > yes, thanks Vince

Kasper D. Hansen (16:05:53): > (posted deleted; I discovery a typo)

2025-03-26

Alex Mahmoud (10:48:39) (in thread): > Thank you for reporting this@Kasper D. Hansen! Feel free to tag me in the future for any docker issues! Both archs should exist, and they do in theghcr.iorepo (eg:https://github.com/Bioconductor/bioconductor_docker/pkgs/container/bioconductor_docker/379430842?tag=devel). It seems the issue here is with the push to dockerhub failinghttps://github.com/Bioconductor/bioconductor_docker/actions/runs/13998034314/job/39201882225#step:10:170when merging the two archs, so the images seem to be built but only successfully pushed toghcr.io. Looking into it to see why DockerHub is acting up

2025-04-01

Sean Davis (10:14:53) (in thread): > https://github.com/Bioconductor/bioconductor_docker/issues/119 - Attachment: #119 Push of arm64 docker image to dockerhub failing, resulting in missing recent arm64 builds of bioconductor_docker > From interaction with Kasper and @almahmoud on Slack: > > > Both archs should exist, and they do in the ghcr.io repo (eg: https://github.com/Bioconductor/bioconductor_docker/pkgs/container/bioconductor_docker/379430842?tag=devel ). It seems the issue here is with the push to dockerhub failing https://github.com/Bioconductor/bioconductor_docker/actions/runs/13998034314/job/39201882225#step:10:170|https://github.com/Bioconductor/bioconductor_docker/actions/runs/13998034314/job/39201882225#step:10:170 when merging the two archs, so the images seem to be built but only successfully pushed to ghcr.io . Looking into it to see why DockerHub is acting up

Sean Davis (10:16:34) (in thread): > ….slack conversation to github issues.

Alex Mahmoud (10:53:41) (in thread): > Thank you@Sean Davis! I put in some debugging statements yesterday to get to the bottom of it, but was surprised when just adding that made more pass, and then forgot to follow up. I restarted a failed GHA job this morning, and it all passed. It seems to be a timing issue, possibly related to Dockerhub rate limiting… Good news is that it pushed both archs to both repos, so ghcr and dockerhub should be fully in sync now cc@Kasper D. Hansen

2025-04-02

Alex Mahmoud (14:45:12) (in thread): > To close this loop, the issue was likely due to rate limiting by Dockerhub, and a retry was added on the merge/push job that takes the separate arch images and merges them into a single multi-arch tag, and wait+retry seems to have been enough for now to fix the ephemeral issue. > TLDR, problem fixed both temporarily and systematically, and both archs should exist as expected! Thank you all for reporting and sorry it took so long for me to get to it - File (PNG): image.png

2025-04-04

Kasper D. Hansen (10:03:53): > @Alex MahmoudWhich github do you want feature requests for the Bioc docker containers?

Andres Wokaty (13:45:31) (in thread): > It’s likelyhttps://github.com/Bioconductor/bioconductor_docker/issues

2025-04-06

Alex Mahmoud (23:44:16) (in thread): > Yes!

2025-04-15

Rasmus Hindström (07:07:40): > @Rasmus Hindström has joined the channel

2025-04-24

Maria Eleni Fafouti (11:10:49): > @Maria Eleni Fafouti has joined the channel