2017-12-08
Kevin Rue-Albrecht (04:12:45): > @Kevin Rue-Albrecht has joined the channel
Kevin Rue-Albrecht (04:12:46): > set the channel description: Interactive SummarizedExperiment Explorer
Federico Marini (04:12:46): > @Federico Marini has joined the channel
Aaron Lun (04:12:46): > @Aaron Lun has joined the channel
Charlotte Soneson (04:21:22): > @Charlotte Soneson has joined the channel
Aaron Lun (04:58:38): > I’m in.
Kevin Rue-Albrecht (05:03:30): > Cool. I’ll be really busy today (2-day week to get things done over here ^^) > but I also meant to ask: do we have a preferred ‘demo/toy’ data set for the app?
Kevin Rue-Albrecht (05:04:17): > (might wanna create a thread on the above message, to keep the channel as ‘clean and lean’ as possible)
Aaron Lun (05:07:13) (in thread): > Try thescRNAseqdataset, in thescRNAseqpackage? > > library(scRNAseq) > data(allen) > sce <- as(allen, "SingleCellExperiment") >
Kevin Rue-Albrecht (05:28:36): > @Federico Marini: happy to let you handle the backbone-stripping part then. I haven’t started yet, and you’ll be much faster going through your own dashboard:slightly_smiling_face:If I can make a suggestion though, I’d propose the first-commit backbone to basically have a single input (the object), and a single output (summary of the object, such as a show method, maybe cleaned up a little). > Basically, just so that everyone can get familiar with the minimal app structure before it grows too much.
Federico Marini (05:37:36): > I’m in too. Thanks for setting the whole thing up
Federico Marini (05:40:10) (in thread): > @Kevin Rue-AlbrechtYes, I’ll make a basic backbone. Not that much of a reason to make it too complex as a starter. > And yes, I’ll go with a single input i.e. our sce object. Happy to stick to the allen dataset!
Kevin Rue-Albrecht (20:56:48): > @Kevin Rue-Albrechtshared a file:First scaffold in branch kevin_1 - File (Canvas): First scaffold in branch kevin_1
Unknown User (20:56:49): > @Kevin Rue-Albrecht commented on @Kevin Rue-Albrecht’s file https://community-bioc.slack.com/docs/T35G93A5T/F04C1HKH5GD : update - File (Canvas): First scaffold in branch kevin_1
2017-12-09
Kevin Rue-Albrecht (04:38:49): > Hi<!channel>, as a summary of the note above, please feel free to toy with the MWE (minimal working example) in branchkevin_1if you want to warm up you Shiny skills.@Federico Marini, feel free to ignore this stub, if you’d like to take a fresh start having learned from your experience withideal. > I’ll be out most of the day today, and I’m just as happy to come back to a new branch that an extension of this one.
Aaron Lun (11:09:43): > Most of these warnings can be eliminated by usingimportFromin theNAMESPACEto avoid clashes.
Aaron Lun (11:10:03): > Just installed it. It runs, and that’s about it.
Kevin Rue-Albrecht (13:35:27): > Good to know it runs on your system Aaron. > Yeah - I really did mean MWE. Basically it just stores the object in the reactive list and confirms it in the value box. In this way any of us can branch off this commit and prototype a single plot without any superfluous overhead. > > Regarding the imports, 1) I’m aware of the trick, I just haven’t spent the time to pick the individual imports. 2) i notice that roxygen causes packages imported in alphabetical order no matter the order I list them in, which can cause unexpected overrides. Things to bear in mind
2017-12-10
Charlotte Soneson (08:49:50): > It runs also on my machine. I have added .travis.yml files to both the kevin_1 and master branches (the kevin_1 branch passes:+1:) so all future changes should be automatically tested
Kevin Rue-Albrecht (09:55:40): > awesome, thanks!
Kevin Rue-Albrecht (10:01:16): > Federico is a better man than I am: he’s taking a safer start sketching on paper first. Just thought I’d offer a template in the meantime
2017-12-11
Federico Marini (04:36:18): > As Kevin said, I preferred starting sketching on paper. This is also due to purely logistic reasons, i.e. the impossibility to code with 3 kids around in the weekend.
Federico Marini (04:37:12): > I’ll put together all the suggestions/MWE-components in what might be the main branch
Federico Marini (04:37:49): > … and with the first things first, I’ll brutally port thescater_guifunctionality over
Federico Marini (04:38:54): > @Charlotte Soneson, is the .travis config already on? I think only you can switch it on for the first time
Kevin Rue-Albrecht (04:39:15): > it’s on, you can find the report on travis, let me find the link again
Federico Marini (04:40:29): > Cool - i found it
Federico Marini (04:40:30): > https://travis-ci.org/csoneson/SEE
Kevin Rue-Albrecht (04:40:39): > also, you don’t have to port all thescater_guifeature by yourself, once the scaffold (I mean just a blank page with maybe a couple of tabs) is available to all, we should be able to each nest in a separate corner of the app and contribute there
Federico Marini (04:41:01): > I’ll add a badge to the README.md
Kevin Rue-Albrecht (04:41:42): > btw, given that the.travis.ymlfile was added as separate commits to themasterandkevin_1branch, we should be careful is we ever want to merge them (conflict!)
Kevin Rue-Albrecht (04:43:02): > one way around that I believe is agit reverton the commit tokevin_1, followed by agit rebase. This way the.travis.ymlwould be added only once through the commit tomaster
Federico Marini (04:44:28): > ok I’ll keep it in mind. > For the nasty warnings of importing too much, I am a big fan ofcodetoolsBioc
Federico Marini (04:44:55): > -> makes a very nice set of suggestions to keep the import/importFrom fields pretty compact
Kevin Rue-Albrecht (04:47:41): > ok, i did spend a tiny bit of time last night trying to identify whichimportcould be turned intoimportFrom, but if there’s a tool for that, all the better
Federico Marini (04:49:20): > been there, done that:smile:codetoolsBioccan handle it quite neatly
Federico Marini (04:50:09): > it is mostly suggestions, so sometimes can be also wrong-y. but most of the times, it really just picks the correct funcs
Kevin Rue-Albrecht (04:51:06): > ‘Note that codetoolsBioC doesn’t belong to the current release or devel version of Bioconductor anymore.’http://bioconductor.riken.jp/packages/stats/bioc/codetoolsBioC/
Federico Marini (04:53:20): > Yeah, it kind of “never was”. But the sources for that are available on the old svn
Kevin Rue-Albrecht (04:54:36): > ok you mean this link then?http://grokbase.com/t/r/bioc-devel/157rt0kdj4/make-codetoolsbioc-available-on-github “Currently, the codetoolsBioC package is available via:https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/codetoolsBioC ” > - Henrik Bengtsson[Bioc-devel] Make codetoolsBioC available on GitHub? - Grokbase > (2 replies) Currently, the codetoolsBioC package is available via: https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/codetoolsBioC Can codetoolsBioC be mirrored on GitHub like the other BioC packages, or is that only possible for built and distributed BioC package? Thanks, Henrik
Federico Marini (04:55:53): > exactly. svn to get it, build it, install it, and that’s it. no fancyness/commodity of biocLite…but still working
Kevin Rue-Albrecht (04:56:05): > ok great, thanks
Kevin Rue-Albrecht (04:58:58): > Separate point: > thanks to my PI (Steve Sansom) I discovered the ‘protected branch’ feature of GitHub:https://help.github.com/articles/defining-the-mergeability-of-pull-requests/ @Charlotte Soneson: Shall we set that up on themasterbranch to have a safety net? and also encourage developing on secondary branches? > Even as a lone developer on several projects, I’ve messed up mymasterbranch more than once - Attachment (help.github.com): Defining the mergeability of pull requests - User Documentation > You can require pull requests to pass a set of checks before they can be merged. For example, you can block pull requests that don’t pass status checks. You can also require that pull requests have at least one approved review before they can be merged. …
Aaron Lun (10:10:13): > Wow, I see everyone’s been busy. Here’s an example of something one of my colleagues created:http://marionilab.cruk.cam.ac.uk/organogenesis/
Aaron Lun (10:11:16): > And another one:
Aaron Lun (10:11:17): > http://marionilab.cruk.cam.ac.uk/mammaryGland/
Kevin Rue-Albrecht (10:32:13): > Cool, thanks for sharing, nice ideas in both cases
Kevin Rue-Albrecht (10:44:34): > I wish I had a link to share, but I couldn’t even find a (public) GitHub repo: last week someone in Oxford presented an R ‘SC analysis toolkit’ with a minimalistic Shiny app that does a job similar toideal, in the meaning that he’s trying to offer an interactive interface to take a user through a workflow, (rather than visualise precomputed result, which I think is a more efficient strategy).
Kevin Rue-Albrecht (10:56:16): > (nothing new btw, just a wrapper around scran/scater/Seurat functions for people who like GUIs)
Federico Marini (11:06:24): > @Aaron LunI can’t really try out the two apps you linked - the apps start and then crash (?) right after that
Kevin Rue-Albrecht (11:09:03): > that’s weird. I just clicked both links again, and they both work for me
Federico Marini (11:11:13): > thoroughly tested i.e. clicked and reclicked on safari, firefox and chrome:stuck_out_tongue:
Kevin Rue-Albrecht (11:12:21): > I can only vouch for Safari:sweat_smile:
Aaron Lun (11:14:42): > I can vouch for Firfefox
Aaron Lun (11:14:58): > Maybe it’s brexit
Federico Marini (11:15:20): > mos def:smile:
Federico Marini (11:33:33): > <!channel>: So, I just uploaded the first complete backbone thingy. > Few things: > - i already added a few of the things we will anyway need for the bioc submission > - I kept it as simple as it was wished to give everyone a first toy to play around with > - the author list is merely sorted on alphabetical order as of now, I did not want to throw coins now:smile:- some parts in the code are relatively heavily commented, so that everyone can have best readability of what happens/should happen
Federico Marini (11:34:38): > I’ll try to check aaron’s apps as soon as I get a VPN into some uk ip addresses:sweat_smile:
Federico Marini (11:35:33): > I don’t know whether/when we should change the repo name, after converging on theiSEE, but this is minor, I’d say
Kevin Rue-Albrecht (11:36:59): > true, although i’d say as long as theDESCRIPTIONand other files have the right name, the repo name can be anything for the time being
Kevin Rue-Albrecht (11:42:59): > can I just ask again for some opinions about makingmastera ‘protected-branch’ ? - Attachment: Attachment > Separate point: > thanks to my PI (Steve Sansom) I discovered the ‘protected branch’ feature of GitHub: > https://help.github.com/articles/defining-the-mergeability-of-pull-requests/ > @Charlotte Soneson: Shall we set that up on the master branch to have a safety net? and also encourage developing on secondary branches? > Even as a lone developer on several projects, I’ve messed up my master branch more than once
Federico Marini (11:47:02): > I have no direct experience with that regard
Federico Marini (11:47:31): > so I’ll let the other chime in
Aaron Lun (11:48:54): > I don’t really know either.
Aaron Lun (11:49:18): > I only messed up my master once, and that was a lesson I only needed to learn once.
Kevin Rue-Albrecht (11:52:25): > right, ok. > I mean we’re all adults and responsible here so it may not be critical if we’re all careful about pulling before pushing or rebasing where applicable, but in super brief the key features I’m aware of are: > * ensure that all required CI tests are passing before collaborators can make changes to a protected branch. > * branches can also be protected by requiring pull requests to have at least one approved review before they can be merged.
Federico Marini (12:04:56): > FYI, passing on travis check:thumbsup:
Charlotte Soneson (12:21:04): > Sorry for being late to the game, I was teaching constantly since 9 this morning (will be the same tomorrow). I have never tried the protected branch, but I guess it wouldn’t hurt to ensure that CI tests are passing before accepting a push?
Kevin Rue-Albrecht (12:35:23): > I do like the idea of a safety net (probably just the CI tests, one approved review sounds over the top). > I’ll propose the following: how about we activate the feature for a test run, and if it proves too restrictive, we disable it ?
Charlotte Soneson (12:36:04): > sounds good to me
Aaron Lun (12:36:47): > YEAH
Kevin Rue-Albrecht (12:37:20): > I appreciate your enthusiasm Aaron:joy:
Federico Marini (12:46:25): > I’m in - I would anyway wait for the travisCI log right after the commit:smirk_cat:
Charlotte Soneson (12:51:15): > As far as I can see, enabling branch protection means that you can’t push directly to the master branch: “When enabled, commits must first be pushed to another branch, then merged or pushed directly to master after status checks have passed.”
Kevin Rue-Albrecht (13:02:00): > yep, I somewhat hinted to that in my original message, but I should have been clearer i guess. > It’s a direct consequence of requiring CI test to passbefore allowing a merge to the protected branch > this way only validated code can make it tomaster
Kevin Rue-Albrecht (13:03:34): > also, even we’re only 2-3 active developers on the project, I’m worried about all of us working directly on the same branch
Charlotte Soneson (13:03:45): > ah right, sorry, didn’t get that from your comment
Kevin Rue-Albrecht (13:06:18): > I think it makes sense having a separate branch for each new feature, so that they can be tested independently of each other during development
Charlotte Soneson (13:07:01): > yes, seems reasonable. I just activated the branch protection
Kevin Rue-Albrecht (13:11:09): > ok great. > so just to conclude, from hereon,mastershould be viewed as a ‘pull-only’ branch > before starting to develop a new feature, each of us should: > > git checkout master > git pull > git checkout -b feature_or_fix_name > git commit -m 'blah blah' > git push origin feature_or_fix_name > > and open a pull request on that branch
Aaron Lun (13:12:26): > Interesting. Sort of like a compromise between full access and an external PR, then.
Charlotte Soneson (13:13:14): > @Charlotte Soneson pinned a message to this channel.
Kevin Rue-Albrecht (13:13:27): > Yep, with the CI acting as a mediator/reviewer/failsafe/i’m out of synonyms
Kevin Rue-Albrecht (15:27:02): > btw, the second most common scenario when working with multiple branches is that someone else merges their changes intomasterbetween the moment one of us branches, and the moment they want to merge too. > In that case, each of us should > > git checkout master > git pull > git checkout feature_or_fix_name > git rebase master > [resolve potential conflicts locally] > [git commit -m 'blah blah if there was any conflict'] > git push origin feature_or_fix_name > > and open a PR as above. > Seehttps://git-scm.com/docs/git-rebase for pretty explanatory pictures > Disclaimer: this is all off the top of my head. Please check a Git bible in case of doubt:grin:
2017-12-12
Federico Marini (12:03:36): > @Aaron LunYesterday from home I was able to check briefly the apps you linked. As you said, the functionality is somewhat basic, but in this scRNA-seq world many things can be potentially interesting “only by checking”
Federico Marini (12:04:37): > Shall we define some priorities here and now?
Federico Marini (12:04:59): > One thing would be to replicate the funcs of scater_gui in a nice way
Federico Marini (12:05:44): > Another thing, we can brutally port some code from the good ol’ ideal for the annotation import, which could be done via org.XX.eg.db packages
Federico Marini (12:08:40): > I’m happy to hear out your ideas/top things
Federico Marini (12:09:07): > … of course, the logo for the tool should be taking place 1
Federico Marini (12:09:15): > :stuck_out_tongue:
Kevin Rue-Albrecht (12:10:29): > when you mentioned sketching this weekend, I was actually wondering whether there’s any place online that we could sketch collaboratively, maybe a Google doc spreadsheet kinda thing?
Kevin Rue-Albrecht (12:11:29): > to clarify, by sketching, I mean a preview of what individual panels would look like
Federico Marini (12:13:28): > no experience on that, but would be quite cool
Kevin Rue-Albrecht (12:16:28): > hm.. Google seems to only offer Docs (Word), Sheets (Excel) or Slides (Powerpoint) > the latter could be used at a small scale, but I’m not convinced > from a quick google search, I just spotted this one that might worth a try:https://sketchboard.io (comes with Slack integration, too) - Attachment (sketchboard.io): Sketchboard.io: Online Sketch Diagramming Whiteboard for Teams > Sketch diagrams on an infinite whiteboard together with your team. Now with Slack integration.
Federico Marini (12:18:59): > This looks sleek
Federico Marini (12:19:11): > I’d say we can give it a try
Federico Marini (12:19:14): > ?
Kevin Rue-Albrecht (12:21:31) (in thread): > I definitely would like to see some annotation integration in the app. Not sure where in the priority list it falls though. > Also, I’d love to see theSCEobject declare in its metadata the annotation package(s) that the app should load, to minimise the actions to take in the app
Kevin Rue-Albrecht (12:22:53): > i’m on it (creating board and inviting by email)
Aaron Lun (12:24:17): > sounds good to me. When do you want to do this? I don’t wake up before 11 am on weekends.
Federico Marini (12:30:17): > -> I sketched the logo to call the day:smiley:
Kevin Rue-Albrecht (12:30:50): > hahaha you were so fast xD
Federico Marini (12:31:05): > @Aaron Lunwe can start just throwing the idea in this channel, and with those then sketch it out?
Kevin Rue-Albrecht (12:31:09): > it took me forever to figure out how to delete my dummy example items
Federico Marini (12:35:58) (in thread): > Ok, I’ll see how feasible/improvable it would be with the annotation
Kevin Rue-Albrecht (12:39:13) (in thread): > hang on, i’m sure i’ve seenExpressionSetobjects for microarray using that strategy
Kevin Rue-Albrecht (12:39:17) (in thread): > just looking it up
Aaron Lun (12:42:22): > I must admit I’m finding this a bit difficult to use. Maybe I’ll just watch.
Aaron Lun (12:43:22): > Too much muscle memory from Inkscape.
Kevin Rue-Albrecht (12:43:50): > hahaha, bit struggling here too, although i do like the panel on the right with lots of little predefined shapes
Kevin Rue-Albrecht (12:44:30) (in thread): > https://www.bioconductor.org/packages/3.7/bioc/vignettes/Biobase/inst/doc/ExpressionSetIntroduction.pdf
Kevin Rue-Albrecht (12:44:44) (in thread): > check out section 4.3 “Annotations and feature data”
Kevin Rue-Albrecht (12:45:58) (in thread): > i’ve seen that@annotationslot being used by GSEA packages, avoiding the need for users to specify the annotation source/version in every function call
Federico Marini (12:46:18): > Ok even if not 100% useful, we know this thing exists:smirk:
Federico Marini (12:48:56) (in thread): > I think it was much more relevant back in the microarray data - I probably never used the slot directly
Kevin Rue-Albrecht (12:49:17): > on the top priorities, I’d place: > 1) clean NAMESPACE imports to avoid warning messages currently thrown byR CMD checkandlibrary(iSEE)(I’m on it)
Federico Marini (12:49:36) (in thread): > what I meant was just a basic functionality to have for example ensemblIDs and then operate with gene symbols
Federico Marini (12:49:53) (in thread): > although, not everytime the 1:1 matching is perfectly there
Kevin Rue-Albrecht (12:50:03): > 2) more generally, polish the current MWE to start off a clean base
Federico Marini (12:50:41) (in thread): > But in any case, let’s just stick this to the potential todo list:wink:
Kevin Rue-Albrecht (12:50:49): > 3) suggestscater::calculateQCmetricsbeing run on theallenobject before passing it to the app
Kevin Rue-Albrecht (12:51:32): > 4) experiment with a couple of (linked?) interactive figures using the QC metrics from (3)
Kevin Rue-Albrecht (12:53:43): > 5) brainstorm general/advanced app settings that be given toiSEE(), but also controlled from widgets in accordingly named app panel(s), so that users may update them from within the app (e.g. ,colours=list(condition=named_character_vector1, donor=named_character_vector2)
2017-12-13
Aaron Lun (12:46:42): > I was going to suggest alphabetical ordering for the names, but I see you’ve already done it.
Aaron Lun (12:46:48): > Based on the first names, of course.
Aaron Lun (12:47:11): > Totally fair and unbiased.
Aaron Lun (12:47:17): > Last names would also work for the time being.
Aaron Lun (12:47:32): > But anyway, the current MWE works on my computer.
Aaron Lun (12:48:04): > suggest usingif (!interactive()) { iSEE(sce) }in the?iSEEpage.
2017-12-15
Charlotte Soneson (07:46:05): > I managed to completely miss all these messages on Tuesday, sorry! For discussion, some things that a user might want to be able to do/set/choose, either when starting the app or from within it (many of which are also in scater_gui): > - which dimension reduction to display (alternatively make one tab for each method that is included in the object, but that might clutter things too much) > - what to color the cells by in the dimension reduction plot (can allow all the columns of colData(sce) + the expression of individual genes, perhaps in different panels?) > - heatmaps of one or more genes, clustering cells or ordering by some annotation > - similarly, boxplots/violinplots of gene expression > - should it be possible to doany
Federico Marini (07:53:33): > @Charlotte Sonesonregarding the calculateQCmetrics part, how long does it take? On smaller datasets, as well as on bigger ones? > I was thinking to have a valueBox for the QC metrics - if not there, red, and you can do that. If there, green, and browsable in a tag. > Would that fit your expectation? That would also be my line - PCA and tSNE are beyond the wall:smile:
Aaron Lun (07:55:04): > I don’t like the idea of re-runningcalculateQCMetricsin the app.
Charlotte Soneson (07:57:02): > Yes, I agree that the dimension reductions need to be done in advance. I’m not totally against runningcalculateQCMetricsin the app, mostly since it would just allow the user to add more “annotations” to the cells, that can be used to color various plots.
Charlotte Soneson (07:58:56): > I’m also wondering whether one could add (and export) new annotations, e.g. by selecting cells in a plot. This could then be used to color or order points in another panel.
Aaron Lun (08:01:56): > I think it would make our lives easier for the user to just runcalculateQCMetricsoutside. I don’t see the advantage in running it inside - worst case, if the user forgets, they can just close the app, runcalculateQCMetricson theSingleCellExperimentobject, and open the app again.
Charlotte Soneson (08:06:17): > Fair enough. So maybe let’s start simple and just allow the user to use the columns available incolData(sce).
Aaron Lun (08:08:53): > The main advantage would be if we wanted to compute particular statistics on the fly, but that’s a few steps down the line.
Aaron Lun (08:09:14): > And usingcalculateQCMetricsto compute specific statistics would be like hitting a fly with a sledgehammer.
Kevin Rue-Albrecht (08:27:50): > As Aaron made the point, I would really rather have as much as possible computed prior to launching the app, so that the app itself primarily focuses on fetching/binding the available pieces of data together for display. At least for a start.
Federico Marini (08:31:24): > Fair enough:wink:
Kevin Rue-Albrecht (08:34:04): > re: dimension reduction, why not just have a dropdown menu in ‘general settings’ to select any one to use throughout the app at any single point in time, rather than having multiple tabs (indeed cluttering things). That’s how I see the purpose of the @reduceDim slot to be honest
Aaron Lun (08:54:26): > Yes.
Aaron Lun (16:22:35): > Hey, did you guys know that you can storeDataFrames…within other DataFrames?https://github.com/davismcc/scater/issues/43 - Attachment (GitHub): Tucking away scater’s QC stats · Issue #43 · davismcc/scater > Currently calculateQCMetrics loads up the SingleCellExperiment with a lot of baggage: library(scater) example(calculateQCMetrics) colnames(colData(example_sce)) # 37 entries! But we can hide it awa…
Aaron Lun (16:23:08): > I am pushingcalculateQCMetricsto return output like this, as it will avoid the bigcolDatavomit that it usually spits out.
Aaron Lun (16:26:35): > However, it will mean that theiSEEinterface will be somewhat more complex, as one will need to interrogate variables in nestedDataFrames.
2017-12-16
Kevin Rue-Albrecht (05:20:09): > oh right, i feel like i distractedly noticed this ‘inception’ feature ofDataFramebeing used by some packages but i’ve never directly made use of it myself. > Indeed, I guess this feature could emulate the ‘paging’ feature that was discussed at the EuroBioc meeting, and -as your GitHub issue hints to- maybe even subdivide QC metrics for the different sets of control features/samples (e.g.mitochondrial,ERCC,blank, …).
Kevin Rue-Albrecht (05:25:52): > At which point, then I guess the app would only need the user to specify which of the ‘top-level’ columns in thecolData``DataFramecontains the scater QC metrics. From that point, the app should be able to navigate to the systematically organised lower-levels to fetch any individual piece of information
Aaron Lun (08:12:32): > It should actually be fairly easy, all you have to do is to have a drop-down list of plottable variables that looks like this:
Condition
Batch
scater_stats: total_features
scater_stats: total_counts
...
So the user can then pick what things they want to plot.
Aaron Lun (10:18:24): > How’s everyone’s inkscape? I will draw out what I was thinking of on inkscape and put it on dropbox.
Aaron Lun (10:39:52): > Ah, that got boring quickly. I”ll just commit some code, then.
Kevin Rue-Albrecht (10:51:51): > haha sorry - got dropped an unrealistic deadline for next week that’s taking me away from all the fun I could have here this weekend:persevere:
Kevin Rue-Albrecht (10:54:07): > i haven’t much experience with Inkscape, but it’s never too late to learn. I’m usually using Illustrator when I need to, but I wouldn’t call myself an expert there either.
Aaron Lun (11:44:58): > I have added proof-of-concept support for variable numbers of reduced dimension scatter plots. The idea is to allow users to add more plots as they wish. Ideally, each plot will have its own options specifying which reduced dimension set to use, the components to plot. Graphical parameters (colouring, etc.) are best left as global, though we could add options to have local graphical parameters as well.
Charlotte Soneson (12:05:44): > Nice. Actually, wouldn’t one common use case would be to have multiple instances of the same reduced dimension plot, but color according to different variables? Like, sample ID and cluster assignment.
Kevin Rue-Albrecht (12:17:12): > i agree that having multiple instances of the same redDim plot with different colouring factors is more attractive than a single instance that users have to switch between the different colours. > While faceting a single figure over the selected colouring factors is undoubtedly the simplest thing to implement, I am worried about the time to display/refresh the figure for large data sets. > A more painful, yet feasible (I believe) implementation would be to let users add new (independent) figure from within the app. Possibly an application of ‘Shiny modules’
2017-12-17
Aaron Lun (10:27:58): > Done. Also, we should change the repository name, “SEE” just does not show up as the first few hits in my browser, I always have to type the full URL.
Aaron Lun (10:30:03): > Currently there is a little delay as all the plots are effectively remade from scratch when the reactives update. This could be improved, probably by moving the parameter memory store out of the reactive list and just modifying it upon construction of the plot.
Aaron Lun (10:49:16): > … which is now done. I suggest running: > > library(iSEE) > example(iSEE, ask=FALSE) > > to gethttps://www.youtube.com/watch?v=z1DzZ7oShl4
Federico Marini (14:53:22): > Well done@Aaron Lun!
Charlotte Soneson (15:30:06): > Cool! I just added a first attempt on violin plots for individual genes.
Charlotte Soneson (15:31:14): > I can change the name of the repository to iSEE (or did you have anything else in mind@Aaron Lun?
2017-12-18
Aaron Lun (04:37:45): > Yes, name change would be great
Charlotte Soneson (04:42:04): > ok, I renamed the GitHub repo to iSEE
Federico Marini (05:00:29): > @Aaron Lunit’s your destiny to be involved in pkgs with puns in it I guess
Federico Marini (05:00:58): > Surprisingly csaw SEE
Aaron Lun (05:02:16): > it’s a gift
Federico Marini (05:02:24): > what is next?:smile:
Federico Marini (05:02:58): > dammit, you could have insisted on something like the latinveni, vidi, vici
Federico Marini (05:03:44): > but I can anticipate that iCame as a name would have some googling issues…
Federico Marini (05:20:10) (in thread): > do we need to change the remote for our local repo or is it automagically redirected?
Charlotte Soneson (05:21:42) (in thread): > Itshould be redirected according tohttps://help.github.com/articles/renaming-a-repository/ , but they strongly recommend changing the remote - Attachment (help.github.com): Renaming a repository - User Documentation > You can rename a repository if you’re either an organization owner or have admin permissions for the repository. …
Federico Marini (05:26:29) (in thread): > ok, just to be safe I changed it
Federico Marini (05:26:53) (in thread): > whenever a new feature is implemented, I can check if everything works:wink:
Charlotte Soneson (05:28:53) (in thread): > :+1:
Aaron Lun (06:01:16): > Just took the violin plots to the next level. There’s an argument for using some of thescaterplotting functions to avoid rewriting all the various bits and pieces: we could discuss this with@Davis McCarthyto make any necessary improvements/generalizations to the interface.
Aaron Lun (06:01:49): > Still waiting for the damn CI integration thing before i can merge the branch. What the hell does “Waiting for status to be reported” mean?
Aaron Lun (06:04:19): > Probably because we renamed the repo. Any thoughts?
Charlotte Soneson (06:10:02): > Looks like the build passed on travis (8 min ago). I just synced the travis account, don’t know if that changes anything
Charlotte Soneson (06:14:51): > Now i havecsoneson/iSEEin the list of repos on travis. Can you try again?
Aaron Lun (06:15:58): > There we go, it’s in progress now.
Charlotte Soneson (06:16:19): > Right, sorry about that. Apparently it didn’t sync with travis automatically
Aaron Lun (07:01:30): > I would be nice to get autocomplete on those text boxes, but I don’t know how to do it.
Charlotte Soneson (07:02:22): > I think it would work withselectInputinstead oftextInput
Charlotte Soneson (07:06:22): > Like the last example here:https://shiny.rstudio.com/gallery/selectize-vs-select.html
Charlotte Soneson (07:32:10): > A couple of other suggestions for the violin plots, that I can fix unless anyone objects: > - the plots look weird if the selected x variable is numeric (the violin is put in the middle of the x range). Perhaps the violin should be removed in this case. > - the alignment of the radio buttons is off if the browser is not wide enough. We could remove inline = TRUE, which would always align them vertically > - allow plotting other expression values than logcounts to make it more broadly applicable
Aaron Lun (07:34:33): > selectInputchokes for me with >10000 possible entries.
Federico Marini (07:38:57): > @Aaron LunIt can be a case where the selectInput needs to be run with updating the selectize widget
Federico Marini (07:39:54): > I had this code snippet in theidealmain function, and I recall it helped speeding up the part to populate the entries:
Federico Marini (07:40:21): > > # this trick speeds up the populating of the select(ize) input widgets, > # see[http://stackoverflow.com/questions/38438920/shiny-selectinput-very-slow-on-larger-data-15-000-entries-in-browser](http://stackoverflow.com/questions/38438920/shiny-selectinput-very-slow-on-larger-data-15-000-entries-in-browser)observe({ > updateSelectizeInput(session = session, inputId = 'avail_ids', choices = c(Choose = '', rownames(values$res_obj)), server = TRUE) > }) >
Charlotte Soneson (07:50:40): > Even without that, it works fine for me with 18,000 values in another app
Aaron Lun (07:51:09): > Huh.
Aaron Lun (07:51:22): > Well, okay, knock yourselves out.
Charlotte Soneson (08:47:33): > Convertinggene.namesto adata.tableinstead of a character vector seems to make it a bit faster (roughly from ~19 to ~7s to initialize a new plot).
Federico Marini (09:15:25): > I have a small stub with some gene info/annotation, but I cannot get it to be included in each of the expression value plots. Maybe it is just trivial and I was unable to get the thing done o_O
Federico Marini (09:15:57): > I’ll push soon to thegene_info_anno_boxbranch
Federico Marini (09:16:20): > and have it in a box which just sticks below the latest plot
Federico Marini (09:23:23): > maybe we could be safe in adding@import shinycompletely in the namespace
Aaron Lun (09:24:48): > shrug . Don’t really like ROxgyen anyway, so do as you please.
Federico Marini (09:25:03): > :stuck_out_tongue:
Federico Marini (09:37:59): > anyway. the push to the branch is done, now merging the pull request after CI checks.@Aaron Lun, would you be able to check if the principle behind the multi-plots also applies for thishtmlOutputobject?
Federico Marini (09:38:53): > -> to test it, select mouse as species in the box on the left (org.Mm.eg.db would be required), and then type e.g. Actb as a gene
Aaron Lun (11:31:57): > Currently playing around with it.
Federico Marini (11:32:05): > Ok, I finally got the mechanism - which is, I have to admit, simple yet powerful
Federico Marini (11:32:37): > sorry for doubling/multiplying the effort
Aaron Lun (11:34:20): > The info box is very cute but I’m not sure it belongs where it currently is.
Federico Marini (11:35:01): > I know. Now it sticks to each of the plot units
Federico Marini (11:35:16): > in the last column on the right
Federico Marini (11:35:41): > feel free to move it around:wink:
Aaron Lun (11:38:07): > Well, it’s a more fundamental issue about how the interface is used in this tab. For these particular scatter plots, there is little “gene discovery”, as the user has to specify a gene name in order to get the plots in the first place. This suggests that they have to know what genes they’re interested in, which reduces the benefit of the explanatory information. This box is best suited to situations where I ask the interface, “Give me the top X genes by variability in a heatmap”, and I then go and explore the identity of those genes. Probably in a different tab.
Federico Marini (11:41:13): > True. I was thinking of a use case when some user checks this bunch of genes, and then later reports back to the PI with a quick but content-full short summary (imagining the PI/Postdoc is not up to date with the whole gene info), plus the possibility to link directly to NCBI for extra db-infos
Aaron Lun (11:44:16): > I think this would be a powerful feature for any plot where the genes are the dots. Its position in the current tab is a bit odd to me. (“Of course I know what Nanog is! Don’t need some stupid computer reminding me of that.” - PI)
Aaron Lun (11:46:33): > There is also the additional aesthetic requirement of keeping each the space for each plot relatively lean, if we are to support multiple plots. Otherwise there’s a lot of scrolling to do, and it starts to look cluttered.@Charlotte Sonesonthis was, in part, why I hadinline=TRUE; I’m fine with changing that if the horizontal doesn’t work, but if the number of options gets any higher we should consider moving the per-plot parameter options into those hidden panels that open up when you click them. Don’t know how to do that, though.
Federico Marini (11:47:54): > I agree - especially for the “no need to tell me what Nanog is”
Federico Marini (11:48:30): > For the hidden panels part, I used inidealthe collpasible panels from shinyBS
Federico Marini (11:48:35): > could be one way to go
Aaron Lun (11:48:59): > Happy for you to give it a shot.
Federico Marini (11:49:24): > from the time I implemented that, it could be that some better ways came along. nevertheless, collapsible panels worked quite nicely
Federico Marini (11:49:35): > I’ll see what I can do:wink:
Aaron Lun (11:50:30): > If that works, we can go crazy with the plot parameters, e.g., supportsize_by,shape_by, etc.
Kevin Rue-Albrecht (11:53:54): > wow. glad to see the project getting to that level of activity! can’t wait to join back in the fun of implementation
Federico Marini (11:56:24): > Ok:slightly_smiling_face:I had some time to invest and be up to date with the whole new features that came!
Federico Marini (11:58:01): > We can move/remodel the output where it is required - indeed, inidealI have it exactly in a situation after clicking with an MAplot
Federico Marini (11:58:04): > https://youtu.be/EckimPqRcCg?t=168 - Attachment (YouTube): Demo for pcaExplorer and ideal
Federico Marini (11:58:58): > One thing that could be a nice support for us all, and not just for this app
Federico Marini (11:59:16): > I recall I used once a pkg to check the structure of an app
Federico Marini (11:59:41): > it was not perfect and it enforced the ui.R/server.R split
Federico Marini (12:00:14): > still.. ShinyTest was quite a nice thing:wink:
Federico Marini (12:00:16): > http://amitkohli.com/announcing-shinytester-a-package-that-helps-you-build-shiny-apps/ - Attachment (AmitKohli.com): Announcing ShinyTester – a package that helps you build Shiny apps > Shiny is awesome, but can be a bit daunting and easy to make mistakes in. I recently came back to Shiny after a hiatus of a few years and it was much more challenging than I feel comfortable admitt…
Federico Marini (12:00:29): > also on CRAN
Federico Marini (12:18:44): > @Aaron Lunone thing I do not get right away: what is the difference in the 2 inputsGene Expression:andX-axis gene expression:?
Aaron Lun (12:51:51): > You can plot one gene against the other. The first one could be called “Y-axis”, if it helps.
2017-12-19
Aaron Lun (04:59:31): > Moved the info box to a better place, with DT::datatable for gene exploration. I would suggest having theorg.*.eg.dbspecification as a function argument toiSEE, as this should be fixed for the entire dataset; in 99% of cases, it doesn’t make sense for the user to change it on the fly. (And in some cases, it wouldn’t make sense at all if the features are not genes.)
Kevin Rue-Albrecht (05:22:07): > Hi@Aaron LunCheck out section 4.3 “Annotations and feature data” > How about encouraging an element$annotationin the@metadataslot ofSingleCellExperimentBased on the linked documentation it could be a character string of length 1, with the name of theorg*.dbpackage - Attachment: Attachment > https://www.bioconductor.org/packages/3.7/bioc/vignettes/Biobase/inst/doc/ExpressionSetIntroduction.pdf
Kevin Rue-Albrecht (05:23:46): > (although, yes, it doesn’t hurt to have an argument toiSEEable to specify/override it on the fly)
Aaron Lun (05:31:12): > Not sure how much help it is to have a dedicated annotation field inSingleCellExperiment. The benefit would only be apparent if many downstream functions need to draw from a specifiedorg.*.eg.db, but this isn’t usually the case with my workflows. I assign all my annotation at the start, and that’s that. TheSingleCellExperiment maintainers would also be reluctant to impose more gene-based structures on the container, as they want it to be general to other types of single-cell data (e.g., methylation, for which anorg.*.eg.dbannotation doesn’t make sense). It would be more transparent and less work foriSEEto accept anorg.*.eg.dbobject directly.
Aaron Lun (05:34:18): > The function would also need to accept thekeytypeand some indication of whichrowDatafield contains the keys (or the row names, ifNULL). This reflects the fact that I usually need to mangle my gene symbols to guarantee uniqueness/non-missingness.
Kevin Rue-Albrecht (05:37:58): > yep, makes sense - i just realise now that it only really makes sense for microarrays which, once designed, can be linked to annotations for the type of features that they measure, sequencing experiments are more diverse in this regard
Federico Marini (07:31:46): > @Kevin Rue-AlbrechtYes, back then it was a welcome feature.@Aaron LunThank you for re-organizing the structure. I’ll soon look into the collapsibles/some better option, and I am also pro having the org.XX.eg.db package as a parameter.
Federico Marini (07:33:51): > I wonder how complex it would be to have some kind of heuristics for autodetection of species and ID type. Not in the scope of the app, but just out of curiosity (anyone of you already tried out?)
Kevin Rue-Albrecht (11:19:16): > yep i did, although it was my first R/Bioconductor package (GOexpress), so there’s a lot i’d like to do differently now
Kevin Rue-Albrecht (11:21:31): > however, the more I look back, the more I think it’s overcomplicating a problem that is much more easily resolved by asking the user to supply any sort of identifier for the annotation resource/dataset to use
Kevin Rue-Albrecht (11:22:39): > you can check out helper functions in this script:https://github.com/kevinrue/GOexpress/blob/master/R/toolkit.R - Attachment (GitHub): kevinrue/GOexpress > GOexpress - Analysies of gene expression data using Gene Ontology
Federico Marini (11:24:42): > Ok, then most likely an overkill:slightly_smiling_face:Good to know, in the end we canreally
Kevin Rue-Albrecht (11:29:46): > Yep, my current view is that it’s a fair request to expect users to know what features are measured in their data set is about, and as a consequence which annotations best describe them.
Kevin Rue-Albrecht (11:32:11): > As a more ‘manual’ input, I’d say it is also reasonable to accept the GTF file that was used to quantitate features, provided the user also declares which field was used as feature identifier
Federico Marini (11:37:10): > hm, gtf tend to be hug-ish - compared to the compact annotation pkgs. Now they even could useAnnotationHub
Kevin Rue-Albrecht (11:46:56): > nice littleBiocFileCachewith that:slightly_smiling_face:
Federico Marini (12:50:50): > @Aaron LunI implemented the collapsible panels, for both redDim and geneExpr
Federico Marini (12:51:01): > Separate branch is collapse_params
Federico Marini (12:51:11): > which soon should be merged into master
Federico Marini (12:54:40): > … ok, now onmaster. For the design part: > - I left the panels open by default, and they can be closed. We can always change that > - for the gene expr: the y-axis gene is left out as it is the main, mandatory one > - hope you don’t mind I put some FA-icons and coloured it up in a suggestive way
Aaron Lun (13:52:18): > Deep in a brutal battle withtestthat, will have a look at the PR once I’m done.
Aaron Lun (15:21:28): > @Federico Marini
Aaron Lun (15:21:34): > oops
Aaron Lun (15:21:43): > Anyway, it looks good, except for a few things.
Aaron Lun (15:23:05): > The first is that, for some reason, runninglibrary(iSEE); example(iSEE, ask=FALSE)on my computer doesnot produce the green box at the top, nor the first reduced dimension plot. It is only after I dolibrary(shinyBS); example(iSEE, ask=FALSE)that everything starts working properly. Perhaps have a look into that, maybe there’s a missing import.
Aaron Lun (15:24:47): > The second is that the collapsible fields reset to their original open/closed state whenever I change the number of plots. I presume that the open-ness of the fields is stored somewhere in theinputfor each collapsible field? This should be added to the memory (reddim_plot_param, I think it’s called) to remember which fields are open and closed, so that this is preserved upon changing the number of plots.
Aaron Lun (15:25:43): > Finally, I think only the graphical parameters need to go into the collapsible fields - the type of plot (PCA, etc.) is critical to its interpretation and so should be left outside - or at least, in a separate collapsible field.
Federico Marini (15:31:36): > @Aaron Lun1 - good catch. the behavior is somewhat strange. It can be it is one of those packages that kick in when explicitly loaded > 2 - that thing I did notice. I’ll look into it further. My aim was to get a version for you all to see and provide feedback - mission accomplished also with… > 3 - good point. I did not want to mess around too much:slightly_smiling_face:
Federico Marini (15:32:53): > One interesting feature: can we project the expression of more than one gene on the redDim plots? I am thinking of a use case with e.g. bunch of markers that behave similarly
Federico Marini (16:06:11): > for point 1, I recalled correctly: seehttps://ebailey78.github.io/shinyBS/install.html#using_shinybs
Federico Marini (16:06:56): > the problem is that if we put the call tolibraryin the pkg, a warning is raised. AndrequireNamespaceis “not enough”
Federico Marini (16:11:57): > if there’s no other way, I’ll see some other options
2017-12-20
Federico Marini (03:18:31): > point 1: it runs fine by just adding shinyBS to the depends - as an alternative I was thinking ofshinydashboard::box. Let’s see which one is better fit for point 2
Federico Marini (03:56:01): > for point 2, I was not able to work out a satisfying solution for all panels, somehow the mechanisms behindupdateCollapseare not so optimally matching the way we are planning to store the parameters. A workaround would be to trigger the updates with the pressing of the new plot button but I am still not fully happy (as I said, I could not find a way yet to set/get the open panels for each without complicating it too much). We could collapse all the previous panels except the latest plot, but that is also not perfect. Would that sound reasonable to you?
Charlotte Soneson (04:48:55): > I don’t have much experience with collapsible panels, but can we give eachbsCollapsePanela (unique)valuein addition to thetitle, and then store somewhere for eachvaluewhether it is open or closed, and set the status correctly in theupdateCollapse?
Charlotte Soneson (04:50:34): > Unrelated, I just added the option to change the assay type in the violin plots.
Federico Marini (04:54:52) (in thread): > I would need to check that, maybe later today. Seems reasonable, but I could not get something like this working right away
Federico Marini (05:01:49) (in thread): > I do not know how to get the attribute open/closed for each, e.g. after we opened/closed manually
Federico Marini (05:02:06) (in thread): > If someone has some time to look into this, I’d be grateful:slightly_smiling_face:
Charlotte Soneson (05:56:56) (in thread): > seems that if you do e.g.input$collapse_redDimPlots1, it will return all the open panels in thisbsCollapsegroup (NULL if there are no open panels)
Charlotte Soneson (06:02:58) (in thread): > I may be on the way to a solution, but have to go to a meeting. Will continue later unless someone else solves it before:slightly_smiling_face:.
Federico Marini (06:11:42): > Following up on my own suggestion: a cool feature would be to have the following: > - autocompletion via selectizeInput/updateSelectize for selecting the gene name > - if one gene is selected, just like as it is now in the redDim plots > - if more than one, do something like the coloring is mirroring e.g. the max of each of the selected genes (say, known markers for cell type) > - and of course, have evtl. more plots on the same page > > If time is on my side, I’ll try to make up a minimal example in another tab, to avoid chaos in the redDims - Attachment: Attachment > One interesting feature: can we project the expression of more than one gene on the redDim plots? I am thinking of a use case with e.g. bunch of markers that behave similarly
Federico Marini (07:42:00): > -> very raw, but to see it in action: first implementation is inhttps://github.com/csoneson/iSEE/tree/multigene_reddims - Attachment (GitHub): csoneson/iSEE > Contribute to iSEE development by creating an account on GitHub.
Federico Marini (07:42:44): > (for trying it out, I kept a bunch of genes, and added housekeeping genes & Ctcf)
Aaron Lun (08:05:35) (in thread): > Looking at this now.
Aaron Lun (08:06:53): > I’m not sure what it means to colour by multiple markers. It seems much more intuitive to (i) use multiple plots, or (ii) ask the user to define some sort of aggregation score that can be used for coloured.
Charlotte Soneson (08:24:09) (in thread): > Ok. I just returned from my meeting, let me know if you want me to commit what I have. Basically, adding avalueto thebsCollapsePaneland checking whetherinput[[paste0("collapse_redDimPlots", arg)]]is NULL or not for each plot in the plot addition seems to go at least part of the way.
Aaron Lun (08:27:47) (in thread): > Give me a half an hour to test out what I’ve got.
Aaron Lun (08:42:13) (in thread): > Okay, done.
Aaron Lun (08:42:57) (in thread): > I have a solution using ourpObjectsmemory rather thanupdate*, which provides a smoother experience (asupdatewill initialize it open and the close it, which is a bit weird).
Aaron Lun (08:46:10): > Currently going through a major reorganization of the various string constants in the code, to avoid things breaking when we decide to rename something; I suggest no one touch anything for a while.
Federico Marini (08:49:03): > @Aaron Lunmessage received:wink:
Federico Marini (08:50:53): > For the multiple markers, i was thinking of the fact that some cell identities might be better defined by the ensemble of expression values for some known markers. > In the end, it can also be that this possibility of choosing >1 gene is default, and an extra parameter could be the function to apply to the values per sample (as of now,max) - Attachment: Attachment > I’m not sure what it means to colour by multiple markers. It seems much more intuitive to (i) use multiple plots, or (ii) ask the user to define some sort of aggregation score that can be used for coloured.
Federico Marini (08:51:53): > It also depends somehow how much we might want to mimic the functionality available in the 10x CellLoupe
Federico Marini (08:52:46) (in thread): > @Aaron Lunand@Charlotte Soneson: thank you for picking up on this!
Aaron Lun (08:53:17): > what do they do?
Federico Marini (08:55:35): > https://support.10xgenomics.com/single-cell-gene-expression/software/visualization/latest/tutorial-celltypes
Federico Marini (08:56:11): > –> > > With CD79A and CD79B in the list and 'Gene Exp Max' as the selected attribute, the coloring of the plot represents the maximum transcript count between CD79A and CD79B in each cell. Selecting a single gene within a list shows the expression level for that single gene. >
Federico Marini (08:57:32): > so… pretty much in line with my brutal draft
Aaron Lun (08:57:55): > I don’t think that’s necessary when we have multiple plots.
Aaron Lun (08:58:25): > It seems like they painted themselves into a corner with only one plot.
Aaron Lun (08:58:37): > And they try to get out of it by allowing some crazy colouring
Charlotte Soneson (08:58:40): > It would also somehow assume that expression values are comparable between genes, which may be the case for 10X tag counts, but not in general.
Aaron Lun (08:58:54): > Good point
Federico Marini (08:58:57): > True@Charlotte Soneson
Federico Marini (08:59:46): > Ok - we can still save something with theselectizeInputthat basically supports autocompletion, which can be very useful all around
Federico Marini (09:00:30): > (I don’t know why theupdateSelectizedid not work correctly, therefore the selection restricted to just 1k genes)
Aaron Lun (09:48:30): > Almost done, waiting for CI. I will work on the interface next to allow people to pass defaults in (e.g., if you’re setting up a shiny server for other people to look at your data).
Federico Marini (09:52:22): > Great work:wink:
Aaron Lun (11:46:31): > still going with the interface, stand by.
Aaron Lun (12:02:27): > Just CHECKing locally on my machine before pushing, stand by.
Aaron Lun (13:02:01): > Currently pushed. For those who are interested: all initial parameters can now be set by passing relevant inputs to theiSEEfunction, with parameter defaults described indefaults.Rd. This means that users can write scripts to directly initialize the interface with the plots of interest, especially useful for Shiny servers. > Internally, all server-related constants are now stored inconstants.R. This probably doesn’t help code readibility but is especially useful if you are to change their values, e.g., for more intuitive naming of user-supplied default arguments. The presence of a central list of constants avoids the need to hunt down every instance of the constant in the package to avoid crashing the interface. Aesthetic constants (e.g., text that shows up for the user’s viewing pleasure) are left as they are, as this matters less.
Aaron Lun (13:03:02): > @Federico MariniIt’s a shame thatshinyBS needs to beDepended. Is this fundamental to its use or is the author just unaware of theImportmechanism?
Federico Marini (13:10:14): > @Aaron Lunyep, I thought the same. But let’s put it like this, it was probably developed with shiny apps in mind that do not reside in a package i.e. have to suffer painfully if the package is not just imported. I don’t know what would be feasible to change the behavior
Federico Marini (13:12:34): > Seeing the version history of the project here…https://ebailey78.github.io/shinyBS/changes.html
Federico Marini (13:12:54): > it does seem not so currently developed
Federico Marini (13:12:58): > pity indeed
Aaron Lun (13:13:50): > Might be worth asking, given it’s CRAN policy to favour Imports over Depends. It might be as easy as them exporting something that was missing. In any case - do we have any alternatives?
Federico Marini (13:16:13): > I am aware of the following: > -boxfromshinydashboard
Federico Marini (13:16:55):
box(..., title = NULL, footer = NULL, status = NULL,
solidHeader = FALSE, background = NULL, width = 6, height = NULL,
collapsible = FALSE, collapsed = FALSE)
Federico Marini (13:17:47): > -> probably fitting the usage we plan with the update of params
Federico Marini (13:17:51): > or…
Federico Marini (13:18:27): > -bsplus
Federico Marini (13:18:30): > http://ijlyttle.github.io/bsplus/articles/overview.html?utm_content=bufferc1240&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer#collapse
Federico Marini (13:19:27): > -> this one seems more complicated to insert in our setting (seems to be tightly linked to buttons?)
Federico Marini (13:20:13): > if some of you have other pieces of knowledge I did not report, please feel free to complete my answer
Aaron Lun (13:30:50): > Thanks.boxlooks like the way to go, then.
Aaron Lun (13:31:00): > Currently a few bugs in themaster, fixing now.
Aaron Lun (13:42:06): > Okay, it’s all good to go. That’s me done for the week, knock yourselves out.
Aaron Lun (13:46:07): > The major remaining to-do is how to handle point selection and brushing, especially when we have lots of points.
Federico Marini (13:51:09): > where do we mainly want the brushing and the interactions?
Federico Marini (13:52:26): > and on top of this: do we want to do something when cluster-like info is available?
Aaron Lun (13:54:05): > Brushing and interactions should be possible for all cell-based plots. For example, I could select some cells that are outliers in PCA, and see how they behave with respect to particular genes. (I should also add some plots for plottingcolDataagainst each other - that’s another thing on the to-do list). Conversely, I might select a cell that has a very large library size and ask where it appears on the PCA plots.
Aaron Lun (13:55:35): > Regarding cluster information - I don’t think it’s necessary if it doesn’t require different treatment from other types of row/colData. Or to put it another way - what kind of stuff that we can show for cluster data, and only cluster data?
Federico Marini (13:57:45): > I was again looking at the CellLoupe tool
Federico Marini (13:59:29): > I don’t know how generalisable and meaningful it would be to do this pseudoDE-analysis, in terms of aggregating in silico the counts/expr values of genes by clusters, e.g. defined via kmeans on the tSNE plot (seems something commonly done, but I’m quite stranger to what is standard, what is extended analyses)
Aaron Lun (14:01:57): > I suspect cluster-specific averages should be computed by the user ahead of time, in which case they would show up in the datatable tab; but in any case, the violin plots would be more informative anyway.
Federico Marini (14:24:30): > Ok I’ll think of something > In the meanwhile, we can fix/complete tiny things as we run up into them
Federico Marini (14:26:05): > @Aaron LunI don’t know how far the goal line would be from the current standpoint. I would say we ended up doing quite most of the main stuff already - and as a nice side effect, I learned a lot alone from the nice trick of multiplot updates
Aaron Lun (14:26:28): > Two major things: brushing and the log of commands
Aaron Lun (14:26:44): > If you report a log, people can just copy-paste the R code equivalent to make the plots they want.
Aaron Lun (14:26:51): > That would be the hope, at least.
Kevin Rue-Albrecht (14:27:26): > sorry that I dropped out of the conversation for a while.@Aaron Lunyou mentioned a to-do list, is there a shared document somewhere where we can keep track of those, and pick up items to implement?
Aaron Lun (14:27:49): > I guess the github wiki would be a good place? Never really used it, though.
Charlotte Soneson (14:28:12): > What about using GitHub issues?
Kevin Rue-Albrecht (14:29:33): > I like the issues indeed, there are tags to mark ‘enhancements’ (which I would use to describe new features). > I’ve used the wiki feature in the past, which is basically managed as an independent repository. But I found that place more useful for static information (e.g. tutorial)
Aaron Lun (14:30:19): > Okay, we can do that.
Federico Marini (14:32:27): > Issues seem the best place. Might even be that some good soul apart from us picks up on some open ones:smile:
Charlotte Soneson (14:33:01): > Or provides additional ones:wink:
Federico Marini (14:33:18): > yeah, or that too:stuck_out_tongue:
Federico Marini (14:49:14): > About the brushing: the linked plot(s?) would be then positioned close to each of the multiplots, right?
Federico Marini (14:50:23): > Example, a heatmap with most variable genes, with clustering enforced on the rows but the columns rather ordered by “selected VS unselected cells”
Federico Marini (14:50:53): > Or… one separate panel, where just one main plot is available
Federico Marini (14:51:12): > I am trying to picture out how it would look like in the end
Federico Marini (14:52:07): > Maybe one panel where all these plots linked to each other would be a tidy thing (tidy not in the Wickham vision)
Aaron Lun (14:53:34): > Let’s not think about heatmaps for the time being. But for brushing, I envision that everything would literally be linked - you could brush on one plot and see the results on any cell-based plot in any of the tabs. This should be very possible, the question is how reactive it should be. Full reactivity would be too slow for large numbers of plots; I am more inclined to have a button on the left with “highlight selected” that a user can press once they’ve gotten the points they want. Such a button could also be plot-specific, making use of another collapsible box.
Aaron Lun (14:54:28): > You could imagine “gating” on a CD4-high population (or indeed, a double-positive population for two markers of interest) using the gene expression plots, and then brushing to see where they show up on the t-SNE. This would provide an intuitive alternative to Loupe’s craazy Max-of-two-values thing.
Aaron Lun (15:02:41): > Okay, some github issues up. The final thing before I retire for the night is for you guys to ponder; do we want to reusescater ’s plotting functions, or do we want to write our own? The former is easier but the latter will definitely give us more customizability - particularly as Davis is rather intermittent at updatingscater , and there’s at least a number of things that are suboptimal for our use cases. However, if you want to do the latter, then someone else will have to take charge of the plotting, because I hate ggplot.
Federico Marini (15:12:10): > Last round for me tonight too: > - reactivity & brushing: I need to revisit what is actually allowed by the shiny framework. I have experience in dragging & clicking, plus interacting with a DT datatable > - for the repro: so far we always save basically all the parameters. It sounds feasible to somehow extract these info back from where we store it, and generate the code, “tab by tab”, and place it in an instance of ashinyAceeditor. > - for the plots. This is somehow related to the brushing, too. ggplot objects are damn easy to insert in the whole reactivity-framework, whereas base objects are trickier. That at least was the last time I checked:stuck_out_tongue:
Federico Marini (15:14:40): > uh, and of course, potential GH issue: > - I can look out how to do the mini-introjs-based tours, if the feature is still wanted. It is already pretty self-explanatory, but some pointers for the first usage can always be welcome
Kevin Rue-Albrecht (15:15:58): > yep - big fan ofggplot2here as well. happy to help with writing our own plotting functions. > even when usingscaterat the command line, i often end up overriding thescale_*which raises a few warning. Not the end of the world, but it would probably be cleaner to implement the individual layers in the app itself, rather than adding them on top of theggplotobjects returned byscater
2017-12-21
Kevin Rue-Albrecht (04:26:29): > @Kevin Rue-Albrechtshared a file:Failed install of rjson package - File (Canvas): Failed install of rjson package
Unknown User (04:26:29): > @Kevin Rue-Albrecht commented on @Kevin Rue-Albrecht’s file https://community-bioc.slack.com/docs/T35G93A5T/F04CS4Z17TJ : Sorry to bother you guys with that, but i can’t find a post online that describes the same issue. Did anyone run into problems installing rjson as a dependency to iSEE ? - File (Canvas): Failed install of rjson package
Kevin Rue-Albrecht (04:27:33): > My laptop installed it fine so I’ll be able to work on there. But the iMac at work fails with the error paste in the post above
Charlotte Soneson (04:38:52) (in thread): > It works for me. I’m always puzzled by these errors so this is just a random guess, but are you by chance using homebrew? I think this can cause problems like this (e.g.https://github.com/igraph/rigraph/issues/191 ) - Attachment (GitHub): Fail to compile with R 3.4.0 (homebrew) under Mac OS · Issue #191 · igraph/rigraph > From the console: installing to /usr/local/lib/R/3.4/site-library/igraph/libs **** R **** demo **** inst **** preparing package for lazy loading **** help ***** installing help indices **** building package indi…
Federico Marini (04:40:58) (in thread): > Same as for Charlotte, but as she said, brew could be the one to blame?
Federico Marini (04:41:28) (in thread): > Sometimes brew is the blessing, sometimes the curse.
Kevin Rue-Albrecht (04:41:32) (in thread): > yep - i’m using homebrew despite having had some bad experiences with it before
Charlotte Soneson (04:42:14) (in thread): > then it’s perhaps worth checking the paths to gcc/clang
Aaron Lun (04:42:26) (in thread): > It’s the old C++11 problem. I have experienced something similar withbeachmat , seehttps://support.bioconductor.org/p/104005/ .
Federico Marini (04:43:42) (in thread): > I recall having nasty experiences back when I wanted to install the primordial versions of the IRkernel, based on the zeromq libraries
Kevin Rue-Albrecht (04:46:04) (in thread): > it may be related, but I ended up installing R-devel from the .pkg installer because when I installed from source, the new R executable was not detected by RStudio, although the terminal had no problem finding it (https://community.rstudio.com/t/rstudio-does-not-seem-to-scan-standard-locations-as-declared/3494/3 ) - Attachment (RStudio Community): RStudio does not seem to scan standard locations as declared > Thanks for your reply. I should have said, I also checked all the paths that RStudio is supposed to scan, and /usr/bin/R doesn’t exist. Thanks for the note about ‘open’ Looking forward to further advice. Kévin Envoyé de mon iPhone
Federico Marini (04:46:51) (in thread): > Uh, R-devel is for me always via .pkg - at least on mac
Kevin Rue-Albrecht (04:47:30) (in thread): > Thanks@Charlotte Soneson, indeed that could also be a reason, as homebrew symlinked g++ and gcc, I’m losing track of which executable is actually installed from where -_-
Federico Marini (04:47:39) (in thread): > It ended up being with less pain then the source
Kevin Rue-Albrecht (04:47:55) (in thread): > I know the feeling Federico ^^
Federico Marini (04:49:15) (in thread): > I wish I could be an expert on this and tell you the solution right away… Could have saved me some major time in other occasions
Aaron Lun (04:52:17) (in thread): > Ah, you mac people.
Kevin Rue-Albrecht (05:17:10) (in thread): > solved (self-slap)
Kevin Rue-Albrecht (05:17:29) (in thread): > > #CXX=/usr/local/bin/gcc-7 > CXX=/usr/local/bin/g++-7 >
Charlotte Soneson (05:17:37) (in thread): > :+1:
Federico Marini (05:17:55) (in thread): > cool!
Kevin Rue-Albrecht (05:17:58) (in thread): > I just switched from the first to the second in~/.R/Makevars
Federico Marini (05:18:12) (in thread): > (not that you self-slap yourself :P)
Federico Marini (05:18:33) (in thread): > good to keep in mind, it can be a next failure reason
Kevin Rue-Albrecht (05:20:03) (in thread): > btw, Fede,R Under development (unstable) (2017-12-20 r73933) -- "Unsuffered Consequences"just compiled from source without incident for me (as opposed to another nightly build a few weeks ago). > I guess it’s some (R)ussian (R)oulette situation:wink:
Federico Marini (05:22:03) (in thread): > I love tonot
Federico Marini (05:22:06) (in thread): > :smile:
Aaron Lun (09:22:14): > I will tackle the brushing on the weekend. I would suggest splitting off the plotting functions into a separate file, where each plot function returns the ggplot object and the call used to generate it. This should reduce the size ofiSEE.R, insomuch as this is possible.
Kevin Rue-Albrecht (09:28:10): > on a bug-fix note: > is it desirable that the ‘gene expression’ box is a free text field, rather than a dropdown? > is this a consequene of@Aaron Lunpointing out issues with dropdown that had too many values?
Aaron Lun (09:28:54) (in thread): > I do wish for a better way to do this. Yes, it would be nice to get some autocompletes here, but a several second delay in loading is pretty brutal.
Kevin Rue-Albrecht (09:30:31) (in thread): > ok, no worries > i can add avalidate(need(...))statement to produce a friendlier message than the redsubscript out of boundscurrently displayed if the gene does not exist
Aaron Lun (09:31:27) (in thread): > Possibly it could also say “do you mean these genes…”
Kevin Rue-Albrecht (09:32:41) (in thread): > I’m almost done with my local duties (two different sets of collaborators want plots for manuscripts to submit over Christmas). I really look forward coding a bit with you guys again
Kevin Rue-Albrecht (09:33:23) (in thread): > I had a feature like that suggestion in myGOexpress. Let me find it again
Kevin Rue-Albrecht (09:33:50) (in thread): > here:https://github.com/kevinrue/GOexpress/blob/master/R/post_analysis.R#L61 - Attachment (GitHub): kevinrue/GOexpress > GOexpress - Analysies of gene expression data using Gene Ontology
Kevin Rue-Albrecht (09:37:18) (in thread): > anyway. don’t worry about it, it’s a small thing and I’m happy to take care of it in the next couple of days. it’ll let me get up to speed with the recent code udpates, and let you focus on the brushing
Aaron Lun (15:15:54): > I recently realized that our current interface is a bit clunky for what people want to do. In particular, I was looking at one of my colleague’s apps (http://marionilab.cruk.cam.ac.uk/organogenesis/ ); to get the same behaviour iniSEE, the user needs to click through three sets of tabs, which is three too many. Thus, I’ve dropped my weekend brushing plans to instead reorganize the interface so that everything is on one page, while still being fully customisable with regards to adding new plots, etc. I won’t do anything tomorrow because I need to go yell at my bank, but you can expect the interface to go through a major overhaul on the weekend. If someone can getshinydashboard::boxto work, that would be great.
Kevin Rue-Albrecht (15:46:15): > ok, i agree that the interface needs a bit of thinking. > i’ll push some code in the next hour where i’ve addressed the ‘invalid gene name’ issue, just cleaning up now
Aaron Lun (15:56:45): > No rush, I’m too tired to do anything oniSEEtoday.
Kevin Rue-Albrecht (15:57:31): > note that for the time being, i’ve put the pattern matching code iniSEE.Ritself, but i also agree with you that we should think twice whether some code can be extracted into separate files. > This is a particularly good example
Aaron Lun (15:57:51): > Yeah, it’s a real shame that shiny code tends to result in a huge file. This makes it difficult for collaborative projects like this, especially given that the weekend will just be taken up by my refactoring.
Aaron Lun (15:58:22): > The refactoring will likely touch everything iniSEE.R, so it’ll be hard for others to work on this without resulting in a giant heap of merge conflicts on Monday.
Kevin Rue-Albrecht (16:00:24): > well, i understand Federico’s original point that putting ui and server in the same file allows sharing some internal variables. > InTVTB, i’ve adopted the intuitively ‘cleaner’ approach of splitting my code intoui.Randserver.R, but that forced me to use some tricks to work around limitation
Kevin Rue-Albrecht (16:24:29): > Yep, i’ll be flying and seeing family so good timing to leave you refactoring stuff. > To follow up on the code extraction, i’m keen to extract the code in a separate.findGene(x, object, where)helper function > Not a top priority (please let me know if you do like the idea though), but in addition to the current behaviour (if gene doesn’t exist, suggest regex matches), that would also give more space to allow a ‘search path’ to scan for the input gene name. > What I’m thinking here iswhere=c("gene_name", "SYMBOL")being a character vector of the field names to scan, akin to a$PATH: if an exact match for the query gene name is not found in therownames, then for example search next inrowData(se)[,"gene_name"], then mayberowData(se)[,"SYMBOL"]. > If none of the fields have an exact match, then fall back on suggesting close matches. > Just to tell you that I’m aware of caveats: anything else thanrownamesmay contain duplicate values. I won’t bother you with details, but I’m thinking of that case too.
Federico Marini (19:22:56) (in thread): > My take on this:selectizeInput, coupled with a updateSelectize call in the server function should work. Importantly, the parameterserveris set toTRUE
Federico Marini (19:23:18) (in thread): > I have a similar implementation inideal
Federico Marini (19:23:31) (in thread): > where the gene list is in the side panel
Federico Marini (19:23:47) (in thread): > inspiration for the solution was taken from here:https://stackoverflow.com/questions/38438920/shiny-selectinput-very-slow-on-larger-data-15-000-entries-in-browser - Attachment (stackoverflow.com): Shiny selectInput very slow on larger data (~15,000 entries) in browser > I have this basic shiny app, and it is blazing fast in the ‘Viewer’, but when I use the ‘Open in Browser’ option, the select input choices take a while to load. selectList <- sapply(1:15000, fu…
2017-12-22
Aaron Lun (10:04:16): > Well, my bank yelling trip has been delayed until tomorrow, so I’ll start fiddling with stuff today instead.
Aaron Lun (10:04:45): > Actually, scratch that, I’ll just do it on Sunday.
Aaron Lun (17:18:34): > I have begun the overhaul; you can see a minimally working example oninterface_overhaul. Note that for the next stage, any of the gene choices in any the plots can be dynamically linked to anydatatablein any other panel, thereby solving our gene selection problem.
Aaron Lun (17:19:53): > This also demonstrates an actual use for the side bar, in keeping track of dynamic addition of panels. The next stage will be to allow for the reordering and resizing of particular panels.
2017-12-23
Aaron Lun (11:38:34): > I wonder whether we’re making life difficult for ourselves by allowing users to dynamically change the number of plots. It would be much easier to have the number of plots fixed upon launching the app (though obviously it can still be altered upon launch, by altering the arguments to theiSEEfunction). This would allow us to avoid the memory store - in particular,datatablewill always forget what was selected, meaning that anyone adding a new plot will wipe out their prior selections in any table panel.
Aaron Lun (11:52:41): > Scratch that, I figured out how to trigger re-creation of thedatatable. It doesn’t save the search, but at least the selected row is unchanged; this avoids triggering recreation of all linked scatter plots.
Aaron Lun (12:00:55): > And now that, too, is fixed. There is a little flash with thedatatableas it re-renders, but I guess that’s okay if people aren’t adding plots all the time.
2017-12-24
Kevin Rue-Albrecht (11:09:40): > Looking forward to the result of the pull request. > To be honest, I am (also, if that’s your point) still wondering whether a dynamic number of plot is more attractive/useful than a well-organised layout with a fixed number of plots.
Kevin Rue-Albrecht (11:16:45): > Anyway. In the meantime: merry Christmas and a Happy new cell!
Aaron Lun (11:17:04): > MY VISION IS COMPLETE
Aaron Lun (11:17:16): > though the brushing is not fully implemented yet.
Kevin Rue-Albrecht (11:23:16): > I just pulled the master branch
Kevin Rue-Albrecht (11:24:02): > Let me just say::astonished::+1:
Aaron Lun (11:30:13): > Needs some serious aestheticization, but the underlying Shiny machinery is all there.
Kevin Rue-Albrecht (11:32:48): > Exactly.
Kevin Rue-Albrecht (11:33:06): > I really like the link with the gene table too
2017-12-25
Aaron Lun (08:01:21): > @Federico MariniI figured out theshinyBS Dependsproblem. InshinyBS/R/misc.R, there is an.onAttachcall; this should be changed to.onLoadto allow people toImportthe package.
Aaron Lun (08:05:55): > Unfortunately, looking at the github issues suggests that the maintainer is no longer responsive. Never mind commits, he/she isn’t even responding to issues.
Aaron Lun (11:12:07): > Well, anyway, all the underlying Shiny machinery is now sorted out. Now we just need people to make it pretty.
Federico Marini (15:02:15) (in thread): > Thank you for checking it out
Federico Marini (15:36:38): > Just peeking in the master since I am quite caught between relatives, kids, and more
Federico Marini (15:37:17): > I have to say I was skeptical at the beginning with the brushing-all option, but hey, this is super neat
Federico Marini (15:37:30): > Excellent work
Federico Marini (15:38:31): > Very nice look:slightly_smiling_face:
Federico Marini (15:39:44): > … and very nice way to say “you all have a merry christmas time” with code:wink:
2017-12-27
Aaron Lun (13:33:41) (in thread): > I’ve put in our own.onLoadto avoid having toDependsonshinyBS . Unfortunately,shinydashboard::boxdidn’t work out because it doesn’t seem to report the open/closed status. Well, not easily, anyway.
2017-12-28
Federico Marini (08:41:38) (in thread): > Uh ok, then, we justImportit. Too bad theboxdid not work out, we would have stick to a pkg we already had in
2018-01-03
Aaron Lun (12:59:58): > <!channel>To get people motivated on this as they come back from hols; I’m thinking that this can be a short F1000Research paper; aiming to submit at the middle of Feb, depending on how much progress we make.
2018-01-04
Charlotte Soneson (08:22:38): > Sounds reasonable to me. Would it be worth having a quick call when everyone is back to plan/split the remaining work, or is Slack enough?
Aaron Lun (08:56:29): > Yes, a quick Skype would be useful.
2018-01-05
Aaron Lun (12:44:25): > Are we all back at work yet?
Federico Marini (15:36:55): > Not yet. (Nicely) lost in snow in the swiss mountains, I’ll be back on Jan 9th
2018-01-06
Aaron Lun (14:27:19): > Okay, what about@Kevin Rue-Albrecht?
Aaron Lun (14:27:30): > I’m thinking of a skype on the 9th.
Kevin Rue-Albrecht (15:45:11): > Hi all, > Yes I am back. Sorry I didn’t open Slack yet (yay! for email notifications). > Happy to Skype to discuss and coordinate. However I’m at a Symposium in town all Tuesday 9th. > If that’s still the best day for everyone, I could still sneak out to take the call with you guys.
2018-01-07
Charlotte Soneson (04:54:44): > As for me, we can do another day than the 9th. I have some meetings planned, but there is at least some time free each day the coming week.
Aaron Lun (06:54:41): > Well, that’s good, because on reflection, I also might have a meeting on the 9th as well. What about… the 10th?
Kevin Rue-Albrecht (07:05:00): > Can we do late afternoon on the 10th? (≥3pm UK time) > I have two meetings of uncertain duration earlier that day (manuscript editing and lab meeting). > Alternatively, 11:30-12:30 (UK time) could work for a sort of lunch call.
Aaron Lun (07:11:14): > Fine with me.
Federico Marini (07:12:19): > 10th is fine also for me. both slots are good
Kevin Rue-Albrecht (07:14:07): > late afternoon could turn into hackathon…:smirk:
Federico Marini (07:15:05): > oh well. in that case it is dangerous:smile:
Charlotte Soneson (08:05:10): > Lunchtime on the 10th doesn’t work for me, but 4pm Swiss time is fine (until ~4:50 when I have to start my commute home:slightly_smiling_face:)
2018-01-08
Aaron Lun (09:46:00): > what’s the time shift with UK? I guess 4 pm there is 3pm here?
Charlotte Soneson (09:59:07): > I think so, that’s how I derived 4pm (since Kevin was available after 3pm UK time).
Aaron Lun (09:59:35): > Okay, 3pm Wednesday UK time works for me.
Kevin Rue-Albrecht (12:43:49): > Any thoughts whether reordering the panels while retaining the widget states is possible at all in the current setting? > I’m thinking about a multiple selection widget where the user can reorder the index of the current panels, and then click a button« go! »
Aaron Lun (12:50:51): > Not sure what you mean - I mean, you can already reorder the panels, and the states should mostly be retained.
Aaron Lun (12:51:23): > <!channel>And is everyone good for 3 pm UK? I don’t have Bluejeans so I can only hope that Skype can handle a four-way call.
Charlotte Soneson (13:02:17): > 3pm UK is fine with me. I’ve done more than 4-way Skype recently, should be fine.
Kevin Rue-Albrecht (13:06:40): > You can reorder them ? It must be a simple thing I haven’t spotted yet then
Aaron Lun (13:11:52): > Yep - that’s what the up/down buttons on the left should be for.
Kevin Rue-Albrecht (13:12:13): > Argh:expressionless:
Kevin Rue-Albrecht (14:09:16): > sorry, i just reopened the app now and finally paid proper attention to the left side panel, beyond the orange ‘remove’ button.
2018-01-09
Federico Marini (04:42:16): > 3pm UK time (a.k.a. 4pm Mainz time) is perfect. Skype is also ok!
2018-01-10
Federico Marini (08:41:38): > Hi everyone, in preparation to the conference call, my username in skype is fede.maro
Charlotte Soneson (08:42:04): > Mine is lottlotta
Aaron Lun (09:17:02): > aaronlun110
Federico Marini (09:45:32): > got the two of you
Kevin Rue-Albrecht (09:45:43): > kal.sunrider
Federico Marini (09:45:45): > only kevin is missing - but I see he’s typing
Federico Marini (09:45:54): > speaking of the devil:slightly_smiling_face:
Kevin Rue-Albrecht (09:46:00): > just got out of a meeting with PI ^^
Aaron Lun (09:50:31): > give me 10 minutes, still banging my head on the wall about something.
Federico Marini (11:00:16): > I should register the trademark for the orchestra plot
Federico Marini (11:00:27): > I see many coming in the next future
Federico Marini (11:00:31): > :slightly_smiling_face:
Federico Marini (11:01:01): > Anyway, as we just said, let’s keep a thread pinned with potentially relevant links/resources for everyone
Federico Marini (11:01:15): > Relevant resources/links here:
Federico Marini (11:01:26): > @Federico Marini pinned a message to this channel.
Federico Marini (11:02:22) (in thread): > https://rstudio.github.io/shinytest/articles/shinytest.html -> unit testing in shiny
2018-01-11
Federico Marini (05:45:57): > So, I am putting together something in thetrack_reprobranch. For now I have a placeholder with the readout of the current active plots, which works (thanks to the nice infrastructure,@Aaron Lun). > More work will follow to actually grab the correct code. > As of now, a proof of principle that you can copy the code is also there. I think it should work on any system (I mainly have macs only, so maybe Aaron can have a quick run and just click, and then ctrl-V somewhere else?)
Federico Marini (05:47:39): > And for the time being, a temporary patch on the up/down/trash buttons would look like empty-text buttons, close to each other instead of vertically aligned
Federico Marini (05:48:23): > maybe if we change the background we do not have to go down the rabbit hole of javascript or whatever other option?
Federico Marini (05:58:39): > p.s. nice to see travisCI machines again running for us:slightly_smiling_face:
Aaron Lun (11:02:35): > I’ll have a look at this tomorrow or the weekend, currently in another debugging battle.
2018-01-12
Aaron Lun (06:52:20) (in thread): > It works but it requiresxclipas a system install, which is not a great idea. Instead of copying to clipboard, can we do something like open a new tab or browser containing all the text?
Aaron Lun (06:56:17) (in thread): > A couple of other suggestions; usepObjectsrather thanrObjects, as the former is not reactive (which is more appropriate for holding code chunks).
Aaron Lun (06:57:21) (in thread): > I wonder whether we can obtain the call to create a ggplot object from the ggplot object itself? This would make life much more convenient.
Federico Marini (08:12:20) (in thread): > About xclip as a requirement: we can have the fallback to open a popup or anything similar where text is included and can be copied. I liked the idea of least number of clicks/actions to get the code
Federico Marini (08:13:36) (in thread): > I do not know whether ggplot objects store the call. Maybe@Kevin Rue-Albrechtand@Charlotte Sonesonknow more?
Charlotte Soneson (08:14:53) (in thread): > I don’t think they store the call (at least they didn’t before). There is a lot of information that perhaps can be used to recreate it (facetting, colors, data etc is all there), but I don’t know if this is more complicated than just regenerating it from the code we (will) write ourselves in the app
Federico Marini (08:17:55) (in thread): > Probably from the scratch will be anyway not so much more complex
Charlotte Soneson (08:19:21) (in thread): > Agree. I think we discussed to have the plotting code in functions, in that case it should just be to extract that code (otherwise we must remember to change the exported code if we change the code internally)
Federico Marini (08:25:05) (in thread): > I’ll start with the aim of getting a working version, then we can refine it with code/functions - although we probably all agree that 3 functions will just be more elegant as wrappers?
Charlotte Soneson (08:27:36) (in thread): > What about trying this:https://github.com/jonocarroll/ggghost - Attachment (GitHub): jonocarroll/ggghost > ggghost - :ghost: Capture the spirit of your ggplot call
Federico Marini (08:31:19) (in thread): > I see it is on cran already
Federico Marini (08:31:50) (in thread): > Ok, I’ll keep it in mind
Federico Marini (08:32:45) (in thread): > De facto we just need something that captures the input$ objects, passes them cleverly as parameters of the functions, and that “would be it”, with a nice oversimplification:slightly_smiling_face:
Aaron Lun (08:41:55): > I’m dealing with the colouring gene text box right now, no one touchdynamicUI.Rorconstants.R.
Kevin Rue-Albrecht (08:42:35) (in thread): > I agree with Charlotte: ggplot stores the data and the aesthetics information, but not the call per se
Charlotte Soneson (09:10:48) (in thread): > Perhaps another option would be to generate the plotting code in the plotting function as a string ("ggplot(df, aes(x = x, y = y)) + geom_point()"), and then return the string as well aseval(parse())the string to get the ggplot object?
Aaron Lun (09:11:21) (in thread): > Yes, that would have been my first thought.
Kevin Rue-Albrecht (09:12:19): > are you talking just UI interface, or behaviour as well? I just spotted a crash situation that I’ll add to the issues:
Kevin Rue-Albrecht (09:13:05): > gene expression plot crashes if coloured by gene expression:ENS... not found in colnames(colData(x))
Kevin Rue-Albrecht (09:14:06): > I think that falls within my ggplot updates though, so don’t worry about it if you were not thinking about it anyway
Charlotte Soneson (09:15:39) (in thread): > So maybe that’s the easiest then: each plot is generated by a function that takes the SingleCellExperiment object and the relevant input$ objects as arguments, and returns a string that is evaluated to generate the plot in the app?
Aaron Lun (09:15:49): > Waiting for the CI for a merge, otherwise I am done. Colouring behaviour is unstable for all plots except the reduced dimension ones, becausescater wouldn’t play nice.
Kevin Rue-Albrecht (09:17:07) (in thread): > I thought exactly the same; i just never did it before and hope that the evaluation won’t cause problems with theaes(...)part. > Although we could always useaes_stringin that case
Charlotte Soneson (09:18:24) (in thread): > The code above seems to evaluate correctly in a small example
Kevin Rue-Albrecht (09:19:34): > got it - hopefully that won’t be a problem once I replace the scater plotting functions with local ggplot calls
Kevin Rue-Albrecht (09:22:23) (in thread): > oh right, yep, i got it to run too, nice!
Aaron Lun (09:37:51): > Done on my end, knock yourselves out.
Aaron Lun (09:38:34) (in thread): > I would suggest putting the [code for creating the final output string on the clipboard] into a new file, to avoid cluttering upiSEE.R.
Kevin Rue-Albrecht (09:45:08) (in thread): > Makes sense. I was looking atplotting.Ralready.
Federico Marini (09:46:32) (in thread): > usingcodetrack.Rfor this:wink:
Kevin Rue-Albrecht (09:51:06) (in thread): > Just looking at branches now reminds me that we should all delete the branches after merging them.
Aaron Lun (09:51:32) (in thread): > Yes, yes we should.
Aaron Lun (09:51:55) (in thread): > How about everyone take care of their own branches, so we don’t delete stuff that others might have work on?
Aaron Lun (09:55:05) (in thread): > Just killed a whole bunch. Happily git tells you which branches are not yet merged to the master, so just remember to not push the deletions for the unmerged ones.
Federico Marini (09:55:48) (in thread): > Seems we were still good boys/gals, we did not make that much of a mess
Federico Marini (09:55:50) (in thread): > :stuck_out_tongue:
Aaron Lun (09:56:12) (in thread): > Just remember to keep your active branches merged with master.
Federico Marini (10:15:44): > So, on my side: > I got a way to capture all parameters, which are now printed out as a debugging text box up on the top. I’ll put some thoughts on how to generate the code
Federico Marini (10:16:00): > One possibility is that the plotting functions come wrapped in something like this:
Federico Marini (10:16:02): > https://github.com/gammarama/intRo/blob/ce769b254a3fc1be3f4827bfa0155ad89d4b4e9d/global.R#L30-L40 - Attachment (GitHub): gammarama/intRo > intRo - Shiny-based statistics learning application
Federico Marini (10:16:15): > …or similar
Federico Marini (10:16:53): > (I actually likedggghost, but I can also think the lesser deps, the better)
Federico Marini (10:20:07): > In a similar fashion to extract the params, we can do the same with params & calls, and then we can decorate the code a little - thinking of##### some comments to improve readability
Federico Marini (10:26:35): > now pushed totrack_repro, which might be a little out of sync with themasternow after the gene selection
Federico Marini (10:36:20): > Build fortrack_reprois passing. If you want to give it a run, please go ahead:wink:
Federico Marini (10:37:10): > I did not merge yet on master, I am probably a little KO for now for not risking dumb mistakes
Federico Marini (10:39:10): > … and as of now the text box is quite cluttering the UI - excuse me for this, but it was to have all params under control at once
Federico Marini (10:51:16): > @Kevin Rue-Albrechtand@Charlotte Sonesona.k.a. ggplot-team: > as of now I am tracking only the parameters taken out of the inputs, each according to each plot type. This does not include the latest changes that@Aaron Lunintroduced for the gene text box, but it should be no problem to accommodate these at the end. > Just giving you a heads up if it helps you to design accordingly the plotting functions
Aaron Lun (11:34:31): > master merged back into track_repro.
Aaron Lun (11:45:17): > Having stared at the code tracking for a while, I have a few suggestions: > > - Move the button to the side bar, to avoid taking up plotting space. > - Generally, I don’t think it’s a good idea to have the code chunks on the same page as the plots, this would get unwieldy very quickly when you start having more than 5-6 plots. > - Putting the code chunk into a Shiny tab is the other option. I prefer opening a new actual browser tab, though, as this will avoid cluttering the plotting space.
Federico Marini (13:59:46) (in thread): > -> The button will go to the side bar, good point. > -> As of now I just put a text box to have at once an overview of whatever changes I did, and live see the params change > > For the final version, I was thinking of one of the following (or a mix): > - popup or similar (modal from shinyBS?) > - shinyAce/text box editor in another tab > - plus as a bonus (viaEnhance?) the commodity to get the code copied to clipboard
Federico Marini (14:00:11) (in thread): > BTW, thank you for merging. I’ll edit the names of the fields then
Kevin Rue-Albrecht (14:17:16) (in thread): > I think it should have beengit rebase master track_reproinstead, but maybe it works that way too
Federico Marini (14:24:46) (in thread): > I’m far from being a git pro, so I’m just glad it worked:smile:
Kevin Rue-Albrecht (14:31:02) (in thread): > the way I remind myself is that it makes more sense to “rebase all the side branches onto the master branch” than to “merge the master branch into all the side branches” - i’m not sure what happens when you merge multiple side branches back into the master branch, after each of the side branch has received a merge from the master (argh that’s a pain to type)
Federico Marini (15:22:58): > For the gg-team, as well for Aaron in case he’s curious
Federico Marini (15:23:56): > @Federico Mariniuploaded a file:pop_modal.png - File (PNG): pop_modal.png
Federico Marini (15:24:31): > Here’s a very raw concept realization of the code-capturing popup (bsModal)
Federico Marini (15:24:58): > not perfect yet for the code generation, but still:slightly_smiling_face:
Federico Marini (15:25:48): > I’m just wondering if it makes sense then for you to develop functions that getexactly the param values as we have them in the UI
Federico Marini (15:26:06): > probably the answer is yes, but I’m open to alternatives
Federico Marini (16:21:22): > So, my part up to the function call is up and running in thetrack_reprobranch
Federico Marini (16:24:09): > now relying only on thebsModalcall, and the user can copy and paste it manually
Federico Marini (16:24:46): > (known problem, minor yet: there is an extra NULL coming at the end of the code, I can’t track it down)
2018-01-13
Aaron Lun (08:08:17) (in thread): > I’ve had bad experiences with rebase, so I don’t touch it anymore.
Aaron Lun (10:35:06): > Pretty cute.
Kevin Rue-Albrecht (10:38:41): > I think it’s the return value of theprintfunction
Kevin Rue-Albrecht (10:39:24): > so ifprintis the last command in a function, that function will returnNULL
Kevin Rue-Albrecht (10:41:46): > it’s the sort of annoying issues I used to face withshowmethods
Kevin Rue-Albrecht (11:59:36): > Hi all, > I’ve just pushed an update on branchggplot_coldata. > It builds Charlotte’s update this morning (branchggplot).@Aaron Lun, I offered a fix to plot colData without a X-axis (that scater doesn’t like). Basically, in the absence of X variable, I rank samples by their Y value
Kevin Rue-Albrecht (12:00:45): > I’ll get started on generating the associated ‘tracking script’, but if anyone has feedback, I’ll take it on board
Aaron Lun (12:01:30): > Currently struggling with some funny behaviour with shinyBS, will upload my changes soon.
Kevin Rue-Albrecht (12:02:23): > @Kevin Rue-Albrechtuploaded a file:ggplot_coldata update - File (PNG): ggplot_coldata update
Kevin Rue-Albrecht (12:03:36): > (just realised it’s much easier to share a screenshot than to ask you to checkout+build+launch the app)
Kevin Rue-Albrecht (12:05:18): > note that I thought about offering a toggle “rank decreasing/increasing”, but once again I think I was trying to go above and beyond on something that users could fiddle from the code we give them
Aaron Lun (12:46:45): > Figured it out;conditionalPaneldoes not play nice with the shinyBS panels.
Kevin Rue-Albrecht (12:56:41): > ouch, well done, i only ever played with the former
Kevin Rue-Albrecht (12:57:59): > quick one: what is the purpose of returninglist(xy=plot.data, plot=out.plot)for dimred? something to do with the brushing?
Kevin Rue-Albrecht (12:59:53): > because i’ve got the data, the plot, and the command for the coldata plot, so i’m looking at the reddim for a reference return value
Aaron Lun (13:13:01): > xywas for selecting the brushed points in other plots. You could also do that with the ggplot object, as long as something about the coordinates gets saved into memory so that it can be accessed from plots linked to it.
Kevin Rue-Albrecht (13:16:45): > ok thanks. so i’ll do the same as Charlotte did this morning, and returnlist(xy = plot.data, cmd = cmd, plot = eval(parse(text = cmd)))instead of only theggplotobject as.make_colDataPlotcurrently does
Kevin Rue-Albrecht (13:17:51): > this way I can do the downstream work iniSEE.Rto catch up the plot from the appropriate list item, while@Federico Marinican grab the command from the other item
Aaron Lun (13:21:35): > Well, we shouldn’t need thexyanymore if we store the ggplot object in the memory.
Kevin Rue-Albrecht (13:23:06): > argh. got you now. didn’t pay enough attention to your “was for selecting”
Aaron Lun (13:27:46): > Ah shit this shinyBS crap is a nightmare.
Aaron Lun (13:28:52): > I’m going to sleep on it. Meanwhile, I’ve merged the UI updates.
Kevin Rue-Albrecht (13:30:16): > ok cool, i’m about to push my final update and offer a merge into theggplotbranch where Charlotte is working
2018-01-14
Charlotte Soneson (10:45:04): > I just pushed some more updates to theggplotbranch, fixing theggplotcode of.make_geneExprPlot(I ended up reusing some of the code fromscater). There are still plenty of things to fix, but if we can agree that this structure works for everyone those can be fixed later. What we have currently are functions that return a list of values, including the command to generate the plot from theseobject (cmd), and the plot itself (plot). I have tried to evaluate as little of the code as possible inside the function (until the return step) to avoid having to duplicate code. Thus, the returned code should reliably generate exactly the plot that is shown.
Kevin Rue-Albrecht (10:56:04): > ah right, I meant to point out that we were running some code twice: once explicitely, the second time when weeval(parse(cmd)))
Kevin Rue-Albrecht (11:03:04): > I still need to clean it up in my own part (coldataPlot), but for instance in.make_redDimPlot, I think all the code that preparesred.dimcan be removed, considering that it will all be done when the command gets evaluated. Follow me?
Charlotte Soneson (11:03:48): > Yes. I haven’t cleaned up.make_redDimPlotyet, so there are some things that are run multiple times. in.make_geneExprPlot, I tried to minimize it.
Kevin Rue-Albrecht (11:07:31): > ok, no worries; i just realised it myself when i was printing one command while executing a slightly different one
Kevin Rue-Albrecht (11:23:25): > Actually, i stand corrected: it’s still necessary to run the commands separately from the evaluation, to generate thexyelement. > Nevertheless, as Aaron pointed out recently, given that the coordinates are stored in theggplotobject, we may remove all things relating to thexyslot at some point
Kevin Rue-Albrecht (11:24:24): > Just testing out the updated code, I get the following error:Warning: Error in .make_geneExprPlot: object 'evals.long' not found
Charlotte Soneson (11:25:49): > Yes, that’s one of the problems. It occurs in the beginning, before everything is loaded and processed. Then it disappears (at least for me). So there needs to be a check somewhere that all the necessary objects exist
Kevin Rue-Albrecht (11:33:15): > yes it settles as well for me after a couple of failed attempts; it’s just weird that it happens at all; you’re right, there must be a failsafe missing somewhere
Aaron Lun (11:56:02): > Note to all: I have “fixed” the collapsible issue I mentioned yesterday, by copying theshinyBS.jsscript into ourinst/wwwand fixing a few bugs (https://github.com/ebailey78/shinyBS/issues/38 ). This ensures that the conditional panels within each collapsible panel do not interfere with the open/closed status of the collapsible panel. However, it means that if you or other packages runlibrary(shinyBS)at any point, the original buggy javascript file will overwrite the fixed one in our package, and the bug will resurface. - Attachment (GitHub): Bug: ConditionalPanel within a CollapsePanel confuses the collapsePanel · Issue #38 · ebailey78/shinyBS > I have a nested conditionalPanel within a bsCollapse. When the conditionalPanel hits a FALSE condition and elements are hidden, the value of the bsCollapsePanel does not get returned properly. That…
Aaron Lun (11:58:57): > The only sustainable solutions are for theshinyBSmaintainer to actually do his damn job and fix the bugs (unlikely); or for us to implement our own collapsible box. The latter is not a bad idea but it would require some decent skill in getting Javascript and Shiny to play nice; I had a go and couldn’t even get pastsyntax error: missing ";".
Aaron Lun (12:05:01): > On a precautionary note, be careful with putting too many things inrObjects; especially modification ofactive_plots, which will trigger re-rendering of the entire UI. Most things can go intopObjects. I’ve noticed that the rendering has recently become slower, though it’s not yet clear why.
Kevin Rue-Albrecht (12:05:20): > any chance we can swing a PR to the shinyBS package? what’s the CRAN policy on unresponsive maintainers?
Aaron Lun (12:06:04) (in thread): > There alreadyis a PR for this issue. There are 8 PRs in total and 56 issues. So… probably not much chance of getting action here.
Kevin Rue-Albrecht (12:06:16) (in thread): > argh - ok
Aaron Lun (12:06:49) (in thread): > CRAN does have a decent policy on unresponsive maintainers - they’ll orphan the package and allow other interested parties to take over. But this does depend on having someone with the skill to take over.
Kevin Rue-Albrecht (12:07:04) (in thread): > oh right, I misread your GitHub link; thought it was your own PR into iSEE
Kevin Rue-Albrecht (12:09:39): > speaking of precautionary note: i notice there’ll be a bit of code conflict merging theggplotbranch back intomaster, as RStudio did the same automated cleanup of trailings spaces in both branches. Simple but a bit tedious to deal with this. Anyone knows where those trailing spaces came from, to avoid the issue in the future if possible?
Aaron Lun (12:10:03): > Probably me, I’m not using Rstudio.
Aaron Lun (12:10:33): > Actually, I’m not sure.
Kevin Rue-Albrecht (12:10:50): > oh right - out of curiosity, what are you using?
Aaron Lun (12:10:52): > vim
2018-01-15
Kevin Rue-Albrecht (06:07:32): > Hi all, I just added some validation code on theggplotbranch to avoid issues when the gene text box contains an invalid gene identifier (either empty default value or gene name not in data). > It seems to catch all errors now (I can’t make it crash)
Kevin Rue-Albrecht (06:10:12): > I need to do some other stuff for a while, but I’d like to apply the fix to the dimred plot as well later today (the dimred reverts to colorless in case of invalid input)
Aaron Lun (06:20:52): > To all - I have finallysolved collapseBoxfunction with its own javascript backend. In this manner, we eliminate any dependence onshinyBS , which avoids many of its associated problems.@Federico MariniI suggest usingshiny ’smodalDialograther thanshinyBS::bsModal.
Aaron Lun (06:24:11): > @Kevin Rue-AlbrechtI suggest regularly mergingmasterwithggplot(not the other way around) to keep the merge conflicts manageable.
Kevin Rue-Albrecht (06:25:14): > I get you. Smaller conflicts to manage if dealt with regularly
Aaron Lun (06:25:35): > I’ll deal with it for now - give me a few minutes.
Kevin Rue-Albrecht (06:26:21): > thanks
Kevin Rue-Albrecht (06:26:52): > I’m knee-deep in contributing a section to a manuscript… so you have time!
Federico Marini (07:32:48) (in thread): > I’ll look into it
Kevin Rue-Albrecht (08:00:10): > ehm.. just tested the new app: seems like yourmergehas cancelled out some of my recent contribution toggplot: the validation message code has disappeared again
Kevin Rue-Albrecht (08:04:09): > think I found it, it’s the ‘merge aaron_edits into ggplot’ that deleted that part
Kevin Rue-Albrecht (08:07:20): > @Charlotte Soneson: can you check your edits to dimred and geneExp plots? I think they were reverted there too
Aaron Lun (08:10:18): > Thats odd, I was pretty careful about that.
Aaron Lun (08:10:35): > OnlyiSEE.Rwould have been affected.
Kevin Rue-Albrecht (08:11:35): > indeed, i think it’s onlyiSEE.R
Kevin Rue-Albrecht (08:11:56): > as far as I can see; the plot appearance has remained updated
Charlotte Soneson (08:12:07): > From what I can see, the dimred and geneExp plot code looks like what I have locally
Kevin Rue-Albrecht (08:14:16): > hmm.. from my perspective, my changes toiSEE.Rfrom this commit have disappeared:https://github.com/csoneson/iSEE/commit/e2d9f530efb50ef4590c602a37b1767f700b0bd4 - Attachment (GitHub): colData plot: show validation error when coloring gene text field is … · csoneson/iSEE@e2d9f53 > …not a rownames(se); do not include an xy element in the return list
Kevin Rue-Albrecht (08:14:36): > (check theifblock after# Do not plot if text field is not a valid rownames(se))
Aaron Lun (08:14:50): > I think I see it.
Kevin Rue-Albrecht (08:14:59): > i’m on it though
Aaron Lun (08:15:11): > okay, sorry. The underlying changes to the shiny machinery were pretty intense. You’ll need to put backmessage(p.out$cmd)in the Gene expression plots as well.
Kevin Rue-Albrecht (08:15:14): > just wanted to point out in case other things disappeared
Aaron Lun (08:17:52): > Though while we’re on this point, it may be more friendly to just refuse to plot the colour in case of an invalid row names, while still plotting the points (this is how reduced dim currently behaves, from my last check).
Aaron Lun (08:19:28): > And then have a littleProgresspanel open up showing the error.
Kevin Rue-Albrecht (08:20:06): > actually, I saw the difference, and I was going to bring that up. > I was also tempted by being friendly to the user, but i thought it more efficient and pedagogic to not plot if the gene field is invalid
Aaron Lun (08:21:06): > Well, as long as the policy is consistent across all plots.
Kevin Rue-Albrecht (08:23:27): > i’ve just pushed my validation code back to ggplot branch
Aaron Lun (08:23:30): > Thanks
Aaron Lun (08:24:25): > I think the Gene expression plots need the same treatment.
Kevin Rue-Albrecht (08:24:39): > didn’t see where to add themessage(p.out$cmd)back though; there already ismessage(cmd)inplotting.R
Aaron Lun (08:25:02): > From the section with the gene expression plots, I think.
Aaron Lun (08:25:37): > On the plus side, this could have been a lot worse if everything had been iniSEE.R.
Kevin Rue-Albrecht (08:29:33): > Anyway, i’m just happy to see my part work again the way I intended it. > Happy to apply the same treatment to the dimred and geneExprs plots later, if no one gets to it before I do.
Kevin Rue-Albrecht (08:32:10): > ah - note to self: I forgot to validate the gene selection when it comes from the gene table: if you unselect the gene, both colData and geneExprs plots show anError: object 'Covariate' not found
Kevin Rue-Albrecht (08:51:32): > ok - problem solved for the colData plot now: both for the free text field and the gene table selection
Aaron Lun (09:03:33): > cool
Federico Marini (10:00:17) (in thread): > Ciao ciaoshinyBS, welcomemodalDialog- done
Charlotte Soneson (10:29:38): > I made some small fixes to theggplotbranch and put back themessage(p.out$cmd)
Kevin Rue-Albrecht (10:41:02): > Nice. If that’s okay with you, I’ll add my little magic to validate the gene name input in the dimred and geneexpr plots
Charlotte Soneson (10:41:38): > Yes, please!
Kevin Rue-Albrecht (10:48:08): > btw, if i read the diff correctly, i would say the merge conflicts come from the fact thatvim(or one of our code editors) probably keeps the level of indentation for empty lines, while RStudio (at least, mine) trims all the spaces from those lines. > I’ll see if I can fix the autocorrect on my side, as I also have issues with RStudio using 4-space indents, even though I set them to 2-space in the project settings. argh
Kevin Rue-Albrecht (11:05:31): > validation message in place for dimred plot; i’ll look at the geneExprPlot later, as there are a few different gene inputs to handle
Aaron Lun (12:19:37) (in thread): > Sweet.
Aaron Lun (12:20:38) (in thread): > I wouldn’t stress out too much about it, as long as we continually merge withmaster, we should avoid/manage merge conflicts.
Kevin Rue-Albrecht (12:59:56): > i’m about to push the validation code for all three plots in the next 5-10 min. > to limit risk of conflicts, i won’t touch constants now, but i think some new ones could be set up (e.g.modeand thelabels of radio buttons indynamicUI.R)
Kevin Rue-Albrecht (13:00:29): > that way, validation message could be made even friendlier
Aaron Lun (13:04:01): > What kind of constants do you need?
Kevin Rue-Albrecht (13:10:11): > let me know how you find the validation messages (when you unselect gene names), but I was thinking silly things like “Y-Axis”, “X-Axis”, “Color by” in addition to the data.frame friendly “XAxis”
Kevin Rue-Albrecht (13:10:50): > basically constants for the labels that the user sees in the UI, in addition to the field names that we use in tables
Kevin Rue-Albrecht (13:11:23): > oops.. one more commit coming your way: forgot to removeprintdebug statements
Aaron Lun (13:14:11): > Ah. Well, you can do what you like, just follow the naming scheme inconstants.Rand stick"Title"to reflect the fact that it’s a user-visible name.
Kevin Rue-Albrecht (13:14:12): > Actually forget about themodething, that was still a memory of my learning curve to understand the encoding of memory column names
Kevin Rue-Albrecht (14:31:52): > @Charlotte Soneson: anything else you wanted to add toggplotbranch? or shall i merge it back tomaster? > I can’t spot any obvious bug right now.
Charlotte Soneson (14:38:27): > Not directly related to the ggplot conversion. There are still things in the gene expression plot that need fixing, specifically in the plotting parameters, I guess it should be possible to choose genes bytextInput, and to choose different genes for the x and y axes (currently they are both linked to the gene table).
Kevin Rue-Albrecht (14:40:32): > true. i noticed the absence of the genetextInputfor that last plot > for the ‘different XY genes’ feature, i can already do it by linking each axis to a different gene table
Charlotte Soneson (14:41:17): > yes, true, but that’s neither very convenient or intuitive:slightly_smiling_face:
Kevin Rue-Albrecht (14:41:40): > yep - took me a minute myself to think it out
Kevin Rue-Albrecht (14:44:28): > which reminds me, i meant to say, if you want to test the latest features efficiently: when opening the app, add 2 more gene tables (3 total). > then you can color dimred with the first one, color colData with the second one, and in the geneExprs plot the first one vs the second one and color by the third one
Kevin Rue-Albrecht (14:45:00): > then you can have fun unselecting genes from the individual tables to see the validation messages in action
Kevin Rue-Albrecht (14:46:50): > which reminds me (chapter 2), that incidentally those input validations have solved the issue of error messages related to the geneExprsPlot
Kevin Rue-Albrecht (14:47:31): > (a lot less fallback code to write, if we train users correctly:wink:)
Charlotte Soneson (14:48:48): > Nice!
Kevin Rue-Albrecht (16:26:16): > Alright, i’m resetting the ‘code conflict clock’ by mergingggplotback intomaster. > I’ve triple-tested all plots, and can’t make them crash (Federico, feel free to give the app to the kids this weekend!)
Federico Marini (17:26:46): > Hehe. Better not, at least the youngest one has a talent in finding out new functions of the trackpad
Federico Marini (17:26:48): > :smile:
2018-01-16
Federico Marini (06:35:09): > <!channel>I did a small implementation of the tour in theminitourbranch
Kevin Rue-Albrecht (06:35:20): > cool, looking forwrd
Federico Marini (06:35:26): > Did we removescaterfrom theDESCRIPTION?
Kevin Rue-Albrecht (06:35:31): > yep i did
Federico Marini (06:35:36): > because we still need it for the examples
Federico Marini (06:35:51): > we explicitly calllibrary(scater)
Federico Marini (06:35:59): > and I got an error while checking locally
Federico Marini (06:36:35): > so I re-added that - or are there better policies for pkgs only used in examples?
Kevin Rue-Albrecht (06:36:38): > ok no worries, i guess it can go back under suggests then? our objective was only to remove the dependency on its plotting functions
Kevin Rue-Albrecht (06:37:16): > off the top of my head, i can’t remember under which section packages needed in example should go
Kevin Rue-Albrecht (06:37:27): > i think it’s suggests, like for unit testing
Federico Marini (06:37:49): > we probably just need normalize and runPCA to be “explicitly” imported - but this does not matter that much, since we load the whole lib for the examples
Federico Marini (06:37:56): > I’ll check too if you want
Kevin Rue-Albrecht (06:39:02): > keep in mind that the example code is only explicitely run on the nighty build machine; user only run that code if they wish to see an example usage. Hence the more relaxed dependency in Suggests, as far as I remember
Federico Marini (06:39:40): > true, but if the R CMD check is not passed (as it does locally for me) we have a problem:smile:
Federico Marini (06:39:56): > so in this scope, it is strange that the CI machines did not raise any issue?
Kevin Rue-Albrecht (06:41:09): > I find the discrepancy strange indeed, but I would have expected the check to pass > hang on, i’ll runcheckon my machine too
Federico Marini (06:41:55): > ok
Federico Marini (06:42:00): > I’m out for lunch
Federico Marini (06:42:05): > I’ll catch up later
Kevin Rue-Albrecht (06:42:19): > ahhh i think i know the issue
Kevin Rue-Albrecht (06:42:58): > i’ve removedscaterfrom theDESCRIPTION, but I haven’t removed the import instructions fromNAMESPACE
Federico Marini (06:43:24): > the import instructions go away with roxygen
Federico Marini (06:43:27): > if you use it
Federico Marini (06:43:34): > the description stays kind of untouched
Federico Marini (06:43:36): > well
Federico Marini (06:43:46): > let’s see what is best for examples
Federico Marini (06:44:05): > iSEEyou guys later:smile:
Kevin Rue-Albrecht (06:44:47): > “The ‘Suggests’ field uses the same syntax as ‘Depends’ and lists packages that are not necessarily needed. This includes packages used only in examples
Kevin Rue-Albrecht (06:45:22): > from the “Writing R Extensions” guidelines
Kevin Rue-Albrecht (06:48:50): > I found the place causing the bug: iniSEE.R, there was theimportFrom('scater' ,...)statement that I forgot to remove. > I’ve only started using roxygen for iSEE, still getting into the habit
Kevin Rue-Albrecht (07:20:23): > i’ve pushed an update that solves thescaterdependency issue
Federico Marini (07:28:27): > perfect
Federico Marini (07:28:37): > thanks for finding & checking
Kevin Rue-Albrecht (07:29:05): > you should probably merge master into your branch, and let me know if if it still checks
Federico Marini (07:29:35): > I’ll look it now
Kevin Rue-Albrecht (08:03:34): > really nice intro feature!
Federico Marini (08:07:07): > Thank you. Could be useful for a first round for people who want to dive in the capabilities
Kevin Rue-Albrecht (09:06:59): > exactly - a couple of collaborators from the experimental side have already shown interest
Kevin Rue-Albrecht (09:08:43): > particularly form a mix of frustration with cLoupe, and the wish to explore the data more interactively than a cycle of emails with their bioinformatician (in one case, me:sweat_smile:)
Kevin Rue-Albrecht (09:15:04): > i think it’s going to be a really helpful tool to both explore data sets and finalise manuscripts: checking out metrics from QC to gene expression estimates and picking representative figures (which gets me thinking about the selling points to keep for the iSEE manuscript)
Federico Marini (09:16:35): > Oh well. “Making life easier” is the best summary
Federico Marini (09:17:36): > but to formalize it, and also shamelessly self-citing my previous package, it is all about combining the ease of interactivity with the technical reproducibility what makes it widely adoptable
Federico Marini (09:17:40): > :smile:
Kevin Rue-Albrecht (09:17:41): > “Making ‘Unbiased Single-cell Analysis’ great again” ?:flag-lr:
Federico Marini (09:18:01): > #MUSAga?
Federico Marini (09:18:20): > trending in 3 … 2 … 1 …
Aaron Lun (09:45:41): > I think I’m late to the battle, but yes,scater belongs inSuggests.
Aaron Lun (09:47:46): > HOLY SHIT THIS INTRO STUFF IS AMAZING
Kevin Rue-Albrecht (09:48:11): > buckle up for branchaaron_edits_2:joy:
Aaron Lun (10:47:30): > Some comments/suggestions: > - Syntax highlighting for the code chunks in the modal; is this possible (@Federico Marini)? > - Move the validation inside the plot function, to reduce the conflation with Shiny infrastructure iniSEE.R? > - Why are we still storingxy?brushedPointsshould be able to use the ggplot if that gets stored incoordinates. > - Let me know when you want to start incorporating the brushing; hopefully the examples inredDimshould be clear.
Kevin Rue-Albrecht (10:51:12): > I left thexyin the plots where Charlotte was working (we distributed them between us), to avoid code conflicts. I can take it out now
Aaron Lun (10:51:19): > okay
Aaron Lun (10:52:05): > Note that.make_geneExprPlotseems to have a bunch of unbound variables.
Kevin Rue-Albrecht (10:52:34): > I thought about moving the validation code inside the plotting functions, but I think they don’t work as expected (cause an error instead of catching it) if they’re not in the application itself. I’ll try again locally and let you know
Kevin Rue-Albrecht (10:54:48): > Indeed, i saw the CMD check notes about the unbound variables. Charlotte probably has a better idea than me how those warnings are generated. I can look into it as well, and whoever comes up first with solution can push it online ?
Aaron Lun (10:56:03): > I see now, they get generated indirectly by theeval.
Charlotte Soneson (10:56:07): > I know where they come from, but I couldn’t agree with myself on the best way to solve them:
Charlotte Soneson (10:56:19): > exactly what Aaron just said:slightly_smiling_face:
Charlotte Soneson (10:56:42): > It can be solved by duplicating some code
Charlotte Soneson (10:56:53): > But I don’t know if there is a better idea
Aaron Lun (10:56:55): > Under what conditions doeseval.longbecome NULL?
Kevin Rue-Albrecht (10:57:05): > To be fair, I want to polish the plotting functions: at the moment, in the absence of color scale, my workaround is to replace the ggplot command by a comment ‘# no coloring’, while Charlotte puts a dummylabs()text value, as it doesn’t get shown anyway
Charlotte Soneson (10:57:49): > It was NULL when the app was started, before all the variables were populated. I think it is solved now with Kevin’s checks
Charlotte Soneson (10:58:24): > Sorry, I am a bit unresponsive today and tomorrow, we have our lab retreat:slightly_smiling_face:
Aaron Lun (10:59:48): > Also, rather thanpaste’ing the commands as we go, it may be better to put them into a vector and just dopaste(, collapse='\n')at the end.
Kevin Rue-Albrecht (11:01:39): > exactly , regarding my polishing of the plotting code, I have been thinking about a stashing the commands in a namedlist(names being ‘prep_data’, ‘ggplot_call’, ‘aes’). At the end, I could reorder that list in the proper order and callpaste(unlist(...))
Kevin Rue-Albrecht (11:02:48): > this way, if any command is absent, it won’t be pasted at all in the final command; instead of the current strategy being to put a dummy command
Federico Marini (11:02:51) (in thread): > - Syntax highlighting: maybe if we use an instance of a shinyAce text editor
Aaron Lun (11:02:58): > I would also break up the code with more comments, it’s a big code vomit right now.
Kevin Rue-Albrecht (11:03:25): > do you mean within each plot? or separate the different plots by comments?
Federico Marini (11:03:32) (in thread): > -> I wanted to keep it pretty compact for the first round. It can nevertheless look like a full instance of editor (autocompletion allowed:smile:)
Aaron Lun (11:03:44): > Within and between functions.
Aaron Lun (11:03:56): > Just do something like##################if you want to separte different functions
Aaron Lun (11:04:04): > just like the different categories iniSEE.R
Kevin Rue-Albrecht (11:04:05): > well, don’t forget that the current code vomit in the console is only for our own debug purpose
Aaron Lun (11:04:20): > Oh, no, I meant inplotting.Ritself.
Kevin Rue-Albrecht (11:04:27): > ahhh
Aaron Lun (11:04:39): > Just scanning through the file, it’s hard to read.
Aaron Lun (11:04:49): > Clarity is espeically important given ourevalvoodoo.
Federico Marini (11:04:53) (in thread): > thank the guys from the galaxy team. I saw once how they switched from video tutorial to “codable” help and I wanted that too
Kevin Rue-Albrecht (11:04:58): > yep - i get you
Aaron Lun (11:05:47): > Could you also stash the plotting commands in another field ofpObjects(e.g.,pObjects$code) so@Federico Marinican print them out incodetrack.R?
Aaron Lun (11:06:24) (in thread): > Hm. I was hoping that there would be a javascript library that just does this highlighting automatically.
Federico Marini (11:07:03) (in thread): > (btw I told you you could have liked that when I said “I#ll just make a simple proof of principle “:smile:)
Kevin Rue-Albrecht (11:07:09): > I could. I wasn’t sure where to stash them, so I thought I’d “deliver” them to Federico in the list returned by the plotting function, and he would take it from there. But ifpObjectsis the right place, I’m happy to put it there
Federico Marini (11:07:29) (in thread): > it’s nice to be right:stuck_out_tongue_winking_eye:
Federico Marini (11:07:57) (in thread): > not that I am aware
Federico Marini (11:08:05) (in thread): > I’ll look around a bit
Aaron Lun (11:08:13): > Yes, that would be the best place, as it persists throughout the Shiny app for the codetracker to pull out at an unspecified point in the future; a list would have to be actively passed to the code tracking function.
Federico Marini (11:08:28) (in thread): > but there i am using a simple verbatimTextOutput, so my options might be limited
Aaron Lun (11:08:45) (in thread): > https://cran.r-project.org/package=highlight
Aaron Lun (11:08:51) (in thread): > Seems to give output in HTML?
Federico Marini (11:10:48) (in thread): > I’ll check it
Kevin Rue-Albrecht (11:14:07): > @Kevin Rue-Albrecht pinned a message to this channel.
Kevin Rue-Albrecht (11:22:04): > btw, i’ll be removing branches from github includingggplot, considering that we’ve merged pretty much everything back into master
Aaron Lun (11:26:54): > Yep
Federico Marini (11:31:39) (in thread): > Highlight does not seem to play nice with us - at least in the way I am trying out
Kevin Rue-Albrecht (11:31:53): > ok - i said something stupid before: validation codedoes work inside plotting functions, even if outside the main application. I would have sworn it didn’t work last time I tried. > I’ll move them now
Federico Marini (11:32:14) (in thread): > (which is a renderUI + htmlOutput)
Aaron Lun (11:34:14): > good.
Kevin Rue-Albrecht (11:37:27): > note that i’ll also clean up the ‘command-building’ aspect as much as possible on the same branch (different commits)
Federico Marini (11:45:47) (in thread): > ok it is not so straightforward as i thought:disappointed:
Aaron Lun (11:50:41): > okay, I’m not doing anything there right now.
Kevin Rue-Albrecht (11:52:50): > actually, you know what, i’ll push the branch now: it already contains the validation code moved out ofiSEE.R. > this way you can safely work iniSEE.Rwhile I only touchplotting.Rto clean up the command-generating code
Aaron Lun (11:56:04): > No worries. I’m really not doing anything iniSEE , I’m struggling withscater code at the moment.
Kevin Rue-Albrecht (11:59:28): > ok. still, i’ll try to clear your way asap. > I’ll take care of thatxything as well before I push, but then I’m not expecting to touchiSEE.Rfor a while
Federico Marini (12:00:09) (in thread): > the user would anyway copy that out in some syntax-aware IDE - but of course I see the added value of such a nice highlighting
Federico Marini (12:00:52) (in thread): > I’m going to put this on hold for now
Aaron Lun (12:08:36) (in thread): > okay.
Federico Marini (12:08:40) (in thread): > Ok, alternative way - need to have some clues on how to really do it (probably needing some intermediate thing-y): aceEditor, on the bottom of the app (could not have it fit in the modalDialog, and unleash the IDE power)
Federico Marini (12:08:45) (in thread): > looks like this:
Federico Marini (12:09:54): > @Federico Mariniuploaded a file:aceeditor.png - File (PNG): aceeditor.png
Federico Marini (12:10:02) (in thread): > (see channel, I could not share it here)
Federico Marini (12:10:29) (in thread): > @Federico Marinimentioned a file:Screen Shot 2018-01-16 at 18.07.10.png . - File (PNG): aceeditor.png
Aaron Lun (12:11:25) (in thread): > Looks nice. Will that go happily in the modal dialog?
Federico Marini (12:11:48) (in thread): > tried, did not work. will re-try
Federico Marini (12:14:45) (in thread): > did
Federico Marini (12:15:00) (in thread): > Still need to check how to get the actualized code
Federico Marini (12:15:18) (in thread): > either via a tmp file or bypassing and giving it directly
Kevin Rue-Albrecht (12:16:52): > cool! so you’re already working on this locally?
Federico Marini (12:17:28): > surprise surprise:slightly_smiling_face:
Kevin Rue-Albrecht (12:17:44): > i’m about to merge the branch that updates 1)validationcode in plotting functions, and 2) don’t returnxy
Federico Marini (12:17:52): > @Federico Mariniuploaded a file:UNLEASH_THE_EDITOR.png - File (PNG): UNLEASH_THE_EDITOR.png
Federico Marini (12:18:06) (in thread): > @Federico Marinimentioned a file:Screen Shot 2018-01-16 at 18.17.04.png . - File (PNG): UNLEASH_THE_EDITOR.png
Federico Marini (12:18:14) (in thread): > something like this?
Kevin Rue-Albrecht (12:18:23): > you’ll soon be able to pick up that code frompObjects
Federico Marini (12:18:39) (in thread): > :victory_dance_emoji, where are you when we need you:
Federico Marini (12:19:33): > :thumbsup:
Federico Marini (12:20:38) (in thread): > I need to clean up a couple of things beforehand and/or do not want to be too rushy
Federico Marini (12:20:48) (in thread): > So I’ll check it later again/clean up
Aaron Lun (12:26:26): > Yeah
Kevin Rue-Albrecht (12:29:36): > tiny little delay: i’m learning to stash stuff intopObjects. Nearly there I believe. > Note that I won’t document the “Command” column ofpObjects$memory$...as one of the ‘parameters’, considering that it is an output rather than an input
Aaron Lun (12:30:23): > Um - I probably woultn’ put it inpObjects$memory.
Aaron Lun (12:30:34): > I would put it in a new list, e.g.,pObjects$commands.
Aaron Lun (12:30:49): > This would be a lot cleaner for everyone.
Kevin Rue-Albrecht (12:30:49): > right, just realised it as I was typing
Aaron Lun (12:31:25): > For even greater ease, turn it into a list of lists, indexed bymode(top layer) andID(second layer).
Kevin Rue-Albrecht (12:31:35): > yep
Aaron Lun (12:31:44): > Thencodetrackonly has to traverserObjects$active_plotsand do some simple indexing.
Kevin Rue-Albrecht (12:32:09): > good news is that I definitely managed to store it. now it’s just about storing itproperly , sorry!:sweat_smile:
Federico Marini (12:32:23): > @Aaron Lunexact, that was basically the long term goal
Federico Marini (12:32:36): > @Kevin Rue-Albrecht: no need to rush:wink:
Federico Marini (12:33:32): > > la gatta frettolosa fa i gattini ciechi
Federico Marini (12:34:01): > kind of follows the > > more haste, less speed
Kevin Rue-Albrecht (12:34:02): > hehehe
Davis McCarthy (12:55:20): > @Davis McCarthy has joined the channel
Davis McCarthy (13:44:05): > Hey all - great to see all this progress. I’m on a loan laptop at the moment, so struggling to get a workable R-dev environment, but will dig into all this ASAP and see if I have any useful suggestions
Kevin Rue-Albrecht (13:50:01): > hi Davis
Kevin Rue-Albrecht (13:50:34): > There is no absolute need for R-devel: I found a hack to get it on collaborator’s R.3.4
Aaron Lun (13:51:03): > Sacrilege
Kevin Rue-Albrecht (13:51:07): > just make it require R≥3.4 in DESCRIPTION
Kevin Rue-Albrecht (13:51:55): > you’ll just have to R CMD INSTALL it, instead of devtools::install_github
Federico Marini (14:20:07): > hi@Davis McCarthy
Federico Marini (14:47:09): > something stalling on travisCI?
Kevin Rue-Albrecht (14:47:21): > yep
Kevin Rue-Albrecht (14:47:32): > i didn’t break it i swear
Kevin Rue-Albrecht (14:47:43): > :stuck_out_tongue:
Kevin Rue-Albrecht (14:47:52): > “We are currently investigating an increased back log for container-based infrastructure”
Federico Marini (14:47:58): > ah ok
Federico Marini (14:48:07): > uh
Federico Marini (14:48:16): > iSEEthe red antenna
Kevin Rue-Albrecht (14:48:18): > i cancelled my build, so yours ‘started’, but it didn’t help
Federico Marini (14:48:37): > i think it is globally stuck
Kevin Rue-Albrecht (14:48:49): > yup. happens
Federico Marini (14:49:48): > Things happen while such services are down, a.k.a. we areforced to work :smile:
Federico Marini (14:49:57): > (well, offline)
Kevin Rue-Albrecht (14:50:49): > $#!+ happens:wink:
Kevin Rue-Albrecht (14:53:20): > btw, i managed to stored the commands inpObjects$commands$[mode], which is finally a list of character vectors. I initialised those with a length matching onto the dimension of the correspondingmemory$[mode]. > This way, you’ll be able to pick the commands of the active plots as he said
Kevin Rue-Albrecht (14:54:39): > (well, to be fair, CI systems being down can also help us pause to chat about developments and plans, if you really miss the online part:wink:)
Federico Marini (15:36:42): > btw, did we agree on the next skype call time?
Federico Marini (15:37:00): > i checked my notes but found no exact info
Kevin Rue-Albrecht (15:48:11): > Don’t think we did. I mainly took note of a list of objectives for this week
2018-01-17
Kevin Rue-Albrecht (05:19:53): > still no progres on the TravisCI side:cry:i have just finished cleaning up my colDataPlot; i think you’ll like it; if so, I’m happy to apply the same pattern to the other plotting functions
Kevin Rue-Albrecht (05:24:16): > the update also includes some new helper functions to build theaesandlabscomponents of the plot
Kevin Rue-Albrecht (05:38:57): > @Federico Marini: my Travis builds just finished and merged. Yours are being built
Kevin Rue-Albrecht (05:40:07): > although now, Github requires that you “Merge the latest changes from master into this branch.” (https://github.com/csoneson/iSEE/pull/46 ) - Attachment (GitHub): welcome aceEditor with syntax highlighting by federicomarini · Pull Request #46 · csoneson/iSEE > Contribute to iSEE development by creating an account on GitHub.
Federico Marini (05:41:43): > ok thanks for the heads up
Kevin Rue-Albrecht (05:42:30): > it’s a bit self-serving: I’m so looking forward to your aceEditor update:smile:
Federico Marini (05:54:19): > pull request merged
Federico Marini (05:54:24): > waiting for CI:slightly_smiling_face:
Kevin Rue-Albrecht (05:58:37): > nice
Kevin Rue-Albrecht (07:31:38): > alright, pushing and merging an update to apply the cleaner code storage for redDim as well (refresher: I did colDataPlot earlier this morning)
Davis McCarthy (14:23:39): > all: do we have a roadmap/plan for features that are planned/in progress? I’m gathering a whole lot of thoughts about the app now, but will try not to harp on about things that are already planned/in progress
Davis McCarthy (14:24:30): > Great work on the package so far! Exciting to see such excellent progress on the package. I’m excited to see how far we can take it
Kevin Rue-Albrecht (15:16:04): > @Davis McCarthy: We haven’t drawn a road mapper se at the moment. > Same thing here: lots of idea flowing through my mind as I clean up the existing code. I started using issues labelled as ‘Enhancements’ as a good place to keep track of ideas, discussion (and reasons to discard them, if applicable). I don’t think we’re excessively worried about giving away ideas that other would implement, are we?
Federico Marini (15:52:54): > One idea that came to my mind for the tracking: what about a way to bookmark the app, probably without the need of shiny’s bookmarks
Federico Marini (15:53:20): > whatever is active gets stored, and that can be passed as a parameter when re-launching the app
Federico Marini (15:53:32): > maybe@Aaron Lunhad something like this envisioned
Federico Marini (15:53:40): > or maybe I am overkilling
Charlotte Soneson (15:57:18): > I think that would be pretty nice. I tried it once (with abookmarkButton), and it worked quite nicely except forDataTables which were somehow not restored.
Federico Marini (15:59:32): > ok, probably not overkilling, but not highest priority then:stuck_out_tongue:
Aaron Lun (16:05:08): > Yes, the memory can be returned to the R session for bookmarking. Some extra work is required to fully preserve datatable memory; currently only the selection is preserved. Brushing will also not be preserved easily.
Kevin Rue-Albrecht (16:24:58): > i’ve never tried to restore inputs and states between sessions: is it at all possible to dumpinputandscein an*.rdafile, and restore them in a later session? or is that just too crude?
Aaron Lun (16:33:13): > I don’t think we can interceptinputand re-set it in a new Shiny instance. We can, however, dumpmemory, which allows us to easily restore most settings by passing the relevant default arguments toiSEE.
2018-01-18
Aaron Lun (13:12:52): > @Kevin Rue-AlbrechtI’m looking atclean_geneexprsnow and it seems to me that your.all_aes_*constants are only ever used inplotting.R. If they’re not cross-file constants, they can be kept withinplotting.Rfor clarity.
Kevin Rue-Albrecht (13:13:14): > crossed my mind too, forgot to ask
Kevin Rue-Albrecht (13:13:47): > i’m just polishing before offering the PR; so i’ll include that in my next commit
Kevin Rue-Albrecht (13:15:19): > I was just about to post: > It’s quite an important update to the geneExprsPlot function. Not in term of function, but of inner working. > I’d really appreciate if you could all build the latest commit, test the app, and let me know if I had a big oversight anywhere
Kevin Rue-Albrecht (13:31:06): > Alright. i’m officially done. pull request submitted. Travis in action > I’ll wait for feedback before I merge it tomaster
Kevin Rue-Albrecht (13:42:25): > @Kevin Rue-Albrechtuploaded a file:To group or not to group.. and commented: The doodle that will go down in history as my flashlight in the midst of inputs to geneExprsPlot:stuck_out_tongue_winking_eye: - File (JPEG): To group or not to group..
Aaron Lun (13:45:37): > Why is it so combinatorial? I would have thought that the colour would be independent of the x-axis.
Kevin Rue-Albrecht (13:49:34): > It’s not the color, it’s the grouping of x and color categorical variables into violins
Kevin Rue-Albrecht (13:51:02): > I meant the color is independent of the axis. But grouping of violins and jitter is affected differently by combinations of x and color. Whether they are categorical or continuous
Aaron Lun (13:51:09): > Ugh.
Aaron Lun (14:01:39): > @Kevin Rue-AlbrechtLooks pretty good, though I would suggest that the points are bounded by the violin if that’s possible - makes it easier to track which point belongs to which x-axis grouping.
Aaron Lun (14:03:33): > Was there anything specifically you wanted us to check?
Aaron Lun (14:05:16): > I have a grant application to edit, but I really don’t want to do it.
Kevin Rue-Albrecht (14:06:06): > briefly functionality: if anything jumped to your eyes as having changed in a way you don’t like
Kevin Rue-Albrecht (14:06:51): > and then whether the code looks more readable/commented to you
Kevin Rue-Albrecht (14:08:26): > also i think the new organisation should have solved that. i’ll check theR CMD check - Attachment: Attachment > Note that .make_geneExprPlot seems to have a bunch of unbound variables.
Aaron Lun (14:16:39): > Okay visual suggestions: > - constraining points within the violin if possible (ggbeeswarm? Can’t remember howscater does it). > - vertical axis labels? I’m open to adding an user checkbox for this, if you think it might help.
Charlotte Soneson (14:25:04): > Another little addition: I just pushed thexygenebranch (with the currentmastermerged into it), which allows gene selection for the x- and y-axes ingeneExprplots via either gene tables or text boxes.
Aaron Lun (14:25:55): > Hold on, I see what’s happened here.
Aaron Lun (14:26:34): > Right, I see where the complication comes from.
Aaron Lun (14:29:33): > Honestly, I think colouring the points is sufficient, which should dramatically reduce the complexity of your code. I don’t think we need to stretch ourselves to break up the points by the chosen coloring factor for violin colouring (which conflates colouring and the position of the points on the plot, which in turn will cause heartache during brushing).
Aaron Lun (14:30:43): > Nonetheless, I do like the violin colours; may I suggest a sensible automatic colouring scheme (nothing too flashy) when colour type is set to “none”, just to give the plot a bit of pizazz.
Aaron Lun (14:31:49): > Onto the code:
Aaron Lun (14:33:11): > - the density of comments is pretty high. Probably could ease off a bit, e.g., lines 57-90. > - looks alright otherwise, though I don’t understand any of the ggplot stuff.
Kevin Rue-Albrecht (14:35:25): > To be fair Charlotte’s code originally had that ‘default colouring’, it disappeared when i linked the colouring directly to theaes, and i had its reinsertion on my low-priority list. Fair point though
Aaron Lun (14:36:56) (in thread): > Looks like you scooped me on the conditional panels…
Aaron Lun (14:37:17) (in thread): > Do me a favour and add it to the column data plots as well?
Kevin Rue-Albrecht (14:54:33): > btw, thanks for the feedback: i’ve copied it into a local to-do list, if no one gets to those things before i can
Federico Marini (15:33:10): > my 2 cents on the violin/swarm: I recall ggbeeswarm and sinaplot doing what we aim to do
Federico Marini (15:33:26): > probably scater’s solution was already good enough?
Charlotte Soneson (15:34:14) (in thread): > Ok
Federico Marini (15:34:42): > maybe i missed it: can we do a gene vs gene scatter plot?
Kevin Rue-Albrecht (15:34:56): > yep: geneExprsPlot
Federico Marini (15:35:06): > (at best with the genes not absolutely linked to the tables)
Kevin Rue-Albrecht (15:35:47): > until Charlotte’s branchxygenesis merged to master , you’ll have to add another gene table
Federico Marini (15:35:55): > aah ok
Kevin Rue-Albrecht (15:36:14): > here - Attachment: Attachment > Another little addition: I just pushed the xygene branch (with the current master merged into it), which allows gene selection for the x- and y-axes in geneExpr plots via either gene tables or text boxes.
Federico Marini (15:36:38): > dammit. read, fede, read:stuck_out_tongue:
Federico Marini (15:36:47): > ok sorry i missed that
Charlotte Soneson (15:54:58) (in thread): > Done
Aaron Lun (15:55:33): > ggplot team; tell me when you’re ready to incorporate brushing.
Aaron Lun (15:55:38): > Otherwise I’m going to sleep now.
Charlotte Soneson (15:55:53): > I think I’m reasonably done, any objections to mergingxygeneintomaster?
Aaron Lun (15:56:06): > shoot first, ask questions later, that’s my motto.
Charlotte Soneson (15:57:59): > Alright, I created the PR, waiting for Travis
Charlotte Soneson (16:12:52): > Ok, merged intomaster, I’ll remove thexygenebranch
Kevin Rue-Albrecht (16:41:47): > I’m happy with the state in which i pushed the ggplot aspect. **I’d say you can safely start incorporating the brushing on the first two plot types (dimRed and colData), while we think about alternatives behaviour for the third one. > I agree that the interaction ofxandcolorposition-dodging in geneExprs may cause problem with brushing that particular plot… although i don’t think it would be easier to brush on ‘regular’ violins. > I already have a pretty good idea how to avoid breaking up the points by the chosen coloring factor, ifneedbe
2018-01-19
Aaron Lun (05:36:45): > All we need are the plot coordinates for brushing, so that’s not really the problem per se. It’s more a conceptual issue regarding the separation of responsibilities (i.e., plotting and colouring). As it is now, it is inconsistent with the behaviour of the other plots. One can also easily imagine how this could lead to very messy plots, e.g., when there are different numbers of colours per level of a factor-likex, plus many levels ofx.
Aaron Lun (05:37:06): > I mean, it’s up to you. But I really think that it’s a lot more maintenance grief than it’s worth.
Kevin Rue-Albrecht (05:42:32): > No worries, I totally agree. > Charlotte put a lot of the ground work to handle this scenario, and before i saw how messy it can get, I also initially felt it would be clearer to ‘dodge’ the colors rather than overlay them
Kevin Rue-Albrecht (05:44:08): > I’ll just have to add an instruction to tell ggplot to group byxonly, even whencoloris categorical. Simple enough. It’s just another aesthetic ofggplotthat can override this type of interactions
Aaron Lun (05:48:39): > Does ggplot automatically try to separate things by colour? That’s insane.
Kevin Rue-Albrecht (05:48:58): > if they are categorical
Aaron Lun (05:49:07): > God.
Aaron Lun (05:49:16): > I thought that was some crazy thing you added in.
Kevin Rue-Albrecht (05:51:31): > nah, that’s why the code is a bit convoluted in geneExprs: we had to deal with the combinations of choices made by the user, but also the behaviour ofggplotin those scenarios
Kevin Rue-Albrecht (05:52:12): > interesting problem though, formally programming the decision-making that we do about layers to show depending on the type of data
Kevin Rue-Albrecht (05:53:55): > > df <- data.frame( > x = factor(sample(letters[1:5], 50, TRUE)), > y = rnorm(50), > color = factor(sample(LETTERS[1:5], 50, TRUE)) > ) > ggplot(df, aes(x, y, color=color)) + geom_boxplot() > ggplot(df, aes(x, y)) + geom_boxplot() > ggplot(df, aes(x, y, color = x)) + geom_boxplot() >
Kevin Rue-Albrecht (05:54:13): > here, some sample code to see the effect of interactions
Aaron Lun (05:54:32): > Missing a comma after they, extra comma after thecolor.
Kevin Rue-Albrecht (05:55:11): > my bad, i just reordered the code before sending it
Aaron Lun (05:55:18): > Ah, that’s just fucked up.
Aaron Lun (05:55:34): > What the hell is ggplot doing?
Kevin Rue-Albrecht (05:55:34): > in a fucked up way, it kinda makes sense though xD
Kevin Rue-Albrecht (05:56:39): > especially with boxplots and violins: you can’t color the one violin perxlevel with the rainbox ofcolorsthat ithisxlevel contains
Aaron Lun (05:57:19): > Yeah, but I just want to colour the points.
Kevin Rue-Albrecht (05:57:25): > you can color the individual data points obviously, but not the grouped distribution
Aaron Lun (05:57:56): > Can’t say I really care about the colour of the boxplot.
Kevin Rue-Albrecht (05:58:39): > I mean the ggplot2 team is pretty smart about their choices. It takes everyone a while to get used to it, but the ‘grammar of graphics’ that it implements is thoroughly thought of
Kevin Rue-Albrecht (06:00:21): > anyway, let me see if I can patch that grouping code that I have in mind; otherwise i’ll have to leave it for later while you get started with the first two types of plots
Aaron Lun (06:03:29): > Well, I think they’re letting aesthetic choices interfere with basic interpretation of the plot.
Aaron Lun (06:04:06): > Perhaps another solution would be to make the violins without colours, and then overlay the coloured points?
Kevin Rue-Albrecht (06:07:03): > That’s one possibility as well. Or to avoid ‘boring black’, make them all a the same color (not linked to an aesthetic)
Kevin Rue-Albrecht (06:08:22): > actually, that’s exactly the result i just got locally
Kevin Rue-Albrecht (06:08:39): > i’ll push the update now so that you can see
Kevin Rue-Albrecht (06:10:03): > i have to say, i understand now why they also split by color: this way you get the distribution within each interaction of levels. That information is lost when you draw a single violin byxlevel. You’ll get what i mean when you see for yourself.
Kevin Rue-Albrecht (06:11:06): > there you go, it’s on branchColor_does_not_group
Kevin Rue-Albrecht (06:28:41): > actually, the PhD student in our group may just have come up with the solution: i’ll see whether faceting helps!
Kevin Rue-Albrecht (06:29:43): > so long as the plot is streched to width=12 and the facets are on a single row, it might just be the best of both worlds
Aaron Lun (06:51:44): > The current branch looks so much better. If you add some jiggle to spread out the points within each violin plot it would be perfect. Also I would probably avoid colouring any of the violin plots at all - even if all points are of the same colour - for consistency, to avoid problems with interpretation where a plot suddenly changes colour due to the loss/gain of one point.
Kevin Rue-Albrecht (06:51:47): > i think we got a winner
Kevin Rue-Albrecht (06:52:03): > i like the facets a lot
Aaron Lun (06:52:07): > Can you brush across facets?
Kevin Rue-Albrecht (06:52:23): > aaaargh, that’s a good question
Kevin Rue-Albrecht (06:53:07): > let me push the code to the new branch, and we can all make our opinion
Kevin Rue-Albrecht (06:54:05): > here you go
Kevin Rue-Albrecht (06:54:14): > branchFacet
Kevin Rue-Albrecht (06:55:11): > argh.. answer is no
Kevin Rue-Albrecht (06:55:33): > but it looks so pretty:unicorn_face:
Aaron Lun (06:55:48): > Oh well.
Aaron Lun (06:56:08): > I think yourColor_does_not_groupis fine.
Aaron Lun (06:56:17): > Just needs a bit of polish.
Kevin Rue-Albrecht (06:56:19): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-01-19 11.55.56.png - File (PNG): Screenshot 2018-01-19 11.55.56.png
Federico Marini (06:56:43): > hashtag meToo for the Color_does_not_group
Federico Marini (06:56:53): > it looks “just clean”
Kevin Rue-Albrecht (06:56:54): > sorry bad example
Kevin Rue-Albrecht (06:57:08): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-01-19 11.56.45.png - File (PNG): Screenshot 2018-01-19 11.56.45.png
Kevin Rue-Albrecht (06:57:42): > no worries, i’ll go with the majority vote
Federico Marini (06:58:18): > If needed to have my choice win, I’ll request to have the kids’ votes count too:smile:
Kevin Rue-Albrecht (07:00:10): > i’m happy knowing that i’ve shown the different possibilities
Kevin Rue-Albrecht (07:28:55): > i’ll merge theColor_does_not_groupin the next few minutes. I’ve polished it a bit. > One interesting side-effect feature: violins are colourless if they contains data points of different colours; otherwise (if all the data points are the same colour) they take up that colour
Kevin Rue-Albrecht (07:31:15): > I kinda like that, as an immediate information. > Also, no matter which default behaviour we choose, there will always be users who would like it different. As long as we have reasonable default that we can integrate with brushing, and that we give the code of the plot to the user, they can fiddle the appearance to their heart’s content
Federico Marini (07:32:02): > aaaaaand if the users have such wishes, they’ll be our guests in changing themselves the code we return:wink:
Kevin Rue-Albrecht (07:33:20): > Yeah - that was my last point. Sorry I get a bit more poetic than clear in my writing sometimes
Aaron Lun (07:42:24): > Fine with me.
Aaron Lun (07:43:00): > @Federico MariniCan we start stitching up the code tracker to the ggplot code inpObjects$memory?
Aaron Lun (07:44:59): > ggplot team: I think we need to think about how to handle the ColData plot, for all choices of x/y covariate/factor. X=covariate Y=covariate is a simple scatter plot; X=covariate Y=factor is a horizontal violin plot; X=factor Y=covariate is a vertical violin plot; X=factor Y=factor is… what? I was thinking of circles in a grid, where points are jittered throughout the circle and the area of the circle is proportional to the number of points.
Aaron Lun (07:46:38): > In theory, the same issue also arises for the gene data plot, for arbitrary assays you could have factors.
Federico Marini (07:47:29): > @Aaron LunI’ll start looking into it. As far as I could seem to “sense” from brief checks of the latest versions, the ggplot team saves the entire codeas well as the values of the parameters?
Aaron Lun (07:47:51): > I think so, so there’s no need to print code defining the parameters as you’ve currently done.
Federico Marini (07:48:24): > I did not know back then how many gifts I could get:smile:
Aaron Lun (07:48:28): > Sure
Federico Marini (07:48:53): > Oh well. It is going to be “just easier”
Kevin Rue-Albrecht (07:48:55): > ah right. thanks for reminding me: i noticed that I was fixing stuff in geneExprs that I wasn’t back-porting to similar situations in colDataPlot
Kevin Rue-Albrecht (07:49:04): > then i forgot
Aaron Lun (07:49:14): > I’d imagine you could actually have one central function for both things.
Aaron Lun (07:49:40): > Hell, we could have one central function for all things
Aaron Lun (07:49:46): > no reason why the reduced dims couldn’t be factors either.
Aaron Lun (07:49:50): > Though that would be sort of weird.
Kevin Rue-Albrecht (07:50:38): > eh.. not sure what you guys mean by ‘storing the parameters’. We identify them, process them, and stitch them in the command, but we’re not returning them individually. Although theheadersection that I recently added would be a fine place for that
Aaron Lun (07:53:50): > I interpreted it as the parameters were part of the returned plot commands.
Aaron Lun (07:54:04): > so no need to define new variables with the set parameter values.
Kevin Rue-Albrecht (07:54:05): > I agree with your idea of refactoring things into functions: you probably noticed that I have done that during my clean up of the three types of plot. > However, I need to emphasise that this was only possible thanks to all the work Charlotte and I initially put into writing draftier versions of those functions. > What I mean is that the refactoring will take place iteratively. It wasn’t possible as long as Charlotte and I were doing things in slightly different ways
Aaron Lun (07:54:25): > Yes, of course, agreed
Kevin Rue-Albrecht (07:56:36): > Ok. For now, I’ll assume that giving the plotting code is implicitely giving away the inputs. That will avoid overcrowding the tracker. Let me know if later I should also explicitely list the individual inputs in the header.
Aaron Lun (07:57:12): > I think implicit inputs in the plotting code is fine - hopefully people should be familiar enough with ggplot to pick it apart.
Federico Marini (07:57:51): > I’d say I take a good look and get back to you if something is weird. Pretty confident it is already everything in good shapes & formats
Charlotte Soneson (08:18:54): > Sorry I’m late to the discussion, had to go to the gym:weight_lifter::slightly_smiling_face:I like theColor_does_not_groupas well, just tried it and it looks nice to me. For the implicit parameters in the code, I think that it is easier that way, and the code gets much shorter.
Aaron Lun (08:59:37): > Alright, looks like we’re all on the same page. Once you guys are happy with the state of the plotting code, I can point out how the brushing will work. (Let’s use only the reduced dims as an example, for the time being.) Either I can tell you guys how to do it, or you can put in some code showing me how to plot/highlight a subset of points inggplot .
Kevin Rue-Albrecht (09:04:16): > well, i’ve never set up brushing so far, and i’m trying to get some things done before the weekend, so I’m happy to let you or Charlotte get started on the brushing. I’ll try to follow the conversation though,
Davis McCarthy (11:37:40): > have you figured out settinggroup=1in ggplot to avoid grouping by colour if not desired?
Davis McCarthy (11:38:00): > (apologies if obvious/redundant - I only just skipped the many messages since this morning)
Davis McCarthy (11:38:32): > Also at the risk of making redundant comments, here are some thoughts from a look at the package over the last couple of days (which I guess is Wednesday’s version?):
Davis McCarthy (11:38:40): > - Help and tour is good (needs some fleshing out, but great start) > - I find the “brushing” confusing. The term itself is not familiar to me, so > I don’t immediately intuit what it should do and the help doesn’t explain > it in enough detail. I suggest we have a help item particularly for > brushing that could pop up for people to see/remind what it is without > going through the tour again. > - Standard R help/docs not available when Shiny app is running, so needs > to be completely self-explanatory (have docs/explanation available from > app) > - Do we have a roadmap/plan for features we want to add and where to track progress? > - I don’t like the default ggplot2 colormap for continuous variables; not > a useful scale and too “vanilla” > - Would really like information on mouse hover - should be an option to > select what to show on hover > - Related to the above, for an “interactive” tool the bokeh R/Python > package is a good example of what we’d like: easily move + zoom plot > area, details on mouse hover over points > - Can I select one or more cells, either by clicking or selecting by area > and have the names saved to file or R session? Would be great for > defining cell sets both for QC and biology > - Option to add a smoothed fit to 2D scatter plots (gene expression and colData) > - For gene expression plot I seem to get expression for the same gene > plotted on both axes. Do I need to define a second gene table to plot > expression of two genes against each other? I don’t find this very > intuitive; at the very least needs explanation. > - Inconsistent plot style between reduced dim and other plots > - “Extract the R code” looks like it just has variable definitions, but > does not have function calls or the code that would let me reconstruct > the plots outside of the app. That’s the aim, right? > - ‘overview first, zoom & filter, details on demand’ is the ‘Visual > Information-Seeking Mantra’ for large datasets - we should apply this! > - Will this run from a remote cluster? Two use cases: > (1) I have a very large dataset and can’t/don’t want to analyse it on my > local laptop/desktop. Will I be able to access the shiny app locally > through ssh port forwarding or similar? (the rmote pkg by Ryan Hafen is > a good example/motivation there) > (2) I produce a dataset for my > biological collaborator on my institute’s cluster. The dataset is too > big to transfer/they don’t have the setup to run iSEE locally on this > big dataset. Can I make the app available for them to access through a > browser? > - Could we do the above but from a cloud environment (AWS/GCP etc)? This > would be great for exploring HCA data which will end up on all of those > platforms
Aaron Lun (11:38:50): > Jesus
Federico Marini (11:39:13): > DAVIS!
Federico Marini (11:39:14): > :smile:
Davis McCarthy (11:39:42): > Sorry - have been noting things down as I was able between other things
Aaron Lun (11:39:57): > Should break things up into threadable comments
Federico Marini (11:40:03): > we should probably address each point separately. is a github issue a good place for this?
Federico Marini (11:40:12): > something like we can keep things checked
Aaron Lun (11:40:29): > Yes, create one issue per point above.
Federico Marini (11:40:32): > with a box [ ] that gets [x] once addressed
Davis McCarthy (11:40:56): > I have severely limited time over the next few weeks, but will keep trying to comment and contribute (hopefully some code too) where I can!
Davis McCarthy (11:41:28): > Would you like me to turn these into GH Issues now?
Kevin Rue-Albrecht (11:43:26): > @Davis McCarthynot sure on which version of the app your suggestiongroup=1is based. But I think that’s the thing that I solved in my latest updates
Kevin Rue-Albrecht (11:44:07): > totally unrelated: not sure if you guys are aware the ‘Single Cell Genomics Day Livestream’ in onhttp://satijalab.org/scgd/livestream/ - Attachment (satijalab.org): Single Cell Genomics Day Livestream > Lab Webpage —
Aaron Lun (11:44:57): > Probably goes into#random
Kevin Rue-Albrecht (11:45:26): > sorry yeah
Davis McCarthy (11:45:45): > @Kevin Rue-Albrechtyep, looks like you solved the grouping stuff: nice one!
Charlotte Soneson (11:48:59): > Also the gene-vs-gene plotting is addressed in the latest versions
Kevin Rue-Albrecht (11:50:10): > > ‘Inconsistent plot style between reduced dim and other plots’ > I think I solved that while standardising the code of the three functions
Kevin Rue-Albrecht (11:52:53): > > Standard R help/docs not available when Shiny app is running, so needs > > to be completely self-explanatory (have docs/explanation available from > > app) > We could have atabPanelthat only contains static text of documentation (?)
Aaron Lun (11:53:47): > People only need to learn once; I would suggest putting stuff in the vignette. Possibly also a button that opens the vignette from the app.
Kevin Rue-Albrecht (11:55:27): > Indeed, I would definitely prefer pointing to documentation outside the app as much as possible. That was the only ‘in-app’ solution I could think of.
Federico Marini (11:55:57): > -> for this, I have had a similar experience withidealandpcaExplorer
Federico Marini (11:56:05): > both are almost shiny-only pkgs
Federico Marini (11:56:20): > the vignette is useful (as well as required, but we knwo that)
Federico Marini (11:56:54): > but the live help is mainly the persistent reminder of what needs to be done
Davis McCarthy (11:57:21): > OK, I’ve added those thoughts as issues on GitHub now.
Kevin Rue-Albrecht (11:57:39): > > I have a very large dataset and can’t/don’t want to analyse it on my > > local laptop/desktop. > I think we’ve drawn the line atvisualising preprocessed data. I would discourage any kind of computation within the app
Davis McCarthy (11:57:48): > Some you’ve already dealt with! That’s great
Federico Marini (11:58:10): > one thought additionally: some text from the vignette could be “duplicated” and inserted in a collapsible element. People can click it, read it if needed, then close it. Maybe something like…
Kevin Rue-Albrecht (11:58:16): > I am also very interested in point (2) though
Davis McCarthy (11:58:19): > I may not be able to/want to open a 12Gb SCE object on my laptio
Davis McCarthy (11:58:22): > laptop
Federico Marini (11:58:35): > a help icon in the top right corner - this triggers another modal with the juice of the vignette?
Davis McCarthy (11:58:44): > and I think making datasets available to collaborators is a massive potential for this package
Davis McCarthy (11:59:06): > And that makes the availability of help/docs within the app even more important
Davis McCarthy (11:59:29): > I’d like to share the data with the folks who generated it without moving the dataset or requiring them to install anything locally
Davis McCarthy (12:00:17): > I agree about not computing anything in the app
Kevin Rue-Albrecht (12:01:25): > I have also been thinking about large data sets. I would really love to see a feature that triggers when the data set is over certain thresholds of data points. Kind of a Google Maps dynamic level of detail: e.g. show groups of cells as filled ellipses
Davis McCarthy (12:01:52): > Yes, that sort of thing would really elevate this.
Kevin Rue-Albrecht (12:02:03): > However, that would require a bit of pre-computation to ‘know’ which groups of cells to collapse together
Davis McCarthy (12:02:11): > how can we show millions of cells without trying to plot millions of cells at once?
Kevin Rue-Albrecht (12:02:22): > I thinkgeom_density_2d()will be key
Kevin Rue-Albrecht (12:03:11): > although that would use color to show the local density of cells; which makes color unavailable for another covariate as currently possible
Aaron Lun (12:04:43): > I think we need something that preserves the point-based nature of things. There are strategies for density-dependent downsampling that we can use; certainly we don’t have to show all the points once the pixel space has been completely occupied. I’m sure we can easily determine whether the pixel space has been saturated within a local area, and basically stop plotting. This would allow us to keep colours for points and also brushing.
Aaron Lun (12:05:24): > This would be dependent on the size of the points; obviously if you docex=0.00001then you’ll never saturate the pixel space no matter how many points you have.
Kevin Rue-Albrecht (12:07:22): > Also, having too many data points will generate huge vector graphic files if we allow users to export plots from the app. > (I know we give them the code, but I assume we may add a ‘Save as …’ button to export them from within the app, no?)
Davis McCarthy (12:07:46): > yes, on both counts
Aaron Lun (12:08:10): > You could just right click and have save as image.
Kevin Rue-Albrecht (12:08:30): > that’s only saves a bitmap, no?
Aaron Lun (12:08:40): > yep, that’s the idea.
Aaron Lun (12:08:46): > If they want something else, they can just run the code.
Kevin Rue-Albrecht (12:09:33): > yeah, i was just getting to thinking this way. can’t give them manuscript-ready figures and a button ‘Save as Nature Letter’
Davis McCarthy (12:11:19): > they can do that if we’re all listed as co-authors
Davis McCarthy (12:11:31): > with veto rights for shitty papers
Federico Marini (12:13:54): > We should add tiny watermarks to the images
Federico Marini (12:16:10): > I recently stumbled into a publication with a figure clearly done with pcaExplorer - wrote them because I was curious. Found out they neither cite/refer/acknowledge that. And for a paper withTranscriptional landscape...in its title, tonot
Kevin Rue-Albrecht (12:16:22): > hehehe … I was also tempted to put:copyright:in the header of the code printed by the plotting functions
Federico Marini (12:16:47): > https://www.nature.com/articles/s41598-017-01468-y - in case you are interested –> nature sci rep states openness also in their editorial policies. Oh well.:stuck_out_tongue: - Attachment (Scientific Reports): Transcriptional landscape of epithelial and immune cell populations re > Transcriptional landscape of epithelial and immune cell populations revealed through FACS-seq of healthy human skin
Aaron Lun (12:17:04): > Scientific reports…. ’nuff said.
Davis McCarthy (12:18:15): > No good:hankey:
Federico Marini (12:18:22): > Well, the not so funny part is that this data could be relevant to a project of a coop partner of mine. And I wrote to the editors innovember
Federico Marini (12:18:33): > as well as to the authors
Federico Marini (12:18:36): > well
Federico Marini (12:18:41): > we’re going OT, sorry
Kevin Rue-Albrecht (12:21:12): > bah.. it’s Friday:wink:
Federico Marini (12:24:59): > my friday is about to end with a post-christmas party of my new lab
Federico Marini (12:25:14): > the boss pays a fully fledged dinner to everyone:slightly_smiling_face:
Kevin Rue-Albrecht (12:25:40): > means we can come over?:innocent:
Federico Marini (12:26:46): > it’s in one hour from now. if you can make it, you can be my guest:smile:
Kevin Rue-Albrecht (12:27:32): > unfortunately…:stuck_out_tongue_winking_eye:
Federico Marini (12:31:01): > <!channel>, one thing I want to ask you. I am attending in 2 weeks from now a “single cell data” group meeting, and one of my mentors suggested I could give a short showcase of what (quoting his words) “this new shiny shining thing does”. Are you guys fine with that?
Federico Marini (12:31:22): > It would be not more than a flashlight-like talk
Kevin Rue-Albrecht (12:33:39): > Well, no matter how many more features we plan to add in the future, I am already proud enough of the current state of things to attract a user base, so my opinion is ‘give them the full tour!’
Charlotte Soneson (12:34:50): > Yeah, sounds good to me too. I think both marketing and user feedback are good at this stage.
Federico Marini (12:36:47): > I will report useful comments I will get there - although, I think you as developer base might already have (much)more exposure to scRNA-seq datasets than what local groups might have here in MZ
Aaron Lun (12:40:26): > Fine with me.
Federico Marini (12:40:58): > perfect, I’ll wait for Davis to have the unanimity
Federico Marini (12:41:07): > (if that is an english word)
Kevin Rue-Albrecht (12:41:51): > otherwise, just unleash the kids on him:stuck_out_tongue:
Federico Marini (12:42:23): > one way or another we willall
Federico Marini (12:44:14): > BTW. My chance here to also thank you for pinging me back in Cambridge and having me hop on the iSEE-truck. It made > a- my december mood much more motivated > b- my skills better by looking at your code and coding styles > c- happy overall to get busy on something I wanted too and fill up the dull pure-service days
Federico Marini (12:44:31): > So, thank you. Takk, merci, grazie.
Federico Marini (12:45:56): > (d- the nice, refreshing enthusiasm of cooperative work where everyone isreally motivated to get things done)
Kevin Rue-Albrecht (12:46:15): > Same here. With the experience of developing another Shiny app by myself, I can say that it’sa lot more stimulating and enriching to work with you guys
Charlotte Soneson (12:46:51): > Totally agree!
Federico Marini (12:48:56): > I hope I don’t lose the fire once this is terminated:stuck_out_tongue:but it is really a different shift the one we use here. not autopilot, but jeez, everyone has its own nitro boost, and that’s just great:pray:
Kevin Rue-Albrecht (12:50:57): > hold on until Davis’ next batch of reviews to lose fire:stuck_out_tongue:
Kevin Rue-Albrecht (12:51:38): > he was just getting warmed up
Aaron Lun (13:03:24): > Well, as with all these projects, the best person to resolve an issue is the person who raised it.
Kevin Rue-Albrecht (13:09:01): > let me pass that point to my PI too (Steve Sansom):grin:
Aaron Lun (13:10:43): > Does he still remember how to code, or is he too far gone?
Kevin Rue-Albrecht (13:12:09): > oh no, he is on top of things with regard to coding
Aaron Lun (13:47:23): > I will try to deal with the brushing tomorrow, just for the reduced dimension plots.
2018-01-20
Aaron Lun (07:49:54): > is anyone there?
Aaron Lun (07:55:55): > Thebrush_integrationbranch demonstrates how abrushBycolumn is added to theplot.data, via a set of commands. I don’t know how to impose a transparency layer or to recolour the plot, so I’ll leave that to the ggplot team; but note there are a couple of helpful values inparam_choicesto drive this: > > param_choices[[.brushEffect]]==.brushColorTitle # use param_choices[[.brushColor]] to color > param_choices[[.brushEffect]]==.brushTransTitle # use param_choices[[.brushTransAlpha]] to choose alpha > param_choices[[.brushEffect]]==.brushRestrictTitle # subset plot.data to restrict it to the brushed points >
Aaron Lun (08:00:50): > I’ve only done this for the reduced dimension plots so far; however, it should be simple to propagate this to all other plots. I would suggest factoring out all colouring and brushing code to separate internal functions to avoid code duplication, given that the process of choosing the brushing points and colouring factors is more or less the same across plots. (I’ve already done a bit of this.)
Aaron Lun (08:02:30): > I’m going for lunch now, will be back in an hour.
Davis McCarthy (08:47:50): > ^^@Kevin Rue-AlbrechtVery happy for you to present and get some feedback!
Kevin Rue-Albrecht (08:55:47): > @Davis McCarthy: i think you meant@Federico Marini:smile:
Kevin Rue-Albrecht (08:58:27): > @Aaron Lun: I can take care of the transparency layer. I’ve got some code for that
Kevin Rue-Albrecht (09:07:02): > i assume you mean that we should continue coding on yourbrush_integrationbranch then? (to avoid merging tomastersomething that is only partially implemented)
Aaron Lun (09:13:41): > yes.
Aaron Lun (09:14:19): > Are you working on it right now? I was going to refactor the colours out - we’ll probably hit some merge conflicts if we work on it at the same time.
Kevin Rue-Albrecht (09:14:45): > no, i’m actually about to go out for a couple of hours, you’re good to go
Kevin Rue-Albrecht (09:16:01): > Plus, i got a bit ahead of myself: I meant that i know how to add a transparency layer without problem, however i still need to get familiar with what’s coming in from the brushing, and how to handle the different modes
Aaron Lun (09:16:56): > good, bcause I don’t want to work on this manuscript that I’m meant to be doing.
Kevin Rue-Albrecht (09:16:57): > feel free to leave a couple of print statements, to point me/us in the right direction
Kevin Rue-Albrecht (09:17:08): > hahaha exact same thing here
Aaron Lun (09:22:01): > Also, the currentse_nameis unsafe, as a user could have named their input SCE as"plot.data". In which case, the SCE variable would clash with our internal plotting variables. It would be safer to force users to define their SCE asseat the top of the reproduced code, e.g.,sce <- plot.datafollowed by our redefinitions.
Kevin Rue-Albrecht (09:23:13): > Agreed. I’ve kept racking my brain about a better way to deal with this aspect
Kevin Rue-Albrecht (09:23:42): > i can easily change it back; except if you want to do it
Aaron Lun (09:23:47): > I’ll do it.
Kevin Rue-Albrecht (09:23:50): > cool
Kevin Rue-Albrecht (09:24:48): > while we’re on the subject of where the input comes from, i’ve been thinking about supporting RDS files available online, likeiSEE(url="https://www.dropbox.com/s/2p2071rxqekari5/allen_sce.rds?dl=0 ")
Kevin Rue-Albrecht (09:25:37): > Haven’t done anything yet, but to me this would have an awesomeness factor >Inf
Kevin Rue-Albrecht (09:26:42): > (this is a real link that points to my public Dropbox where I put an RDS file this morning)
Kevin Rue-Albrecht (09:40:57): > Just toying with the idea above, i’ve run into this issue when I download it from Dropbox as a tempfile() and runreadRDS:https://support.bioconductor.org/p/74573/ I don’t have a problem to runreadRDSon the local file though. > Anyway, i’ll go offline for a bit and inspect that later
Aaron Lun (10:55:49): > Done. Lines 70-84 inplotting.Rare set for implementing the brushing effect in reduced dimension plots. Depending on how that goes, we may be able to spin it out into a separate function for application to all plots.
Aaron Lun (10:57:10): > Note a few additional changes. (1) As I mentioned before,se_nameis no more.codetrack.Rwill have to definesefrom the name of the input object, as well asall.coordinates <- list()for holding brushing coordinates.
Aaron Lun (10:58:26): > (2) Intermediate variables needed to determine whether something is groupable is now evaluated directly from the commands incmds. This reduces redundancy, compared to them being defined separately in every case.
Aaron Lun (10:59:41): > (3) Evaluation is now performed within a separate environment to avoid variable conflicts. In addition,cmdshas been altered to separate commands intotodoanddone, to avoid wasting time re-evaluating executed lines.
Aaron Lun (11:00:23): > (4) DefiningColorByis constant across all plots, so this has been spun out into a separate function. The validation code also exists within this function, which halves the total number of lines inplotting.R.
Aaron Lun (11:01:35): > (5) I was wrong about being able to useggplotobjects directly inbrushedPoints, so we have to store the resultingdata.frames explicitly in memory. This is achieved with some modifications toiSEE.R.
Kevin Rue-Albrecht (11:11:26): > Nice work. I’ll look at the update code soon, just got a couple of other things to do before
Kevin Rue-Albrecht (11:14:57): > Regarding the Dropbox issue I pointed out above, it seems that neithercurl_downloadnordownload.filemanage to download the RDS file. > As I’m writing this, I just solved it:"https://<...>?dl=1"lets both functions download the file
Kevin Rue-Albrecht (11:18:40): > @Aaron Lun: i see you’ve got a PR, so I take it the code is stable as it is, and we’ll can all just branch out ofmasterto work on separate features once the PR is merged?
Aaron Lun (11:19:36): > No, I was just pressing buttons mindlessly. I’d like to see an example of brushing working before merging.
Kevin Rue-Albrecht (11:21:05): > ok. I just couldn’t resist looking at the code already: can you just explain the difference betweentodoanddoneincmdsnow, to save me time figuring it out from the code?
Aaron Lun (11:29:31): > commands are initially put intotodo. Upon evaluation (e.g., to figure out whether a colorby value is a factor or covariate), the commands are shuffled intodone. This avoids them being re-evaluated; see thePLOT EVALUATIONpoint in.make_geneexprsplot.
Aaron Lun (11:30:12): > Evaluation in the middle of the function is motivated by point (2) above.
Kevin Rue-Albrecht (11:31:08): > oh right, i got you
Kevin Rue-Albrecht (11:32:04): > indeed, i was a bit uncomfortable having to collect and store certain covariates in order to make decisions at the end. nicely done!
Kevin Rue-Albrecht (11:32:16): > and i see what you’ve done in.evaluate_remainder
Aaron Lun (11:32:40): > Well, John didn’t hire me just for my good looks.
Aaron Lun (11:33:08): > (Probably more of a 70:30 split.)
Kevin Rue-Albrecht (11:34:40): > one more question aboutbrushing: i get what ‘Restrict’ and ‘Color’ as supposed to do. Is ‘Transparent’ supposed to ‘hide’ the brushed points?
Aaron Lun (11:36:20): > The non-brushed points should be transparent.
Kevin Rue-Albrecht (11:36:50): > ahhhhh, good that you told me, makes sense
Aaron Lun (11:38:36): > Note that with restrict, the non-brushed points should not even show up (i.e., not just non-visible - they can’t be on the plot). This is necessary to allow FACS-like gating.
Kevin Rue-Albrecht (11:38:52): > yep, i imagined that
Aaron Lun (11:39:18): > good
Kevin Rue-Albrecht (11:41:00): > hmm.. just thinking about the ‘Color’ though: is it completely overriding the coloring parameters of the recipient plot (non-brushed points are set to black and only brushed points are colored), or should it be a ‘merge’ of the plot’s coloring, with the brushed points being ‘overplotted’ with the selected color?
Aaron Lun (11:41:52): > Merge.
Aaron Lun (11:42:17): > Let the user figure out what colour choice does not clash. Defaults to red, I think, which should be fine.
Kevin Rue-Albrecht (11:42:47): > Yep, sounds good.
Aaron Lun (11:42:53): > And the entire thing defaults to transparent, which is even better.
Kevin Rue-Albrecht (11:43:41): > i am so looking forward to the final result:smile:
Kevin Rue-Albrecht (12:52:28): > Anyone familiar with theBiocFileCacheandExperimentHubfunctions? > I’ve pushed a demo of how I’d expect users to access remote data sets, but at the moment it’s using a basiccurlcommand. See BranchurlIf anyone’s interested, I’m sure Lori would be happy to advise them
Kevin Rue-Albrecht (12:53:39): > The magic command to test the branch isiSEE(url = "https://www.dropbox.com/s/2p2071rxqekari5/allen_sce.rds?dl=1 ")And yes.. I’m aware it takes a bit of time to download the file, hence my point about using the Bioc cache
Kevin Rue-Albrecht (13:36:24): > I’ll merge it intobrush_integration, where i branched from. It’s dead easy to remove if we really don’t like it.
Aaron Lun (13:37:09): > I’ll leave the rest of the integration of the brushing to you guys, then.
Aaron Lun (13:37:18): > Merge it withmasterwhen you’re done.
Federico Marini (18:03:04): > midnight surprise coming
Federico Marini (18:03:47): > ggplot team, thanks to your new code, the tracking part works like a charm
Federico Marini (18:03:55): > and with super simple clean code
Federico Marini (18:04:42): > I added a small comment intro on the code, plus the indication to run and reportsessionInfo
Federico Marini (18:05:16): > on branchtrack_cmdsyou can give it a ride
Federico Marini (18:06:05): > @Aaron Lun: I don’t know what the plans for the brushing are, but: > - do we want it to be replicated in the tracked code? > - is it possible at all?
Kevin Rue-Albrecht (18:13:18): > heya
Kevin Rue-Albrecht (18:13:29): > mega ultra super suprise coming from the ggplot team too
Kevin Rue-Albrecht (18:14:04): > you remember that rocket that exploded because a programmer forgot a semi colon?
Kevin Rue-Albrecht (18:14:08): > that was me this evening
Kevin Rue-Albrecht (18:14:35): > i forgot to quote the colour to highlight brushed points by
Kevin Rue-Albrecht (18:14:45): > unfortunately, that colour was#FF0000
Federico Marini (18:15:02): > ihih:stuck_out_tongue:I branched frommasterso I did not see that happen
Kevin Rue-Albrecht (18:15:15): > and i’ve been fighting with an ‘unexpected end of input’ all evening
Kevin Rue-Albrecht (18:15:40): > that bloody hashtag was commenting out the end of the line
Federico Marini (18:15:51): > shiny can be nasty in error communication
Kevin Rue-Albrecht (18:16:35): > i’m all good now, i had all the rest correct, so it’s about 10 min before I can push the update
Kevin Rue-Albrecht (18:16:47): > for all three types of brushing
Kevin Rue-Albrecht (18:18:52): > let me push the brushing tonight, and we can make a decision tomorrow
Kevin Rue-Albrecht (18:19:24): > the app has become self-aware to the point that I think it will give the code of brushing by itself
Federico Marini (18:19:34): > I took the liberty to up the z. version number
Kevin Rue-Albrecht (18:52:37): > Hi<!channel>! > From the ggplot team to you: > Brushing is now fully integrated (all three kinds) to the dimRed plot. PR is open on branchbrush_integrationand will merge as soon as ready. > A few notes: > 1) axes of the dimRed plot are shown again, as the ‘restrict’ mode of brushing ‘zooms in’ the selected data points. Let me know if you think that we should preserve the original axes ranges instead. > 2) i plotexactly the number of samples in the data set, i donot overplot even when merging the colouring parameters of the plot and the coloring of brushed samples. Each subset of samples is in its own layer, and follows independent colour aesthetics. > 3) be gentle with the input buttons, let the app refresh before you click another button. i’ve had a few times the radio buttons flicker between two states because i clicked them too quickly. > 4) enjoy!
Kevin Rue-Albrecht (19:34:53): > Alright, i’m officially done for the night. Catch up tomorrow!
2018-01-21
Charlotte Soneson (02:20:07) (in thread): > This looks great! Regarding 1), I think the ‘Restrict’ mode should zoom in. Preserving the original axes can be achieved by ‘Transparent’ with the parameter set to 0.
Kevin Rue-Albrecht (05:55:57): > I’m aware of the build error; I’m adding a section about using theExperimentHubthat I willeval=FALSEas soon as I’m sure that the commands are correctly running locally.
Kevin Rue-Albrecht (06:12:48): > alright, i’m off again until the next bug report, if anyone wants to work on thebrush_integrationbranch
Kevin Rue-Albrecht (07:02:59): > @Aaron Lun, can you just look at my latest commit and polish it as you did before? > Testing theExperimentHubdata set made me realise we still hadn’t thoroughly addressed the “categorical covariate with too many levels” situation. > I have taken care of it in the colData plot with my ‘redundant evaluation’ approach which would benefit the same cleanup as you did before
Kevin Rue-Albrecht (07:04:00): > I’ll have to take care of this also for the dimRed plot, when coloring with such a covariate. Might do that first, so that you can bulk update the two situations
Kevin Rue-Albrecht (07:19:30): > ok, job done for dimRed now (through.process_colorby_choice)
Aaron Lun (07:19:51): > I actually think the plotting commands for all three plots can be centralized into one function.
Aaron Lun (07:20:04): > This function should dispatch according to whether X/Y are covariates or factors.
Kevin Rue-Albrecht (07:20:06): > probably
Kevin Rue-Albrecht (07:21:02): > i guess that’s better left as the last step of refactoring, what do you think?
Aaron Lun (07:22:50): > I think we should do it now, as the colour and brush applications will then easily follow.
Aaron Lun (07:23:05): > Can you give me some code to make horizontal violin plots?
Kevin Rue-Albrecht (07:23:32): > do them vertical and addcoord_flip() +somewhere in theggplotcall
Aaron Lun (07:23:41): > That might not be compatible with brushing.
Kevin Rue-Albrecht (07:24:27): > i’ll check now but i don’t think horizontal violins can be generated directly fromaes()
Aaron Lun (07:25:00): > Or maybe it is compatible, I don’t know.
Aaron Lun (07:25:04): > Haven’t tried.
Kevin Rue-Albrecht (07:25:13): > separate note: last one standing: when a colData with too many levels is used for X axis of the geneExprs plot
Kevin Rue-Albrecht (07:25:29): > i’ll put together a MWE to check the violin story
Aaron Lun (07:25:45): > Just checked out the brushing. YEAH
Aaron Lun (07:26:16): > Except that I think restrict should NOT zoom in; will elaborate on GH.
Kevin Rue-Albrecht (07:26:58): > there you go: > > df <- data.frame( > x = rnorm(100), > y = factor(sample(letters[1:5], 100, TRUE)) > ) > ggplot(df) + > geom_violin(aes(x, y)) > ggplot(df) + > geom_violin(aes(y,x)) + > coord_flip() >
Kevin Rue-Albrecht (07:27:14): > you’ll need to usecoord_flip
Kevin Rue-Albrecht (07:28:10): > from your brushing code, I think you can handle that with something like: > > "brushedPts <- shiny::brushedPoints(all.coordinates[['%s']], > list(xmin=%.5g, xmax=%.5g, ymin=%.5g, ymax=%.5g, > direction='%s', mapping=list(x=***y*** y=***x***)));" >
Kevin Rue-Albrecht (07:28:39): > or maybedirection="yx"instead of"xy", I don’t know enough about brushing at this stage
Kevin Rue-Albrecht (07:29:04): > (the stars were just to highlight the change, not actual code)
Aaron Lun (07:29:06): > It should be okay, it just copies what’s on the shiny object so if shiny is smart enough to handle it, so are we.
Kevin Rue-Albrecht (07:29:36): > yep - i’ll leave you to it then. I still have to prepare a lab meeting for Tuesday
Aaron Lun (07:30:45): > One more thing.
Aaron Lun (07:30:59): > Can you give me some code to plot covariate X/Y?
Aaron Lun (07:31:07): > i.e., two covariates against each other.
Aaron Lun (07:32:29): > I’d like to have circles of varying size on a grid, where each circle represents a combination of X/Y covariates and points are scattered randomly within each circle.
Aaron Lun (07:33:19): > The are of each circle should be proportional to the number of points.
Aaron Lun (07:33:50): > If you give me some code that can plot circles of varying size and center positions, I can do the rest.
Kevin Rue-Albrecht (07:42:55): > something like that? > > df <- data.frame( > X = rnorm(100), > Y = rnorm(100), > Size = runif(100, 1, 2) > ) > ggplot(df) + > geom_point(aes(x = X, y = Y, size = Size), alpha = 0.5) >
Aaron Lun (07:46:10): > That’s close, but I’ll be wanting to overlay points inside the circles. The circles will be like the violin plots.
Kevin Rue-Albrecht (07:47:30): > in that case, I suppose you’ll want to overlay two layers: > * one with the circles proportional to their content > * one with the individual data points all sized identically (setsize=...outside theaescall)
Aaron Lun (07:50:44): > Hm. I’ll just put that in a to-do clause for the time being, I’ll let one of you guys handle it.
Kevin Rue-Albrecht (07:53:32): > ok. > i’m sure you’re thinking of the important caveat that the ‘summary circles’ will need to be pre-computed by ourselves, and even so, that summary circles may: > * overlap one another (how will we deal with data points in the intersect?) > * not overlap a data point that is assigned to them
Aaron Lun (07:56:29): > This is all relatively easy.
Aaron Lun (07:56:39): > A bit of maths should be sufficient.
Kevin Rue-Albrecht (07:58:49): > sounds like you’re on top of this:slightly_smiling_face:i shine more in heuristic than in math
Kevin Rue-Albrecht (08:09:46): > Does the latest commit work for you, with regard to brushing? > I getinvalid 'times' argumentif i turn on brushing from colData plot to dimRed plot. > I’ve just added code locally to take care of the ‘restrict’ feature that you pointed out on GitHub? Restrict shouldnot zoom in anymore.
Aaron Lun (08:10:15): > Okay. How do you incorporate brushing into the violin plot?
Kevin Rue-Albrecht (08:10:15): > However, I don’t want to commit-push until I see it working locally
Kevin Rue-Albrecht (08:10:32): > as a sender or receiver?
Aaron Lun (08:10:35): > Receiver
Aaron Lun (08:11:18): > Well, whatever, I’ll just give it a shot.
Kevin Rue-Albrecht (08:13:19): > well, > ‘restrict’ is the simplest: > both yourgeom_jitterandgeom_violinlayers should now work on their own data usinggeom_jitter(aes(...), data = subset(plot.data, BrushBy))
Kevin Rue-Albrecht (08:14:43): > for the other two modes, you’ll need two layers ofgeom_jitter: > * one for the brushed pointssubset(plot.data, BrushBy)* one for the otherssubset(plot.data, !BrushBy)
Aaron Lun (08:14:50): > Don’t worry about it, I will just refactor and you can handle the rest.
Kevin Rue-Albrecht (08:14:56): > ok no worries
Aaron Lun (08:32:28): > Probably best not to commit anything ATM. Shit is getting real over here.
Kevin Rue-Albrecht (08:36:53): > no worries. as said, i don’t like commiting stuff i haven’t seen working locally first
Aaron Lun (09:30:43): > Refactoring is done. See if you understand what’s going on in the latest commit.
Kevin Rue-Albrecht (09:32:35): > will do, thanks for the update
Aaron Lun (09:32:59): > Note that transmitted brushes from violin plots won’t work properly, as the jitter is not applied to the plot.data but to the plot itself.
Kevin Rue-Albrecht (10:01:49): > huh.@Aaron Lundid you change the behaviour of colData plot on purpose or inadvertently by refactoring when X-axis = None ?
Aaron Lun (10:02:01): > Yes, on purpose.
Kevin Rue-Albrecht (10:02:08): > ok, just checking
Aaron Lun (10:02:17): > Didn’t really see the value of a rank plot if we have a violin.
Aaron Lun (10:02:45): > Anyway, w.r.t. transmitted brushes; what this means is that we will have to extract the x-coordinates from the plot after it is generated to override the plotting coordinates inplot.data.
Kevin Rue-Albrecht (10:02:57): > Fair enough. And more homogeneous behaviour with geneExprs plot, actually
Kevin Rue-Albrecht (10:03:14): > ahh… got you
Kevin Rue-Albrecht (10:05:05): > which reminds me, you said recently that you can’t extract coordinates from the ggplot object returned by the function? how come? data is always stored in thedataslot of ggplot objects
Aaron Lun (10:06:09): > Well, I had thought that we could directly callbrushedPointson the ggplot object, but that turned out to be a fail. So I just stored theplot.datainstead. Perhaps we could avoid the need for overwriting by usingggplot$data.
Kevin Rue-Albrecht (10:06:32): > i think so
Kevin Rue-Albrecht (10:08:15): > also, wrt brushing on violins, considering that the X-jitter is only aesthetic, can’t we just supply the Y-range of the brush input tobrushedPoints? I can’t see a reason why users would only want to brush the left side of a violin for instance… (random subset maybe?)
Kevin Rue-Albrecht (10:08:58): > alternatively, we just need to teach users to brush the full width of the violin to catch all points in the Y-range that they aim for
Aaron Lun (10:10:08): > Well, one can imagine that if you have multiple violins but you only want to capture the top half of the left-most violin…
Kevin Rue-Albrecht (10:11:27): > yep, then brush the full width of the left-most violin over the Y range that you aim for
Kevin Rue-Albrecht (10:12:01): > works for me
Aaron Lun (10:12:15): > datawon’t do it, it doesn’t store the jittered points.
Kevin Rue-Albrecht (10:13:16): > we don’t jitter over the Y axis, so as long as your brush is correct on the Y axis, and wide enough to cover the width of the violin you’re shooting at, it works for me
Kevin Rue-Albrecht (10:13:50): > to convince yourself, make the dimRed plot receive the brush from the geneExprs plot
Kevin Rue-Albrecht (10:14:07): > put the geneExprs X axis todriver_1_s
Aaron Lun (10:14:34): > Do we just need to cover the center of the violin?
Kevin Rue-Albrecht (10:14:40): > and brush the top half of the GN220 violin
Kevin Rue-Albrecht (10:15:06): > you’ll see the right-most cloud of points light up in the dimRed plot
Kevin Rue-Albrecht (10:15:33): > not the centre, the full width of the violin(s) that you’re interested in
Kevin Rue-Albrecht (10:16:11): > got it?
Aaron Lun (10:16:12): > Looks like it works with just the center.
Kevin Rue-Albrecht (10:16:16): > ah
Aaron Lun (10:16:41): > That’s actually alright, because it means you only have to cover the center when you have lots of violins.
Kevin Rue-Albrecht (10:16:52): > yep, you’re right, sorry
Aaron Lun (10:17:13): > Okay, that’s good enough.
Aaron Lun (10:17:19): > Let’s proceed with the rest.
Kevin Rue-Albrecht (10:17:35): > Exactly. > And I don’t want to hear users complain that they can’t do multiple selection (if they don’t want a violin in the middle)
Aaron Lun (10:19:25): > Cool. Integrate the brushing with the violin plots and let’s merge this branch.
Kevin Rue-Albrecht (10:20:02): > ok. At the moment, I’m putting back in the code to maintain axis range upon restrict
Kevin Rue-Albrecht (10:20:28): > in theory, it’s two lines. can’t wait to test
Aaron Lun (10:20:31): > In.scatter_plot(), I presume.
Kevin Rue-Albrecht (10:20:43): > yup
Aaron Lun (10:21:02): > The nature of the restriction will probably differ with violins - you’ll need empty spaces for missing factors.
Kevin Rue-Albrecht (10:21:51): > do we want to keep empty spaces for them? ggplot has an optiondroplevelsfor this kind of scenario
Aaron Lun (10:22:07): > Yes, to be consistent with what restirct does elsewhere.
Kevin Rue-Albrecht (10:22:26): > ok then,droplevels=FALSEit is
Aaron Lun (10:23:18): > Will character vectors need to be coerced to factors for this to work?
Kevin Rue-Albrecht (10:24:08): > axis range preservation for scatterplot: done
Kevin Rue-Albrecht (10:24:26): > ggplot automatically coerces character vectors to factors
Aaron Lun (10:24:31): > okay, cool.
Kevin Rue-Albrecht (10:24:53): > (hence my.nlevelswrapper around the realnevels)
Kevin Rue-Albrecht (10:25:58): > niiiiiicegeom_quasirandom!
Kevin Rue-Albrecht (10:28:58): > btw, can you help me figure out in the vignette, how I couldrunTSNEefficiently on theExperimentHubexample? I’ve tried reducingn_dimred, but my laptop still takes ages. Something I’m missing?
Aaron Lun (10:29:46): > How many cells? I can run 4000 on my desktop in 1-2 minutes.
Kevin Rue-Albrecht (10:30:42): > > > dim(eh1) > Features Samples > 23368 7706 >
Aaron Lun (10:31:04): > Run irlba outside and feed the PCs intoRtsne .
Kevin Rue-Albrecht (10:31:28): > data set is “RNA-Sequencing and clinical data for 7706 tumor samples from The Cancer Genome Atlas”
Kevin Rue-Albrecht (10:45:37): > @Kevin Rue-Albrechtuploaded a file:ExperimentHub: EH1 - File (PNG): ExperimentHub: EH1
Kevin Rue-Albrecht (10:45:39): > thanks
Aaron Lun (10:53:57): > What’s the necessary code to setylimandxlimin ggplot?
Kevin Rue-Albrecht (10:57:48): > those are functions rather than arguments in ggplot, basically, you add a statementxlim(15, 20) +in your ggplot call
Kevin Rue-Albrecht (10:57:54): > that’s one way
Aaron Lun (10:57:57): > Good enough.
Kevin Rue-Albrecht (10:58:20): > i tend to preferscale_x_continuous(limits=c(15,20), ...) +
Kevin Rue-Albrecht (10:58:37): > because you can simultaneously define more than just the limits
Kevin Rue-Albrecht (10:59:01): > btw, i’m almost done for the violin brushing
Kevin Rue-Albrecht (11:01:45): > … or were you trying to to somthing to the violin axis ranges?
Aaron Lun (11:01:52): > No, I’m implementing the zooming.
Kevin Rue-Albrecht (11:02:18): > i’ve got the brushimg sorted, but i’m still in the situation were restrict zooms in the violin
Aaron Lun (11:02:46): > I’m just dealing with the scatter plots for now, the violins should be untouched.
Kevin Rue-Albrecht (11:03:46): > Cool. Facing the problem that we talked about ealier: preserving axis ranges/ticks whether X, Y or both are discrete or continuous
Kevin Rue-Albrecht (11:03:50): > arf
Kevin Rue-Albrecht (11:04:08): > you know what, i’ll push already as it is, so that I have a checkpoint to work from
Kevin Rue-Albrecht (11:19:02): > ok.. i’ve got an implementation.. which breaks if the brush is empty:confused:
Kevin Rue-Albrecht (11:21:29): > doesn’t cause a full blown app crash, just a warning message in the plotting panel and the console.. but still, i’m not particularly happy
Kevin Rue-Albrecht (11:29:47): > oh… that is so cool: if you color by CancerType the dimRed plot in TSNE coordinates of the ExperimentHub data set that I demo in the vignette, it matches nicely the TSNE clusters
Kevin Rue-Albrecht (11:30:38): > they have so many covariates in that data set, that it took me time to figure out one that was matching the tSNE distribution
Kevin Rue-Albrecht (11:30:43): > anyway
Kevin Rue-Albrecht (11:31:26): > it’s not single cell, but i think it’s a cool demo of a relatively big data set that the app handles surprisingly smoothly
Kevin Rue-Albrecht (11:34:05): > @Kevin Rue-Albrechtuploaded a file:@Federico Marini: for your presentation@Federico Marini: for your presentation
Aaron Lun (11:38:06): > Got a slightly buggy zooming working now.
Kevin Rue-Albrecht (11:39:53): > someone’s got to put the foundation stone to every house
Aaron Lun (11:43:32): > Realized I was doing it stupid
Aaron Lun (11:43:44): > It’ll probably take an hour to fix, let me think about it.
Kevin Rue-Albrecht (11:45:48): > No worries.
Aaron Lun (12:00:48): > irlba should go into suggests.
Aaron Lun (12:03:36): > zooming infrastructure has been built. See lines 297-305 ofplotting.Rto see how it is applied to scatter plots.
Kevin Rue-Albrecht (12:04:57): > @Kevin Rue-Albrechtuploaded a file:just another use example of use case - File (PNG): just another use example of use case
Kevin Rue-Albrecht (12:05:31): > irlba done
Aaron Lun (12:07:17): > I’m done for the day. See if you can get the zooming to work for the violin plots, and just merge the branch. We’ll discuss how to handle horizontal violin plots and double X/Y factors later.
Kevin Rue-Albrecht (12:10:04): > Ok. I’m pretty beat for the day as well. I’ll see what I can do, or it might tomorrow
Aaron Lun (12:10:06): > Will_to_work==0is evaluating toTRUE.
Kevin Rue-Albrecht (12:10:27): > not usingidentical?
Kevin Rue-Albrecht (12:10:42): > you daredevil:wink:
Aaron Lun (12:10:59): > I’m out of control, man.
Aaron Lun (12:11:10): > or maybe my will to work is integer.
Kevin Rue-Albrecht (12:12:48): > Anyway, really good progress today.
Kevin Rue-Albrecht (12:28:14): > @Kevin Rue-Albrechtuploaded a file:just to give everyone a quick idea of the zoom feature - File (PNG): just to give everyone a quick idea of the zoom feature
Federico Marini (12:28:49): > just got back from the zoo
Federico Marini (12:29:06): > and about the screenshot. looks as cool as we hoped:slightly_smiling_face:
Federico Marini (12:29:10): > well done
Federico Marini (12:29:40): > one thing catching up on previous messages: careful with using xlim and ylim: ggplot CUTS out the ones excluded
Federico Marini (12:29:54): > to really “zoom” we should use coord_cartesian
Aaron Lun (12:30:08): > Note sure what you mean by cuts out.
Kevin Rue-Albrecht (12:30:14): > same here
Federico Marini (12:30:30): > well…
Federico Marini (12:30:50): > the points are cut out “even more” than what the limits you specify
Federico Marini (12:31:16): > I’m up for dinner now, later I can send you a link/MRE demoing the two
Kevin Rue-Albrecht (12:31:19): > i mean, we only ever use xlim to “preserve” larger axis ranges than the ‘restricted’ data would leave
Kevin Rue-Albrecht (12:31:35): > we never uses xlim to zoom in and cut out points
Aaron Lun (12:32:37): > I’m currently using thescale_xlim_continuousto zoom in.
Aaron Lun (12:32:54): > Don’t fully understand the difference, but feel free to edit; it’s on the lines I mentioned above inplotting.R.
Kevin Rue-Albrecht (12:34:21): > ahh i think i got what Federico meant, from reading?coord_cartesian
Kevin Rue-Albrecht (12:34:42): > “Setting limits on the coordinate system will zoom the plot (like you’re looking at it with a magnifying glass), and will not change the underlying data like setting limits on a scale will.”
Kevin Rue-Albrecht (12:36:22): > i think that applies particularly for violin plots in our case: > If you ‘cut out’ points that are outside the axis limits, you’ll change the shape of the violin somewhat. > If you only want to “zoom in”, you need to keep all the data points, and zoom in the cartesian coordinates
Aaron Lun (12:36:40): > Okay… I guess this sounds like what we want.
Kevin Rue-Albrecht (12:38:15): > with that said, I’m happy to stay in the discussion today, but I can’t think about writing more code until tomorrow
Aaron Lun (12:44:46): > I’m done. I’m going home.
Aaron Lun (12:45:48): > But I guess I’ll fix your last bug report before that.
Aaron Lun (12:47:10): > Give me a MWE?
Aaron Lun (12:48:25): > I think I see it.
Kevin Rue-Albrecht (12:48:57): > you mean what to click?
Kevin Rue-Albrecht (12:49:23): > just open the app, and in colData plot selectdriver_1_sfor the X axis
Aaron Lun (12:49:26): > Yep.
Kevin Rue-Albrecht (12:50:02): > i think if you solve that one, you solve them all, as Sauron said
Aaron Lun (12:56:38): > Pretty sure it’s due to your lines 360-361 inplotting.R; commenting them out makes it work again.
Kevin Rue-Albrecht (12:57:23): > i’ll have a look
Aaron Lun (12:58:31): > Okay, I’m actually going home now.
Kevin Rue-Albrecht (13:00:11): > Ok, well i’ll just comment them out then. It was not a pretty implementation anyway
Kevin Rue-Albrecht (13:00:20): > catch up later
Kevin Rue-Albrecht (13:06:50): > done.
Federico Marini (13:19:38): > So, here would be the example
Federico Marini (13:19:49): > > df <- expand.grid( > X = seq(-3,3,by=0.5), > Y = seq(-3,3,by=0.5) > ) > ggplot(df) + geom_point(aes(x = X, y = Y)) > ggplot(df) + geom_point(aes(x = X, y = Y)) + xlim(-2.9,2.9) > ggplot(df) + geom_point(aes(x = X, y = Y)) + coord_cartesian(xlim = c(-2.9,2.9)) >
Federico Marini (13:20:11): > the second still includes the coords where the points would be but cuts them out
Federico Marini (13:20:18): > like a subsetting
Federico Marini (13:20:31): > while nr. 3 does more like a “pure zoom”
Federico Marini (13:21:01): > (nr.2 also says in the console that indeed some points are removed, i.e. the 26 rows…)
Federico Marini (13:23:28): > @Kevin Rue-Albrecht: thank you for the use case scenario. Do you all in the<!channel>also have an impactful example for the single cell case?
Kevin Rue-Albrecht (13:25:47): > What sort of impactful example?
Federico Marini (13:27:06): > a single cell dataset, where I know that e.g. this subset of cells is interesting because of [reason 1], and it is possible to identify them both in the redDim plot as well by checking the expression of [marker gene]
Kevin Rue-Albrecht (13:30:37): > Well initially I meant to look for a single cell data set in the experimenthub, but then I spotted that big TCGA data set that was cool too
Kevin Rue-Albrecht (13:32:59): > If you can find a single cell data set in the ExperimentHub, that’ll help make the Bioc core team happier as well as facilitating reproducibility for us
Kevin Rue-Albrecht (13:40:57): > there: > > library(ExperimentHub) > ehub <- ExperimentHub() > View(data.frame(mcols(query(ehub, "single"))))` >
Kevin Rue-Albrecht (13:44:03): > argh.. actually, it’s only the 1.3M neurons in there so far
Federico Marini (14:23:31): > So you got the GSE62944 based on the paper from the piccolo lab
Federico Marini (14:23:35): > ok
Federico Marini (15:09:33): > Just as a check: we agreed in the conference call on the most urgent features to implement to have the operational version of the app. > I noted down: > - brushing ->@Aaron Lun- reproducibility ->@Federico Marini- pimping up plots ->@Charlotte Soneson&@Kevin Rue-Albrechtminor ones: > - introjs based tour > - genes selectable also without need of linked table > - give code in most usable way to the user > - thoughts on the manuscript > - cosmetics on buttons & co
Federico Marini (15:09:45): > Seems to me we are at good pace
Federico Marini (15:11:07): > I’ll check also the open issues and report here again
Aaron Lun (15:52:01): > All hands to the violin plots. Get them playing nice with the brushing and zooming, and we’re good to release an alpha version.
Federico Marini (15:52:41): > Good to hear!
Federico Marini (15:52:51): > I can finalize the introjs tour
Kevin Rue-Albrecht (15:56:17): > I’ll look into the violins again. I’ve mustered a bit of will over dinner again
Federico Marini (15:57:46): > Is something with acliprintegration to copy to clipboard the text an overkill? Something that does not force the user to have clipr installed
Kevin Rue-Albrecht (15:58:05): > clipr for what? the tracker?
Federico Marini (15:58:14): > -> as Kevin said, click, ctrl-A ctrl-C just works…
Federico Marini (15:58:18): > yes
Kevin Rue-Albrecht (15:59:14): > i solemnly swear to answer all user contacts complaining about the lack of a ‘copy to clipboard’
Kevin Rue-Albrecht (16:00:22): > (please don’t write a cron job to send me emails about it)
Kevin Rue-Albrecht (16:04:10): > @Federico Marini, think of branching from the side-branch when it is as advanced as this one. that’ll help minimise conflicts (just in case)
Federico Marini (16:05:13) (in thread): > I did it from master only because I knew it would have been one line
Kevin Rue-Albrecht (16:07:26) (in thread): > Ok right. I didn’t check at the code change itself. I just know myself to think i’ll change one thing, and next thing i know, RStudio reindented all the file I worked on.
Kevin Rue-Albrecht (16:23:57): > Brushing in place for violins. Not sure why the zooming doesn’t work as for dimRed though (Aaron implemented the first one, and I haven’t bothered looking how yet)
Kevin Rue-Albrecht (16:24:39): > I’ll swap thescale_*(limits=)for cleanercoord_cartesiancalls now
Kevin Rue-Albrecht (16:27:25): > nevermind: found Aaron’s code
Kevin Rue-Albrecht (16:45:31): > <!channel>i confirm thatscale_*calls are now extinct, and that violin plots can be zoomed in
Kevin Rue-Albrecht (16:45:45): > how are we doing on that checklist ?
Kevin Rue-Albrecht (16:48:24): > @Federico Marini: on the ‘cosmetics of buttons’, i just remembered that the up/down arrows to move plots around can only be clicked in their leftmost part. Is that the case for everyone or just my RStudio viewer/Safari? Can something be done about it? adding a space character or something?
Kevin Rue-Albrecht (16:51:24): > it’s the only remaining thing on the checklist that I can think of
Kevin Rue-Albrecht (16:55:01): > ah no, one last thing of the to-do list that Aaron will recognise:ARGHH!
Federico Marini (17:04:12) (in thread): > I also ran into that. Need to check it on othr browsers.
Federico Marini (17:04:47) (in thread): > I like them as compact as they are, but as I said to Charlotte already, I am no js wizard
Federico Marini (17:05:04) (in thread): > and buttons work almost too nicely in shiny to evade them:stuck_out_tongue:
Kevin Rue-Albrecht (17:05:45) (in thread): > absolutely no worries
Kevin Rue-Albrecht (17:06:22) (in thread): > in fact, it’s soooo satisafying to nitpick about tiny things like this: means that we’re basically there
Kevin Rue-Albrecht (17:51:27): > off to sleep now
2018-01-22
Kevin Rue-Albrecht (03:17:27): > @Aaron LunI think I can give you a ggplot base plot to tweak for the ‘ARGH’ scenario
Kevin Rue-Albrecht (03:18:24): > I’m just on my way to work now
Kevin Rue-Albrecht (03:41:42): > Might post it in the#randomchannel too, but still more relevant for us: > the date of the “Oxford Single Cell Symposium 2018” has been announced recently:https://talks.ox.ac.uk/talks/id/1c7d836c-855b-4b86-9243-acdcd66bc727/ “Speakers so far confirmed are: Wolf Reik (Babraham Institute); Ido Amit (Weizmann Institute of Science)”
Aaron Lun (04:08:43): > I am dealing with the ggbeeswarm problem today.
Kevin Rue-Albrecht (04:14:35): > ok - i have 99% implemented a draft of the ARGH scenario. I’m scaling the area of dots organised in grid by the number of data point within each combination. > Only issue is to define the scale to display in the legend. Proportions rarely go to 1, generally stay < 50% so half the scale is ‘generally’ useless
Kevin Rue-Albrecht (04:15:26): > I’ll find a reasonable scale, and ping it over to you for further tweaking
Aaron Lun (04:16:12): > WE shouldn’t need the sizes of the circles in the legend; this would be analagous to having widths of the violin plots in the legend.
Aaron Lun (04:16:38): > Let me fix the quasirandom before you commit. Or make sure it’s in a different function.
Kevin Rue-Albrecht (04:18:11): > do you mean I can just hide the legend of circle sizes?
Aaron Lun (04:18:54): > Yeah.
Kevin Rue-Albrecht (04:18:56): > let me know when you commit, then i’ll merge, push with the legend so you can see how bad it looks, and then i’ll remove the legend in the next commit
Kevin Rue-Albrecht (04:50:18): > does ggbeeswarm also involved looking at the horizontal violins? I may have thought of a solution. testing soon
Aaron Lun (04:52:28): > We won’t be using ggbeeswarm anymore, commit pending.
Kevin Rue-Albrecht (04:52:36): > awww
Kevin Rue-Albrecht (04:52:57): > i had never used it before, but i started liking that guy
Kevin Rue-Albrecht (05:04:20): > ok, i got the layout right, just need to throw the brushing and zooming back in
Aaron Lun (05:07:32): > violin scatter is fixed, assuming that ggplot converts factors into x-positions with a simpleas.integer(as.factor(x)).
Aaron Lun (05:07:46): > which it seems to do (and that’s what shiny assumes that ggplot does, anyway).
Aaron Lun (05:09:24): > Horizontal may be as simple as doing a coord_flip and hoping that shiny is smart enough to handle it.
Kevin Rue-Albrecht (05:10:10): > exactly,coord_flip, with a twist:
Kevin Rue-Albrecht (05:11:03): > you’ll have to swap the aestheticsaes(x = Y, y = X), so thatcoord_flipswaps them back in their rightful place
Kevin Rue-Albrecht (05:12:12): > i was adding anflip=TRUE|FALSEargument tobuild_aes(), but things got messy, so I’m back to focusing on the brushing and zooming of ‘two discrete covariates’ scenario
Kevin Rue-Albrecht (05:13:25): > I’d like to finish and push my update before either of us gets working on the flip thing, to minimise risk of code conflicts
Aaron Lun (05:14:11): > I’m not doing it, so it’s all yours. Have a look at how I handled the violin plot pre-processing if it helps.
Kevin Rue-Albrecht (05:16:00): > ok cool
Kevin Rue-Albrecht (05:19:23): > you removed irlba ?
Aaron Lun (05:19:27): > It’s in suggests.
Aaron Lun (05:19:35): > Why would it be in Imports?
Kevin Rue-Albrecht (05:20:39): > arf, my mistake, blind copy-paste i guess
Aaron Lun (05:21:37): > In any case, I think we need to rethink our Restrict strategy
Aaron Lun (05:21:54): > To demonstrate, set up three column data plots with Y-axis = NREADS and x-axis = driver
Aaron Lun (05:22:17): > Have plot 1 transmit to plot 2 , and have plot 2 transmit to plot 3.
Aaron Lun (05:22:29): > Set plot 2 to restrict.
Aaron Lun (05:23:09): > Brush the middle violin in plot 1, and brush the (now missing) first violin in plot 2.
Aaron Lun (05:23:20): > You’ll see the first violin in plot 3, even though it shouldn’t show up at all.
Aaron Lun (05:23:37): > In short, we need to do a hard subset ofplot.datain cases where restriction is specified.
Aaron Lun (05:24:03): > This is true of the scatter plots as well.
Aaron Lun (05:24:24): > I need to do something else ATM, but will be back onto this.
Kevin Rue-Albrecht (05:24:57): > ok, can’t check it immediately as i’ve got uncommited changes, but i’ll keep it in mind
Kevin Rue-Albrecht (05:25:11): > really got to finish my lab meetings slides too, today
Kevin Rue-Albrecht (05:25:26): > anything to postpone it is attractive:wink:
Kevin Rue-Albrecht (05:28:22): > PS: giving ourselves the code of the plot is so AWESOME for testing and debugging:pray:
Aaron Lun (05:48:16): > I wonder whether it is reasonable to expect downstream brushes to stay the same when upstream brushes change.
Kevin Rue-Albrecht (06:05:01): > maybe some resetting/clearing should be placed in reasonable places
Aaron Lun (06:06:29): > Ideally they should always be cleared. But this would make it painful for chained plots if you changed the top-level brush and had to re-do all the bottom level brushes as well.
Aaron Lun (06:06:56): > I’ll have to think about how best to implement this (to preserve brush memory and whatnot).
Aaron Lun (06:08:33): > Meanwhile, let’s reset all brushes to avoid weird shit happening.
Kevin Rue-Albrecht (06:11:14): > i’ve just pushed an update that lays out the bi-categorical plots, however brushing and zooming are still disabled
Kevin Rue-Albrecht (06:12:56): > i just need do something else, i’ll be right back to finish it up
Kevin Rue-Albrecht (06:58:44): > woot woot - polishing the brushing for bi-categorical plot
Kevin Rue-Albrecht (07:09:03): > tadaaaaa
Kevin Rue-Albrecht (07:09:20): > zooming and brushing implemented for bi-categorical plots
Kevin Rue-Albrecht (07:09:24): > feel free to adjust
Kevin Rue-Albrecht (07:16:48): > I’ve started feeling ashamed that iSEE writes cleaner ggplot calls than I do myself:grimacing:
Aaron Lun (07:40:35): > Looks good; I’ll make some changes (the encircling cirle should be colorless, and the points should lie within the circle).
Aaron Lun (07:42:21): > How can we directly control the circle radius?
Kevin Rue-Albrecht (07:42:37): > it’s thesizeaesthetic
Kevin Rue-Albrecht (07:42:47): > ahh wait
Kevin Rue-Albrecht (07:43:16): > look at?scale_radius
Aaron Lun (07:43:27): > Okay, will fiddle around with it; stand by.
Kevin Rue-Albrecht (07:43:39): > over to you
Kevin Rue-Albrecht (07:47:56): > i think this merge will be worth a version bump, btw:slightly_smiling_face:
Aaron Lun (08:17:47): > Well, the good news is that I’ve got the points where I want them.
Aaron Lun (08:18:07): > The bad news is that circles don’t appear as circles due to the differences in scaling between axes.
Aaron Lun (08:18:36): > Might be better to use rectangles, then.
Aaron Lun (08:18:43): > Or squares.
Aaron Lun (08:18:56): > How do I draw a rectangle in ggplot?
Kevin Rue-Albrecht (08:22:14): > geom_pathi think
Charlotte Soneson (08:22:14): > geom_rect()orgeom_tile(), depending on what you want to specify:http://ggplot2.tidyverse.org/reference/geom_tile.html
Kevin Rue-Albrecht (08:22:18): > or that
Aaron Lun (08:23:46): > You know what, I’ll just push and let you deal with it.
Aaron Lun (08:24:10): > Just replace the circles with squares, usingRadius(effectively half the square side length).
Aaron Lun (08:26:10): > Otherwise we would have to adjust the sampled point coordinates according to how the circles appear on the plot, which would be a real pain in the ass.
Kevin Rue-Albrecht (08:42:56): > @Kevin Rue-Albrechtuploaded a file:concept-logo-1.pdf - File (PDF): concept-logo-1.pdf
Kevin Rue-Albrecht (08:43:47): > :innocent:
Charlotte Soneson (08:52:51): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-22 at 14.51.41.png and commented:geom_tile()can do something like this, is that what you intended (the side lengths are2*Radius)? - File (PNG): Screen Shot 2018-01-22 at 14.51.41.png
Kevin Rue-Albrecht (08:54:17): > That looks cool to me. Just with a little less vertical jitter.
Aaron Lun (08:55:29): > Note the bug fix.
Charlotte Soneson (08:55:33): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-22 at 14.54.50.png and commented: I didn’t do anything to the jitter here. This is the corresponding plot with the circles, for comparison. - File (PNG): Screen Shot 2018-01-22 at 14.54.50.png
Aaron Lun (08:55:59): > I won’t touch gridplot for the time being, but I’ve fixed restriction everywhere else.
Charlotte Soneson (08:57:02): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-22 at 14.56.42.png and commented: Ok, with the bug fix… - File (PNG): Screen Shot 2018-01-22 at 14.56.42.png
Aaron Lun (08:57:05): > Yeah
Aaron Lun (08:57:10): > that looks pretty good.
Aaron Lun (08:57:31): > Can we make the boxes colourless with outline only? Mimics what’s in the violin plots.
Aaron Lun (08:58:36): > Also,@Kevin Rue-Albrecht; note the use ofexpand=FALSEwhen zooming, which ensures a faithful zoom.
Kevin Rue-Albrecht (08:59:46): > yeah i saw it, yet didn’t really think about the reason why, I just trusted you. I get it now
Aaron Lun (09:00:33): > Alternative to the rectangles; if you want to be really creepy, we could identify the outer surface of the scattered plots and draw a line throughout it. It’ll probably even look like a cell.
Kevin Rue-Albrecht (09:01:21): > that’s how i pictured usinggeom_path
Kevin Rue-Albrecht (09:02:18): > but i’m far too lazy to implement that
Aaron Lun (09:02:23): > https://blogs.oii.ox.ac.uk/bright/2013/06/10/convex-hulls-with-ggplot/ - Attachment (Jonathan Bright): Convex Hulls with ggplot > I found this code buried in an old google group discussion which I thought I would repost. As with everything ggplot wise hat tip to the incredible Hadley Wickham. Often it’s nice to break do…
Aaron Lun (09:02:32): > But let’s take it easy. Bigger priorities right now.
Kevin Rue-Albrecht (09:03:21): > can someone addcoord_cartesian``iSEE-pkg.R?
Charlotte Soneson (09:03:55): > In the end, the convex hull of jittered points may look similar to a rectangle anyway, since it is not allowed to go “inwards” anywhere…
Aaron Lun (09:06:59) (in thread): > Feel free.
Aaron Lun (09:07:32): > If we were able to do this, we would sample from a more circular shape, and allow the hull to approximate it.
Aaron Lun (09:07:46): > I mean, let’s not think about this yet, let’s deal with the other major issues.
Aaron Lun (09:08:04): > I’m starting to regret mentioning this at all.
Aaron Lun (09:08:10): > :slightly_smiling_face:
Kevin Rue-Albrecht (09:08:10): > the vignette EH1 example produces an “Error: ‘x’ contains missing values” that I’m just tracking down now, only for the colData plot
Kevin Rue-Albrecht (09:08:24): > haven’t encountered it for the allen data set yet
Aaron Lun (09:09:12): > There is amergein the coldata for the gridplot, that may be the cause.
Aaron Lun (09:09:17): > Maybe. I’m not sure.
Kevin Rue-Albrecht (09:10:04): > wouldstat_ellipsebe too approximate for this@Aaron Lun?https://ropensci.github.io/plotly/ggplot2/stat_ellipse.html
Aaron Lun (09:10:47): > That might be okay, if I sample from a circle in the first place.
Charlotte Soneson (09:12:18): > I’ll push the rectangles in the meanwhile…
Kevin Rue-Albrecht (09:12:52): > I’ve pushed thecoord_cartesianimport.
Charlotte Soneson (09:14:21): > Usinggeom_tile()generates weird warnings thatheightandwidthare unknown, but everything seems to be working fine and apparently this is “more complicated than expected”:https://github.com/tidyverse/ggplot2/issues/1904 - Attachment (GitHub): Regression with width aesthetic in documentation · Issue #1904 · tidyverse/ggplot2 > A warning is omitted when using the width aesthetic, but still seems to produce identical plots. Example from ggplot2 documentation for geom_tile: 2.1 Documentation, Dev Documentation library(ggplo…
Kevin Rue-Albrecht (09:16:24): > well, if it works, just push the code, and we can all try to help track down, i’d say
Charlotte Soneson (09:16:29): > It is pushed
Kevin Rue-Albrecht (09:20:20): > keep up, Kevin, keep up ^^
Kevin Rue-Albrecht (09:26:56): > hm .. puzzle
Kevin Rue-Albrecht (09:27:22): > mah.. if Hadley can’t solve, I’d be bold to keep trying
Aaron Lun (09:27:22): > MWE
Aaron Lun (09:27:33): > What are we talking about - the mixxing values?
Kevin Rue-Albrecht (09:28:18): > the width/height warning ofgeom_tile
Kevin Rue-Albrecht (09:28:31): > you just need to select a discrete covariate for the colData plot
Kevin Rue-Albrecht (09:28:59): > nothing wrong with the UI, just some frustrating warnings on the console
Charlotte Soneson (09:30:38): > seems to be because of the different ways to specify rectangles. I quickly triedgeom_rect, but it doesn’t play well with the factors (you need to define the coordinates of the corners explicitly)
Kevin Rue-Albrecht (09:31:27): > i think we’re fine, we can just put the link above in the FAQ section of the vignette if anyone asks
Aaron Lun (09:31:43): > We can do the explicit coordinates.
Aaron Lun (09:31:47): > That’s not too hard.
Aaron Lun (09:32:01): > I mean, we have the positions and widths already.
Charlotte Soneson (09:32:24): > Yeah, just have to explicitly convert the factors to the corresponding numeric values
Kevin Rue-Albrecht (09:32:24): > Following on the GitHub issue from yesterday, I’ll probably add an FAQ about Seurat object. I see this one coming from far away
Aaron Lun (09:34:18): > However the vignette is being written, make sure we go one line per sentence, rather than have huge one-line paragraphs.
Kevin Rue-Albrecht (09:34:53): > yep - i’m bad with manuscripts in that way too ^^
Aaron Lun (09:45:59): > Note that currently, indenting is only done correctly when the previous line ends with “+”. Otherwise the code tracker looks a bit funny.
Kevin Rue-Albrecht (09:49:15): > I liked that. Are there still particular code generating places that need tidying up? I’ll open a session myself to check
Aaron Lun (09:50:53): > Oh, and it needs to end with a “+”. Literally. No spaces.
Kevin Rue-Albrecht (09:51:14): > hahah you got caught in that one too? ^^
Kevin Rue-Albrecht (09:52:36): > @Federico Marini: can we control the size (height, in particular) of the code editor/modal?
Aaron Lun (10:00:24): > Why is size=1.5 in unbrushed scatter plot?
Kevin Rue-Albrecht (10:03:19): > Dunno. I don’t remember setting this
Federico Marini (10:03:27) (in thread): > Yes we can
Federico Marini (10:03:37) (in thread): > I gave a default of 800px
Federico Marini (10:03:40) (in thread): > we can adjust it
Kevin Rue-Albrecht (10:04:32) (in thread): > Ok. On my screen (MacbookPro and iMac) it looks a bit small
Kevin Rue-Albrecht (10:04:47) (in thread): > But I don’t want to cause trouble to other users
Federico Marini (10:05:48) (in thread): > It can be made bigger, don’T worry
Aaron Lun (10:18:30): > Did you track down the EH1 problem?
Aaron Lun (10:18:56): > :wq
Aaron Lun (10:18:59): > whoops
Kevin Rue-Albrecht (10:24:13): > not yet, sorry
Kevin Rue-Albrecht (10:24:40): > actually, I wrote the print statements for debug, but got sidetracked
Aaron Lun (10:27:33): > I will merge the monster once the CI passes.
Kevin Rue-Albrecht (10:28:35): > ok
Kevin Rue-Albrecht (10:29:11): > mind if I extend the sliders to a minimal plot/panel width of 2 ?
Aaron Lun (10:29:28): > I don’t think you can see anything with that.
Kevin Rue-Albrecht (10:29:35): > I never thought I’d complain about this, but the iMac screen is too large
Kevin Rue-Albrecht (10:29:49): > I have a 16:9 view of my tSNE
Kevin Rue-Albrecht (10:30:19): > https://www.dropbox.com/s/am0kznhmcfey42m/Screenshot%202018-01-22%2015.30.10.png?dl=0
Kevin Rue-Albrecht (10:30:58): > maybe min=3, actually
Kevin Rue-Albrecht (10:31:31): > (but still a default of 4, that’s always been a good fit for my laptop)
Aaron Lun (10:34:33): > Well, I guess so. We’ll want to provide options to tune the row height as well, but this is later.
Aaron Lun (10:35:06): > Real tower of hanoi stuff
Kevin Rue-Albrecht (10:35:22): > i don’t if that changed, but I had to hack around to make row height a parameter inTVTB
Kevin Rue-Albrecht (10:36:21): > i never found a way to control it within an app, so I let the user set it as an argument of the function that launched the app
Aaron Lun (10:36:34): > That might be easiest.
Aaron Lun (10:36:53): > Well, I mean, we can control it, but we’ll have to re-render the entire UI.
Aaron Lun (10:37:10): > Which is fine.
Kevin Rue-Albrecht (10:50:43): > just to be clear about issue ‘Colormap for continuous variables’: > Feel free to define/choose a reasonable default colormap. > What I meant is that I’m interested in setting up the infrastructure for users to supply their own favorite colormaps.
Aaron Lun (10:51:02): > Yes, that’s what I thought as well.
Aaron Lun (10:51:26): > Allow them to specify a factor colormap and a continuous colormap.
Kevin Rue-Albrecht (10:51:46): > I know we said that users can always take the code and edit it in an interactive session, > but I know it can make a world of difference for them to navigate the data set with their own familiar colorscales
Kevin Rue-Albrecht (10:59:50): > I’ll rerunroxygenbtw: documentation out of sync withiSEE-pkg.RPlease hold off any change as this tend to reflow a lot of lines
Kevin Rue-Albrecht (11:01:53): > bah. apparently not this time. just updated theNAMESPACE
Aaron Lun (11:03:19): > These boxes were inspired.
Aaron Lun (11:03:33): > It makes so much sense; you can compare heights across rows, or widths across columns.
Federico Marini (11:03:53): > I am almost done with bringing further substance to the introjs tour
Federico Marini (11:05:30): > incoming soon in theextended_tourbranch
Federico Marini (11:06:14): > with changes that should not affect at all whatever other news came in for the brushing part
Federico Marini (11:08:59): > if I read correctly, now the whole brushing is already in master
Federico Marini (11:09:06): > so I can safely merge there
Kevin Rue-Albrecht (11:11:01): > you guys don’t have error messages instead of panels, about ggplot functions that the app can’t find? > for me it’s all the ones we recently added. > weird thing is the desktop has the issue, while the laptop doesn’t > so basically, I test on the desktop, code on the laptop, push, pull, and keep going:laughing:
Kevin Rue-Albrecht (11:12:47): > ok, we’re good now
Kevin Rue-Albrecht (11:14:05): > PS: min width of 2 is perfect now, for me
Kevin Rue-Albrecht (11:14:15): > https://www.dropbox.com/s/hlfnhueoguqj51q/Screenshot%202018-01-22%2016.14.11.png?dl=0
Aaron Lun (11:16:01): > coord_flip almost but not quite right.
Aaron Lun (11:16:36): > zooming doesn’t work, and the y boundaries aren’t being preserved when restricting.
Kevin Rue-Albrecht (11:17:26): > @Kevin Rue-Albrechtuploaded a file:possible demo data set - File (PNG): possible demo data set
Kevin Rue-Albrecht (11:17:51) (in thread): > ok, let me know if you do them all, or if you want to send some my way
Kevin Rue-Albrecht (11:18:23): > @Federico Marini: I’ll have a meeting with the collaborators who generated this 10x data set, see if I can send it your way for the talk
Kevin Rue-Albrecht (11:19:03): > it’s about to be submitted, so I’m hoping they’ll be happy to show off
Kevin Rue-Albrecht (11:19:44) (in thread): > let’s make this a thread
Kevin Rue-Albrecht (11:21:31): > @Aaron Lun: for this data set, I’ve just dumped the relevant slots of the Seurat object in the appropriate places of an SCE. Again I know your point is not that it’s particularly difficult, only a matter of separating responsibilities between packages developers
Aaron Lun (11:32:57) (in thread): > Fixed.
Kevin Rue-Albrecht (11:33:44) (in thread): > I know you don’t know them, but feel free to send me arguments to help me convince them that’s it’s harmless to them. I’m wondering myself what harm it could cause them: it’s a good data set
Kevin Rue-Albrecht (11:35:05) (in thread): > (who the audience will be, etc.)
Kevin Rue-Albrecht (11:35:48) (in thread): > Ok. I’m building again themasterbranch ATM, and I’ll investigate that EH1 bug
Aaron Lun (11:38:33): > <!channel>I think that all of the major structural features for the alpha release are implemented. (Brush memory is explicitly not going in yet, this requires a major think of how to incorporate that into the memoryand trigger replotting of dependent plots.) I will leave it to everyone to implement their own little improvements.
Aaron Lun (11:39:18): > I will be implementing defaults for the gene tables sometime this week, but that’s it from me.
Aaron Lun (11:40:55): > @Charlotte Sonesondid you have a shiny server?
Charlotte Soneson (11:42:14): > Yes, we have a shiny server set up
Aaron Lun (11:42:56): > Could you test it withiSEEinapp.R, a la issue 64?
Charlotte Soneson (11:45:13): > Yep. I’m on the train now and away most of the evening, but I’ll try later or tomorrow.
Aaron Lun (11:45:18): > Okay.
Kevin Rue-Albrecht (11:45:40): > Different scenario, but we have an RStudio server over here. And I’ve been successfully running the app as a second tab in my browser
Kevin Rue-Albrecht (11:46:16): > so basically, all the computation happens on the cluster in this case too
Kevin Rue-Albrecht (11:48:27): > hang on, i’ll post that in the issue, i’d say it’s still relevant
Federico Marini (11:52:40) (in thread): > I’ll prepare a draft-y email
Federico Marini (11:52:52) (in thread): > but probably I’ll be fine with a good tour on the TCGA set
Aaron Lun (12:11:12): > Also, it’s branch pruning time.
Federico Marini (12:13:26): > :rocket:
Federico Marini (12:16:04): > uh
Federico Marini (12:16:22): > did thebrush_integrationdisappeared before merging the PR?
Aaron Lun (12:16:30): > No, should be okay.
Federico Marini (12:17:22): > This one errored for my merge
Federico Marini (12:17:35): > https://travis-ci.org/csoneson/iSEE/builds/331910373
Federico Marini (12:17:45): > but I see the edits in the master. Little puzzled, but it seems to have kept the edits
Aaron Lun (12:18:24): > Not sure what happened there.
Aaron Lun (12:18:55): > Don’t worry about it, I think it tried to pull at the same time I pruned it.
Federico Marini (12:19:13): > could be
Federico Marini (12:19:22): > anyway
Federico Marini (12:19:36): > happy we got to this point so quickly:slightly_smiling_face:
Kevin Rue-Albrecht (12:40:19) (in thread): > ‘that escalated quickly’:wink:
Kevin Rue-Albrecht (12:48:23): > @Aaron LunI found the issue with EH1
Aaron Lun (12:48:33): > go on…
Kevin Rue-Albrecht (12:48:33): > > > plot.data$jitteredX <- vipor::offsetX(plot.data$Y, > + x=plot.data$X, width=0.4, varwidth=FALSE, adjust=0.5, > + method='quasirandom', nbins=NULL) + as.integer(plot.data$X); > Error in density.default(y, n = nbins, adjust = adjust) : > 'x' contains missing values >
Aaron Lun (12:48:47): > goddammit
Kevin Rue-Albrecht (12:48:58): > can i just feed the non-NA values to offsetx ?
Aaron Lun (12:49:01): > this must have been throwing with ggbeeswarm as well.
Kevin Rue-Albrecht (12:49:26): > or does the vector need to be the length of data?
Aaron Lun (12:49:37): > Yes, it does.
Kevin Rue-Albrecht (12:49:37): > argh nevermind
Kevin Rue-Albrecht (12:49:50): > i need to stop typing as i think
Aaron Lun (12:50:04): > We’ll have to intercept before it. Add aplot.data <- subset(plot.data, !is.na (X))I think
Aaron Lun (12:50:35): > Same for Y.
Kevin Rue-Albrecht (12:50:40): > ok
Kevin Rue-Albrecht (12:50:41): > thanks
Aaron Lun (12:52:13): > I wonder whethersubset(plot.data, !is.na (X) & !is.na (Y))would be enough.
Kevin Rue-Albrecht (12:52:26): > should be
Kevin Rue-Albrecht (12:53:12): > hang on:&or|?
Kevin Rue-Albrecht (12:53:32): > because your first bit seemed would result in|
Aaron Lun (12:53:52): > Well it would need to be non-NA for both X and Y, otherwise it would all fall apart.
Kevin Rue-Albrecht (12:54:20): > fair enough, just checking, as I’m getting near brain-fried for today
Aaron Lun (12:54:24): > Actually, maybe it only needs to be non-NA for x.
Aaron Lun (12:55:04): > No, probably best to do it for both. Wouldn’t make sense anyway.
Kevin Rue-Albrecht (12:55:13): > i’m racking my brain whether we’re throwing away useful information or not
Aaron Lun (12:56:22): > In NA’s? Well, we could define these as a separate group.
Aaron Lun (12:56:45): > But if that were the case, people should have explicitly defined them as “unknown” or something.
Aaron Lun (12:56:50): > If it’s missing, it’s missing.
Aaron Lun (13:05:15): > Discarding them is technically correct. The best kind of correct.
Aaron Lun (13:06:08): > Putting a new “NA” field is no good; what about neuraminidase?
Kevin Rue-Albrecht (13:07:27): > Yep - sorry i’m slow, i’m getting there. Juggling with a few last figure panels for a manuscript.
Kevin Rue-Albrecht (14:13:09) (in thread): > Ok. Let’s keep each other posted
Kevin Rue-Albrecht (14:13:59) (in thread): > I’ll meet them tomorrow or Friday
Kevin Rue-Albrecht (14:14:56) (in thread): > But I’ll use iSEE in my lab meeting tomorrow and I’ll ask my PI already
Federico Marini (15:05:02): > I think I might have found a problem in the brushing. To reproduce it, here are screenshots and code
Federico Marini (15:05:35): > @Federico Mariniuploaded a file:Screen Shot 2018-01-22 at 21.03.00.png - File (PNG): Screen Shot 2018-01-22 at 21.03.00.png
Federico Marini (15:05:49): > @Federico Mariniuploaded a file:Screen Shot 2018-01-22 at 21.03.21.png - File (PNG): Screen Shot 2018-01-22 at 21.03.21.png
Federico Marini (15:05:55): > when the brushed points are kept transparent, it seems to me they are not subset correctly
Federico Marini (15:06:00): > by color they are
Federico Marini (15:06:03): > @Federico Mariniuploaded a file:Screen Shot 2018-01-22 at 21.03.33.png - File (PNG): Screen Shot 2018-01-22 at 21.03.33.png
Federico Marini (15:06:11): > using the EH1 example
Federico Marini (15:07:41): > > se <- sce1 > all.coordinates <- list() > > ################################################################################ > ## Reduced dimension plot 1 > ################################################################################ > > red.dim <- reducedDim(se, 'TSNE'); > plot.data <- data.frame(X = red.dim[, 1], Y = red.dim[, 2], row.names=colnames(se)); > plot.data$ColorBy <- colData(se)[,'CancerType']; > > # Defining the plot boundaries > xbounds <- range(plot.data$X, na.rm = TRUE); > ybounds <- range(plot.data$Y, na.rm = TRUE); > > # Generating the plot > ggplot() + > geom_point(aes(x = X, y = Y, color = ColorBy), plot.data) + > labs(x = 'Dimension 1', y = 'Dimension 2', color = 'CancerType') + > coord_cartesian(xlim = xbounds, ylim = ybounds, expand = TRUE) + > theme_bw() + > theme(legend.position = 'bottom') > > # Saving for brush transmission > all.coordinates[['redDimPlot1']] <- plot.data > > ################################################################################ > ## Column data plot 1 > ################################################################################ > > plot.data <- data.frame(Y = colData(se)[,'bcr_patient_barcode'], row.names=colnames(se)); > plot.data$X <- colData(se)[,'CancerType']; > brushedPts <- shiny::brushedPoints(all.coordinates[['redDimPlot1']], > list(xmin=30.007, xmax=51.089, ymin=-16.559, ymax=14.046, > direction='xy', mapping=list(x='X', y='Y'))); > plot.data$BrushBy <- rownames(plot.data) %in% rownames(brushedPts); > plot.data$X <- as.factor(plot.data$X); > plot.data$Y <- as.numeric(plot.data$Y) > plot.data$GroupBy <- plot.data$X; > > # Defining the plot boundaries > ybounds <- range(plot.data$Y, na.rm = TRUE); > > # Setting up the data points > plot.data <- subset(plot.data, (X) & (Y)); > set.seed(100); > plot.data$jitteredX <- vipor::offsetX(plot.data$Y, > x=plot.data$X, width=0.4, varwidth=FALSE, adjust=0.5, > method='quasirandom', nbins=NULL) + as.integer(plot.data$X); > > # Generating the plot > ggplot(plot.data, aes(x = X, y = Y, group = GroupBy)) + > geom_violin(alpha = 0.2, scale = 'width') + > geom_point(aes(y = Y, x = jitteredX), subset(plot.data, !BrushBy)) + > geom_point(aes(y = Y, x = jitteredX), data = subset(plot.data, BrushBy), color = '#FF0000') + > labs(x = 'CancerType', y = 'bcr_patient_barcode') + > coord_cartesian(xlim = NULL, ylim = ybounds, expand = TRUE) + > scale_x_discrete(drop = FALSE) + > theme_bw() + > theme(legend.position = 'bottom') > > ################################################################################ > ## Gene expression plot 1 > ################################################################################ > > plot.data <- data.frame(Y=assay(se, 'exprs')['1/2-SBSRNA4',], row.names = colnames(se)) > plot.data$X <- colData(se)[,'CancerType']; > brushedPts <- shiny::brushedPoints(all.coordinates[['redDimPlot1']], > list(xmin=30.007, xmax=51.089, ymin=-16.559, ymax=14.046, > direction='xy', mapping=list(x='X', y='Y'))); > plot.data$BrushBy <- rownames(plot.data) %in% rownames(brushedPts); > plot.data$X <- as.factor(plot.data$X); > plot.data$GroupBy <- plot.data$X; > > # Defining the plot boundaries > ybounds <- range(plot.data$Y, na.rm = TRUE); > > # Setting up the data points > plot.data <- subset(plot.data, (X) & (Y)); > set.seed(100); > plot.data$jitteredX <- vipor::offsetX(plot.data$Y, > x=plot.data$X, width=0.4, varwidth=FALSE, adjust=0.5, > method='quasirandom', nbins=NULL) + as.integer(plot.data$X); > > # Generating the plot > ggplot(plot.data, aes(x = X, y = Y, group = GroupBy)) + > geom_violin(alpha = 0.2, scale = 'width') + > geom_point(aes(y = Y, x = jitteredX), subset(plot.data, !BrushBy)) + > geom_point(aes(y = Y, x = jitteredX), data = subset(plot.data, BrushBy), color = '#FF0000') + > labs(x = 'CancerType', y = '1/2-SBSRNA4 (exprs)') + > coord_cartesian(xlim = NULL, ylim = ybounds, expand = TRUE) + > scale_x_discrete(drop = FALSE) + > theme_bw() + > theme(legend.position = 'bottom') >
Federico Marini (15:09:22): > plus, as an enhancement: for the plots where the brushing is originated, should we do something like adding a shaded rectangle for the selection?
Kevin Rue-Albrecht (15:09:52): > <!channel>i’ve would like to offer an idea: > As I’ve mentioned a couple of times, I’m co-author on a manuscript that is about to be submitted (~next 10 days) and includes some 10x data that we’ve processed “Seurat-style” (with a hint ofscater::downsampleCount). > Tomorrow, I’m giving a lab meeting where I would like to suggest to my PI adding as a supplemental a ZIP of the current version ofiSEEpreloaded with a SCE object of the data set (e.g. .on Attach()). Obviously i would also point to the GitHub repo, but I’m sure journal would like a snapshot that doesn’t change over time as supplemental. > It’s the same data set that I’d personally be happy to share with Federico for his presentation next week, if the collaborators agree. > I think that would be a good precedent to start giving visibility to iSEE, as well as enhance the manuscript. > I would like to hear your opinions before talking to Steve (my PI). > PS: I would probably downgrade the R requirement to R≥3.4 at least for the first submission
Kevin Rue-Albrecht (15:11:23): > I’ll leave you to think while I’ll go get some dinner:slightly_smiling_face:
Federico Marini (15:12:41): > to avoid collisions of dependencies and so: what about a docker image, built on R-devel?
Federico Marini (15:12:54): > before aaron calls the SACRILEGE card:smile:
Kevin Rue-Albrecht (15:44:14): > i’ve never taken the time to learn how to make a container, but that sounds like the way to go
Kevin Rue-Albrecht (15:48:13) (in thread): > i like the idea, i’m always bothered when the brushed area disappears when the plot is refreshed, while you haven’t actually deselected anything
Kevin Rue-Albrecht (15:48:51) (in thread): > it would be nice to have a more persistent shaded rectangle
Kevin Rue-Albrecht (15:56:48) (in thread): > Indeed, i’ve seen that as well. I’ll have to look into it, it must be aroundsubset(..., BrushBy)andsubset(..., !BrushBy)
Kevin Rue-Albrecht (15:58:25): > alternatively, we could maybe submit the RDS file only (of the SCE) and point to the GitHub repo.
Kevin Rue-Albrecht (15:59:23): > “this object can be visualised interactively using iSEE (link)”
Kevin Rue-Albrecht (16:13:51) (in thread): > You know what I think they are subsetted correctly. > I think that despite the transparency, some groups have some many data points overlaid that it gives the impression of being selected
Kevin Rue-Albrecht (16:14:32) (in thread): > i’ll reduce the point size locally, to see if i can desaturate the areas and prove my point
Kevin Rue-Albrecht (16:16:01) (in thread): > Actually@Federico Marini, i don’t even need to touch the code: > just stretch the colData and geneExprs plots to 12 units of width, and you’ll see the effect disappear everywhere except the truly brushed data points
Kevin Rue-Albrecht (16:16:30) (in thread): > wanna add it to the FAQ?
Kevin Rue-Albrecht (16:18:43): > @Kevin Rue-Albrechtuploaded a file:stretching plots can desaturate misleading brush/transparency effect - File (PNG): stretching plots can desaturate misleading brush/transparency effect
Aaron Lun (17:24:56) (in thread): > This is what I was referring to when I was talking about preserving brushing memory. This is not trivial. Don’t expect it anytime soon.
Kevin Rue-Albrecht (17:25:54) (in thread): > ok. didn’t realise that’s what you meant. noted
Aaron Lun (17:26:41) (in thread): > I don’t really understand what the problem is.
Aaron Lun (17:27:20): > What am I meant to be seeing here?
Kevin Rue-Albrecht (17:27:28) (in thread): > have you tried running the EH1 data?
Kevin Rue-Albrecht (17:28:00): > you see different shades of grey, depending on the density of data points?
Aaron Lun (17:28:06) (in thread): > I’ve never run it, and I can’t do it on this computer at home.
Aaron Lun (17:28:22) (in thread): > Note that if you have enough points, transparency won’t be particularly useful. That’s a fact of life.
Kevin Rue-Albrecht (17:28:36): > (with the left arrow, you can navigate the earlier screenshots sent by Fede)
Kevin Rue-Albrecht (17:29:10): > considering his selection, only the semi-last violin should have data points selected
Kevin Rue-Albrecht (17:29:25): > 3-4 screenshots ago, you’ll see that other violins look like they have data points selected, too
Kevin Rue-Albrecht (17:30:03): > so basically, my last screenshot above illustrates that what Fede reported as an issue, is actually only an artefact of overplotting
Kevin Rue-Albrecht (17:30:31) (in thread): > no worries. just checking. see my answer in the main channel
Aaron Lun (17:30:41): > Hm. Perhaps we should set “restrict” as the default then, and add a note to the FAQ, as suggsted.
Kevin Rue-Albrecht (17:30:59): > to be fair, i do like transparency as a default
Kevin Rue-Albrecht (17:31:27): > it’s really putting the emphasis on the ‘selection’ aspect of brushing
Aaron Lun (17:31:31): > Hm…. well, maybe just a note in the FAQ.
Kevin Rue-Albrecht (17:31:37): > i was just doing that
Aaron Lun (17:32:00): > The defaults are pretty easy to change programmatically, so I’m not so worried about that.
Kevin Rue-Albrecht (17:32:24): > and this time i don’t even have to write the question myself ^^ copy-pasting from Fede’s message
Kevin Rue-Albrecht (17:35:02): > .. actually, adapting a bit ^^
Aaron Lun (17:35:02): > Oh, before I take a shower; the other thing to solve are the buttons on the LHS, which don’t seem to click properly.@Federico Mariniany thoughts?
Kevin Rue-Albrecht (17:35:41): > i discussed that with him, let me find what he said (basically, that’s its not an obvious thing)
Aaron Lun (17:35:58): > Yes, I remember it too. We may have to divert back to “UP” and “DOWN”.
Kevin Rue-Albrecht (17:36:06): > I suppose you mean the up/down arrows. I noticed that only their left side ‘clicks’
Kevin Rue-Albrecht (17:37:03): > Yeah. Not so bad either. I was actually wondering whether adding some whitespaces in the button would help. I’ll have to remember to try
Kevin Rue-Albrecht (17:37:53): > at the moment, i’m desperately throwing in a few more slides in my lab meeting for tomorrow
Kevin Rue-Albrecht (17:38:22): > anyway, I haven’t told Steve about the app yet, so i’ve got my wildcard to buy time:slightly_smiling_face:
Aaron Lun (17:38:24): > I usually just wing it.
Kevin Rue-Albrecht (17:38:52): > yeah.. he’s a bit more formal than us, unfortunately ^^
Aaron Lun (17:57:36): > I think we should have another skype before the final push to alpha. What time is everyone free?
Kevin Rue-Albrecht (17:58:18): > i agree. let me see
Davis McCarthy (17:58:19): > I’ve been getting going with Docker/Singularity containers recently so I could help with that if needed. A good way to give a snapshot ofiSEEat an early stage to go with a manuscript. The container won’t change, so that’s good for the manuscript - the container doesn’t necessarily have to make the source code available if that’s a concern but you could reference it as coming from version x.y.z. or commit XXXXXX in the GitHub repository so that people know where it came from. Then you could cite the repo for people to check out if they want to try thelatest
Davis McCarthy (18:00:27): > also we could add a dockerfile to theiSEErepo and have a docker container built on Dockerhub orQuay.io when triggered with a push to the repo - would make it easy to distribute to systems where installing software is a pain
Kevin Rue-Albrecht (18:01:10): > nice
Kevin Rue-Albrecht (18:03:40): > First of all, i’ll see tomorrow what they think about pre-releasing their (preprocessed to counts) data in this way. They might not like the idea, although it would help them just as much to keep a snapshot like this. Who knows
Kevin Rue-Albrecht (18:07:08): > if tomorrow, >3pm UK time for me; otherwise Wednesday & Thursday afternoons look fairly clear to me
Aaron Lun (18:45:43): > I can skype tomorrow as well.
Aaron Lun (18:46:19): > except at 6 - 6:30, which is another skype.
Aaron Lun (18:46:37): > see what@Federico Mariniand@Charlotte Sonesonhave available.
Davis McCarthy (18:48:39): > I could skype after 3pm Uk tomorrow and Wednesday is pretty free for me
2018-01-23
Charlotte Soneson (01:26:11): > By tomorrow, I suppose you mean today:slightly_smiling_face:I am available between 10 and 4:45 Swiss time, so I guess the option would be at 4 in that case to fit with Kevin’s availabilities. Wednesday 10-12, 1:30-4 Swiss time, Thursday until 2 Swiss time is also fine.
Federico Marini (01:38:28): > today is good for me in the afternoon
Federico Marini (01:38:41): > from 3pm german-swiss time
Federico Marini (01:39:30) (in thread): > I’d favor color then as a default brushing
Federico Marini (01:39:38) (in thread): > red is nicely popping out to the eyes
Federico Marini (01:39:45) (in thread): > to see right away what happens
Federico Marini (01:40:02) (in thread): > if more cosmetic changes are coming in the mind of the user
Federico Marini (01:40:18) (in thread): > I’d say she/he can do that afterwards
Federico Marini (03:01:04) (in thread): > I had some experience with Docker “back then”, maybe in the meanwhile things evolved somewhat differently/got easier or harder
Federico Marini (03:01:26) (in thread): > What I also think could be useful would be to make the package available as a bioconda recipe
Federico Marini (03:01:45) (in thread): > I’m in the process of getting the recipes approved for my packages
Federico Marini (03:03:09): > tomorrow is also pretty good
Federico Marini (03:03:21): > tomorrow now really means tomorrow:slightly_smiling_face:
Federico Marini (03:04:04) (in thread): > buttons: the idea of buttons that would be entirely clickable is appealing too, and better than its current status
Federico Marini (03:04:12) (in thread): > let me check if there are other options
Kevin Rue-Albrecht (04:05:40): > Sounds like today 3-3:45pm (UK) works. Shall we pencil it down? > Seems also that next couple of days offer good backup plans if we have to cancel
Kevin Rue-Albrecht (04:07:35): > I woke up early and wrote the container for color maps before breakfast. Just have to integrate it in the app now, which means simple calls to ’scale_color_*’
Kevin Rue-Albrecht (04:08:39): > I like the smell of code in the morning
Charlotte Soneson (04:09:44): > Today 3-3:45 UK time sounds good:slightly_smiling_face:
Kevin Rue-Albrecht (04:11:27): > I’ve got a morning full of seminars… so code should be done by lunch:innocent:
Charlotte Soneson (04:12:56): > I will try to get the app going on our shiny server. Most likely I’ll have to update some things (and try not to interfere with the other apps we have there in the meanwhile)
Federico Marini (04:13:37): > @Charlotte SonesonI think the main issue would be to have r-devel
Federico Marini (04:13:46): > unless (again) we commit sacrilege:smile:
Charlotte Soneson (04:15:48): > Yes. But I guess the main point here is to check that it actually runs, right (which could be done with an older R version)? The functionality can be tested locally. I would have to agree with everyone to update R, since it might break other apps…
Federico Marini (04:20:10): > Go ahead with it, as a starter we only want tosee whether it runs
Aaron Lun (04:21:36): > Okay, 3pm it is.
Federico Marini (04:22:00): > Noted
Kevin Rue-Albrecht (04:29:27) (in thread): > i’m ok with that
Federico Marini (04:30:37) (in thread): > Sorry for flagging that as a bug. I’ll elaborate a little more on my preference
Federico Marini (04:31:30) (in thread): > Color is imho the best because as a user, you do the cognitive task of selecting a group of points, and I think it is the most obvious think to expect to be ready and detect changes in other plots
Aaron Lun (04:31:53) (in thread): > Color depends on compatibility with the existing color scale, which has its own problems.
Federico Marini (04:31:58) (in thread): > if the changes go in the direction of shading out all other points, it could be misleading (especially when using as we might many samples/cells)
Aaron Lun (04:32:00) (in thread): > I would say that restrict is in fact the best default
Federico Marini (04:32:54) (in thread): > Something between color and restrict is the “least misleading”, if you pass me this definition
Aaron Lun (04:35:09) (in thread): > I don’t know what this would entail. But we’re not adding new brushing options at this point, it was a real pain to get the current setup working.
Federico Marini (04:49:03) (in thread): > Sorry, bad wording
Federico Marini (04:49:07) (in thread): > no extra option
Federico Marini (04:49:11) (in thread): > One of them
Charlotte Soneson (04:59:22): > So, everything seems to be running fine on our shiny server (with R 3.4). I just put anapp.Rscript in the app folder on the server with the following content: > > library(iSEE) > sce=readRDS("allen_sce.rds") > iSEE(sce) >
Charlotte Soneson (04:59:37): > Whereallen_sce.rdsis a saved version of the example data set
Federico Marini (05:53:24): > Cool!
Aaron Lun (06:20:55): > I’ll grudgingly admit whoever put ininitialPanelsiniSEE.Rdid an alright job.
Kevin Rue-Albrecht (06:21:21): > hahaha
Aaron Lun (06:24:16): > You will have to explain this ExperimentColorMap thing to me later. It seems much heavier than I thought it would be.
Kevin Rue-Albrecht (06:24:45): > well the class is fairly light (have you tried the example in the man page?)
Aaron Lun (06:25:03): > No, I’m just looking at the code right now.
Kevin Rue-Albrecht (06:25:31): > the integration is a bit more intricate than i thought, as i need to pass the original name of the covariate selected for color through the plotting functions
Aaron Lun (06:25:49): > Hm.
Kevin Rue-Albrecht (06:25:50): > anyway, don’t worry too much about it for now: that’s the point of a side branch
Kevin Rue-Albrecht (06:26:29): > as long as you see a PR, it means i’m not sure of myself either
Kevin Rue-Albrecht (06:27:02): > but i do like the feel of the container, as i’ve tested it
Kevin Rue-Albrecht (06:28:43): > think of it as a lazy/relaxed container: > * you store colormaps for any covariate of interest, don’t worry about the others > when you query it, it return either: > * the colormap that you defined > NULLfor discrete covariates (leave the app apply default ggplot) > viridis::viridis(10)for continuous covariates (best default to discuss)
Aaron Lun (06:29:13): > I presume there should be a check when there are more levels than colors.
Aaron Lun (06:29:36): > How doesggplot handle the colours for continuous variables - viacut?
Kevin Rue-Albrecht (06:30:02) (in thread): > scale_color_gradientn
Kevin Rue-Albrecht (06:30:20) (in thread): > and its friendscale_fill_gradientn
Kevin Rue-Albrecht (06:32:25): > bear with me for a day
Aaron Lun (06:33:02): > Okay.
Aaron Lun (06:33:30): > Meanwhile, I will try to solve the cases where the vipor offsets don’t match up with the violin.
Kevin Rue-Albrecht (06:35:13): > cool
Aaron Lun (06:39:35): > pretty sure it’s something to do with the bandwidth adjustment…
Kevin Rue-Albrecht (06:43:53): > moment of truth over here: testing a custom color scale for redDim plot for the first time
Kevin Rue-Albrecht (06:45:21): > arf.. forgot%sin mysprintf
Kevin Rue-Albrecht (06:48:10): > tadaaaaa
Kevin Rue-Albrecht (06:48:37): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-01-23 11.48.18.png - File (PNG): Screenshot 2018-01-23 11.48.18.png
Aaron Lun (06:48:48): > Fascinating.
Kevin Rue-Albrecht (06:49:19): > anyway, i’ll stop spamming the channel and finish that up to something that you guys can take for a test run
Aaron Lun (06:50:39): > Great. Also, do you have an example where the violin plot disagrees with the scatter? I remember seeing something in one of your images; see if PR 78 helps.
Aaron Lun (06:51:46): > Oh, and I broke the current master, so wait for 78 to merge.
Kevin Rue-Albrecht (06:51:57): > can’t remember a specific issue of that kind
Aaron Lun (06:54:10): > Seehttps://community-bioc.slack.com/files/U6F3QQT6V/F8XE90WJJ/screenshot_2018-01-22_16.16.20.png
Aaron Lun (06:54:10): > @Aaron Lunmentioned a file:Screenshot 2018-01-22 16.16.20.png . - File (PNG): possible demo data set
Kevin Rue-Albrecht (06:54:12): > do you mean last night’s point raised by Fede? the thing about brushing? because we agreed that’s an artefact of overplotting
Kevin Rue-Albrecht (06:54:19): > it’s the only disagreement i can remember
Aaron Lun (06:54:25): > Look at how the scatter is narrower than the violin.
Kevin Rue-Albrecht (06:57:32): > ahhh that one, i thought .scatter_plot and .violin_plot
Kevin Rue-Albrecht (06:58:21): > how to control the scatter to fit the violin layer is beyond me
Aaron Lun (07:29:09): > Well, see if the latest master is any better.
Federico Marini (08:22:09): > For the Skype call:@Davis McCarthywhat would be your username?
Aaron Lun (08:31:42): > I’m also going to add an option to choose the color when color by =“none”
Aaron Lun (08:50:25): > Annotation stuff now belongs in a separate file,@Federico Marinican you add a more graceful error message when the key is not found?
Aaron Lun (08:54:44) (in thread): > Do you have support for general coloring (i.e., the default color of points, other than black)? It would save us from having to add an explicit option for each plot.
Davis McCarthy (08:56:50): > “davis.mccarthy”
Federico Marini (09:06:37) (in thread): > Which branch is it on?
Federico Marini (09:06:45) (in thread): > defaults?
Aaron Lun (09:08:09) (in thread): > Yep, soon to be merged.
Federico Marini (09:10:55) (in thread): > I’d wait briefly then. I’m drafting a couple of presentations for the next days
Federico Marini (09:11:18) (in thread): > (one of them is about our creature here)
Federico Marini (09:11:37) (in thread): > I was thinking of a title likeiSEEing is believing
Kevin Rue-Albrecht (09:56:49) (in thread): > not just yet
Kevin Rue-Albrecht (09:57:11) (in thread): > sorry - lab meeting extended to 2h in total once i started showing them aroundiSEE
Kevin Rue-Albrecht (09:57:34) (in thread): > so i’m only picking up now where i left off at lunch time
Kevin Rue-Albrecht (10:00:07) (in thread): > I’m nearly there for the support of custom colors. Support for ‘default’ (lack of) colors probably shouldn’t be supported by the ‘custom’ system, as this is adding scales (scale_color_*). > What you want to fiddle with are the calls to the individual layers, withcolor=...outside
Aaron Lun (10:00:37): > Skype now?
Federico Marini (10:01:08): > I’m all set
Kevin Rue-Albrecht (10:01:19): > ready to go
Aaron Lun (10:01:19): > Waiting for my mac to boot up
Davis McCarthy (10:40:17): > Are you skypeing now?
Charlotte Soneson (10:40:47): > yes. I think Federico sent a connection request to you
Davis McCarthy (10:44:55): > here if you can add me!
Charlotte Soneson (11:26:40): > Making all the up/down/discard buttons size 3 seems to make them all fully clickable…
Federico Marini (11:28:19): > I’m working exactly on that in my branch:slightly_smiling_face:
Federico Marini (11:28:33): > good thing I probably left one button sized 3
Charlotte Soneson (11:28:42): > Good, then I’ll leave it to you!
Federico Marini (11:28:47): > out of error? looking for serendipity?:smile:
Charlotte Soneson (11:29:14): > Who cares, it works:stuck_out_tongue_winking_eye:
Federico Marini (11:29:29): > Charlotte, can I put the link to the shiny server where you set up the running app?
Federico Marini (11:29:37): > in the upcoming presentation, I mean
Aaron Lun (11:30:21): > Stand by for merge fromdefaults, which contains a number of bug fixes.
Charlotte Soneson (11:32:16): > Ah, I tried it on our behind-the-firewall “try things out”-server…I’ll really have to check with everyone before I update R or point to a new library on the production server…let me see if I can make it work in time.
Federico Marini (11:34:08): > No prob, I can still demo it live in case someone asks
Federico Marini (11:34:23): > I mean, the presentation will almost be demo only
Federico Marini (11:54:23): > @Federico Mariniuploaded a file:logo_concept.png - File (PNG): logo_concept.png
Federico Marini (11:54:45): > This one is a draft of how I had it in mind
Federico Marini (11:55:31): > … and we could add it on the top left of the dashboard bar, as an icon for the app itself
Federico Marini (11:55:40): > at least only the goggles
Kevin Rue-Albrecht (12:04:09) (in thread): > @Federico Marinithink of deleting the FAQ point then when you get to the vignette
Kevin Rue-Albrecht (12:05:27): > I think it’s a good start of logo. I’ll think of suggestions later, i’m low on brainpower right now
Aaron Lun (12:38:59): > I’m happy with putting a logo in the vignette, but I’d rather not clutter the app.
Aaron Lun (12:39:46): > If you must, I would suggest putting a “ABOUT” button next to the button that open/closes the left sidebar. This could also include the acknowledgements and citation information, freeing up screen space.
Kevin Rue-Albrecht (13:38:03): > Alright. Sorry for the delay, busy afternoon even after the Skype. But I’ll do all I can to merge in thecolormapsbranch tonight
Federico Marini (14:42:53) (in thread): > The logo would go rather like in this case:
Federico Marini (14:42:54) (in thread): > https://bioconductor.org/packages/release/bioc/vignettes/TCGAbiolinksGUI/inst/doc/vignette.html
Federico Marini (14:43:15) (in thread): > up left, together with the title itself. so no clutter
Federico Marini (15:06:59): > Couple of things on the aesthetics:
Federico Marini (15:07:19): > - TCGAbiolinks also has BTW some styling o f the sidebar menu
Federico Marini (15:08:36): > they do it like here:https://github.com/BioinformaticsFMRP/TCGAbiolinksGUI/blob/6d91de8effa560ae1992662b67e3df657b886df5/inst/app/ui.R#L84-L102 - Attachment (GitHub): BioinformaticsFMRP/TCGAbiolinksGUI > TCGAbiolinksGUI code development for R/Bioconductor submission http://tcgabiolinks.fmrp.usp.br:3838/
Federico Marini (15:10:04): > - the Bioconductor color I used for the button was[color content]
Federico Marini (15:10:23): > taken directly from a point in the background of the bioc website
Federico Marini (15:11:00): > BiocStyle still has different choices - at least for the vignettes:https://github.com/Bioconductor/BiocStyle/blob/18c0e098854fc87a10e760cf16a5dba5c29c2b18/inst/resources/html/bioconductor.css#L114-L128 - Attachment (GitHub): Bioconductor/BiocStyle > Issues and pull requests for BiocStyle should go here.
Kevin Rue-Albrecht (16:01:25): > We kinda brought up in the Skype that the dimRed plot relies on the availability of thereduceDimslot… which meansSingleCellExperimentobject. Did we land on a decision whether we ‘turn off’ that plot if we get aSummarizedExperiment, or should we simply claim that we supportSingleCellExperimentin the end?
Kevin Rue-Albrecht (16:38:29): > I’ll have to point out that last week I did insertstopifnot(inherits(se, "SingleCellExperiment"))at the very start of theiSEEfunction, to avoid bad surprises with the dimRed plot. We may want to remove it when we’re sure that we handleSummarizedExperimentproperly. > I have just tested what happens if we launch the app with an SCE object with an empty reduceDim slot, and it does not look good:Warning: Error in seq_len: argument must be coercible to non-negative integer
2018-01-24
Aaron Lun (04:53:12): > The fixes should have fixed that.
Aaron Lun (04:55:00) (in thread): > The space we currently have is probably too small to see anything but the simplest logo.
Kevin Rue-Albrecht (04:58:17): > Ok thank. i’ll check soon
Kevin Rue-Albrecht (05:00:08): > (#)random: if we want to mess with the users, we can coin as ‘3D plots’ the density-dependent downsampled plots
Aaron Lun (05:01:34): > ?
Aaron Lun (05:02:04): > Well, I thought a more amusing way to confuse people would be to call our factr X/Y plots “boxplots”.
Aaron Lun (05:02:20): > User: “how do you visualize two factors against each other?”
Aaron Lun (05:02:28): > Me: “Oh, we use boxplots.”
Aaron Lun (05:02:32): > User: “???”
Kevin Rue-Albrecht (05:55:14): > late comeback: it was a nice out-of-the-box idea
Aaron Lun (08:36:13): > Using shinyjs to disable buttons; this is really sweet.
Aaron Lun (08:36:18): > Check outhardened.
Aaron Lun (08:36:33): > Will continue after I get back from some PhD talks that I need to attend.
Federico Marini (09:44:08): > colormap team, do we need to specify the import from Rcolorbrewer too?
Kevin Rue-Albrecht (09:49:05): > ah. maybe not theimportFromstatement inNAMESPACE, given that I used::in the example codeRColorBrewer::brewer.pal(3, "Set2")
Kevin Rue-Albrecht (09:49:48): > Give it a try without (R CMD check) and see if it fails. I can’t think which way is correct right now
Federico Marini (09:52:58): > failing does not occur
Federico Marini (09:53:45): > but I am just thinking that the package should be present. it is for most of us, but I don’t know if this is the case also on the CI machines. Likely yes, as it is a common dependency
Federico Marini (09:53:59): > So I also do not know wjhat the desired approach should be
Federico Marini (09:54:13): > better call@Aaron Lun`?
Kevin Rue-Albrecht (09:55:43): > Cool, thanks for checking. I’m meeting candidates for PhD studentships ATM
Federico Marini (09:56:24): > noproblemo
Federico Marini (09:56:30): > I’m deep in vignetting
Aaron Lun (10:14:51): > Probably should import Rcolorbrewer, just like I import vipor::offsetX even though I don’t really need to.
Aaron Lun (10:18:51): > Hey, someone got rid of myresetOnNew=TRUEcommand. That was why the brushes weren’t disappearing yesterday.
Federico Marini (10:32:27): > If it was me, was not on purpose & if so, sorry in advance
Aaron Lun (10:44:21): > No worries, I’m not going to point fingers.
Aaron Lun (10:44:28): > That’s whatgit blameis for:slightly_smiling_face:
Aaron Lun (10:45:25): > @Kevin Rue-AlbrechtLooks like assay color map is incorrectly called when coloring by coldata. Refactoring now to centralize color choices…
Federico Marini (10:53:18): > eheheh
Kevin Rue-Albrecht (11:22:35): > oopsiedaysies.. 1 copy-paste too many.. i thought i fixed all of those.
Aaron Lun (11:33:32): > Major clean-up to colormap incorporation inplotting.Rinhardened. See if it makes sense to you.
Kevin Rue-Albrecht (11:34:45): > ok thanks.
Aaron Lun (11:36:05): > Note thecolor_FUN, which is a function that generates a command containing theggplot scale_color_*function. All relevant information is effectively embedded in the function closure.
Kevin Rue-Albrecht (11:39:01): > That sounds absolutely perfect. I’ll check it out in detail later this evening.
Aaron Lun (11:47:43) (in thread): > The interface will now be sane when there are no reduced dimension assays, so it is fine to coerce SE objects into SCE objects with empty assays.
Aaron Lun (13:14:25): > Too grumpy right now to review a paper, so will begin the refactoring from hell - to separateplot.datageneration from the actual plotting commands.
Kevin Rue-Albrecht (13:20:23): > woot woot
Aaron Lun (13:26:00): > This is going to hurt. No one touch anything inplotting.Rfor a while. I WILL finish this tonight.
Aaron Lun (13:46:36): > Harder than I thought. It’s not just a case of separating them out, creating all theplot.data, and then creating the plots; this is because theplot.datawill depend on the brushing when restricting, so it may not be possible to define all of theplot.datain the given order of plots. So I would still have to find an overall parent panel.
Aaron Lun (14:46:31): > It is done.
Federico Marini (14:50:15): > I still think John got you for your looks
Federico Marini (14:50:57): > but there seems to be a skill portion which is not to be straightforward hidden:slightly_smiling_face:
Federico Marini (14:51:02): > great job:slightly_smiling_face:
Aaron Lun (14:51:53): > The graph structure incodetrack.Rwill be similar to what we will be using for the brush memory, to push changes in brushing to upstream plots in order to update downstream plots.
Federico Marini (14:59:07): > are you about to merge it already into master? if so, I could already merge it into our vignette draft version
Federico Marini (15:50:57): > One thought that crossed my mind right now talking to Charlotte
Federico Marini (15:51:20): > I think it is a pity we did not apply for any funding for this
Kevin Rue-Albrecht (15:52:11): > where could we have applied for funding, for a sort of mini-project like that?
Federico Marini (15:52:34): > no idea. and especially with our tight time frame that we chose to stick to
Federico Marini (15:52:41): > I mean not for the money per se, but damn this thing came out damn good
Kevin Rue-Albrecht (15:53:07): > true - we would have finished the app before the application was even reviewed xD
Federico Marini (15:53:11): > as charlotte said, often life scientists do apply for things that are there at >90% completion
Kevin Rue-Albrecht (15:53:28): > i see your last point
Kevin Rue-Albrecht (15:53:50): > rolling on something that’s already working to fund the next one
Federico Marini (15:53:52): > I don’t know if we can fix it. Or somehow, get something more than mere glory out of it:stuck_out_tongue:
Kevin Rue-Albrecht (15:54:34): > although it would open all sorts of other complications, given that we’re all from different labs and places
Federico Marini (15:56:08): > also true. but I felt it was a small missing piece
Federico Marini (15:56:18): > again, not that it would have been the primary motor
Federico Marini (15:56:37): > would have been a kind of official seal on the co-op
Kevin Rue-Albrecht (15:56:51): > i totally agree with this
Kevin Rue-Albrecht (15:58:12): > i personally joined the fun for the mere glory out of it, as you say, knowing we’re contributing to the community a solid tool that consolidates our combined expertise
Federico Marini (16:02:01): > exactly, that brought a nice amount of fire within
Federico Marini (16:02:07): > good starter for 2018
Kevin Rue-Albrecht (16:03:57): > but in any case, we’ll have to formalise the co-op at some point when it gets to the manuscript. I don’t know exactly how our respective PIs will deal with that
Kevin Rue-Albrecht (16:04:33): > “dammit, we left our postdocs on the loose again”
Kevin Rue-Albrecht (16:44:20) (in thread): > @Aaron Lunyes there should be. > I’ll implement a method in theExperimentColorMapclass that take aSEand anECMobjects and that checks them for compatibility/discrepancy. > I’ll place that check among the first things that are done when launching the app, and throw an error/close the app if the test fails.
Kevin Rue-Albrecht (16:46:20) (in thread): > I must admit that I’ve invested a significant amount of time on iSEE recently (everyone has), and I need to catch up with local duties tonight
Aaron Lun (17:09:15): > I can’t imagine that our PIs would want in on the manuscript.
Aaron Lun (17:10:20): > I mean, with this kind of thing, every author had better be contributing code.
Aaron Lun (17:10:46): > The ideas guy can get behind the ideas guy who can code.
Kevin Rue-Albrecht (17:11:56): > eh.. i understood everything…. except the last bit:sweat_smile:
Aaron Lun (17:12:00) (in thread): > Just get colormap to a functional state, and I’ll handle the rest.
Aaron Lun (17:13:10): > Well, some people justify their presence on projects by saying that they contributed ideas. But ideas are cheap, it’s getting them to work that counts.
Kevin Rue-Albrecht (17:14:30): > agreed
Aaron Lun (17:14:53): > On an unrelated note, I talked to John last night about the iSEE manuscript and confirmed that he won’t be in the author list. Not that it really matters, I do this on my free time anyway, so I could well argue that he doesn’t pay me anything for this.
Kevin Rue-Albrecht (17:15:27): > i’m thinking the same here
Aaron Lun (17:15:30): > “Free time” being a somewhat liquid commodity in my world.
Kevin Rue-Albrecht (17:18:45): > it’s just that last time I submitted an application note to Oxford Bioinformatics (rejected, needless to say), it was costly enough to involve my PI
Aaron Lun (17:18:57): > Moneywise?
Kevin Rue-Albrecht (17:19:03): > yep
Aaron Lun (17:19:08): > Meh.
Aaron Lun (17:19:19): > UCam has open access things that pay for it.
Aaron Lun (17:19:33): > And I can beg some money off John.
Aaron Lun (17:19:40): > It’s not a problem.
Kevin Rue-Albrecht (17:19:54): > hahaha ok then
Kevin Rue-Albrecht (17:20:13): > anyway, i don’t want to get ahead of myself
Aaron Lun (17:20:38): > It’s good to discuss authorship ahead of time when everyone’s still got a cool head.
Aaron Lun (17:21:31): > I don’t care about the rest of the author order, as long as I’m last and corresponding.
Kevin Rue-Albrecht (17:21:36): > yep, we’ll have to gather everyone though, i see lights are on on the left, but that doesn’t mean everyone’s listening in
Aaron Lun (17:22:18): > It’s going to be an equal-first business for all of you guys anyway, so it’s just a matter of who gets the “et al.”
Aaron Lun (17:22:44): > And by “guys” I mean that in a gender-neutral way.
Kevin Rue-Albrecht (17:22:48): > what i have really enjoyed about this business of ours is that i could really feel everyone in for the glory of a good app
Aaron Lun (17:22:48): > Obviously.
Kevin Rue-Albrecht (17:23:01): > not for authorship, as is so common these days
Aaron Lun (17:23:11): > Oh…:sweat:
Aaron Lun (17:23:40): > Well, as I rack up more corresponding authorships, I’m going to get fewer ego-satisfying “Lun et al.”s
Kevin Rue-Albrecht (17:23:47): > hehehe
Aaron Lun (17:23:57): > sigh
Kevin Rue-Albrecht (17:23:59): > i’m so sorry that we’ll take that away from you
Aaron Lun (17:24:18): > I mean, what’s the point if I don’t see my name being cited?
Kevin Rue-Albrecht (17:24:18): > we’re contributing to your demise xD
Aaron Lun (17:24:52): > I mean, if the journals are using a reference sstyle that only shows three authors, I’ll be the only one in the “and others”.
Kevin Rue-Albrecht (17:25:03): > lol
Kevin Rue-Albrecht (17:25:49): > the other silly thing that crossed my mind was whether we could set the record for the paper with the largest number of “these authors contributed equally”
Aaron Lun (17:26:12): > I’m pretty sure those consortium papers go to 5 or more.
Kevin Rue-Albrecht (17:26:41): > but honestly, Fede already said it few times, but kudos on the infrastructure
Aaron Lun (17:26:42): > God, don’t get me started on the HCA Hackathon paper that I’ve got in prep.
Aaron Lun (17:27:00): > I mean, every partipicant is an author? Screw that.
Kevin Rue-Albrecht (17:27:26): > did i miss much?
Kevin Rue-Albrecht (17:27:46): > (might take this to our own conversation if we’re going off track)
Aaron Lun (17:29:02) (in thread): > I can only see suffering in that direction.
Aaron Lun (17:30:34) (in thread): > It would be an administrative nightmare involving 4 different institutions and pay schemes.
Aaron Lun (17:31:56) (in thread): > We wouldn’t be asking for more than 20% time from each of us, for maybe a few months; probably not over 20K GBP in total, including indirects. Very few funding streams for that kind of stuff, especially for a pure computational project.
Aaron Lun (17:33:46) (in thread): > And seriously, it would take less work to do it than to apply. I applied to the BBSRC once and ended up finishing the roposed work before I got the decision back (rejected, of course).
2018-01-25
Charlotte Soneson (01:25:21): > I am on board with everything you said about the authorship yesterday night. And in the end I feel like I probably contributed less critical insights than you guys, so I’m not going to fight for the first position:slightly_smiling_face:
Charlotte Soneson (01:25:43): > And for the largest number of “These authors contributed equally”, I already have a paper with 9…
Federico Marini (03:01:07): > Hopping in: I also think my Chefin as well as my mentors do not want to be in the author list
Federico Marini (03:01:21): > not even their ideas went in so:stuck_out_tongue:
Federico Marini (03:02:03): > for the fees, I am aware there are some fundings for paying the OA part also at my university
Federico Marini (03:05:55) (in thread): > For this I totally agree. The win is probably not worth the effort
Federico Marini (03:06:35) (in thread): > I’m glad it went pretty smooth. And I got to learn a lot. So for me, WIN
Federico Marini (03:06:37) (in thread): > :slightly_smiling_face:
Kevin Rue-Albrecht (03:20:25): > a voice from the heavens? - Attachment: Attachment > Here’s a sidebar theme with bioconductor colors for anyone interested :smile: #4390A7,#076570,#A2C353,#404042,#86A645,#FFFFFF,#F79F66,#F15340
Kevin Rue-Albrecht (03:26:35): > i’ll pass the word to Steve about the plan for author list; so far the the excitement was more about getting it to run on our cluster… and other immediate deadlines about manuscripts to be submitted:sweat_smile:
Kevin Rue-Albrecht (03:28:07): > also, looking for open access publishing support in Oxford, I foundhttp://openaccess.ox.ac.uk/wellcome-and-coaf/ but i need to check whether it applies to me (Steve pays me from a WT strategic award, but ‘The policy applies to researchers in Wellcome Trust Centres.’ and I am in the Kennedy Institute, right next door to the WTCHG) - Attachment (openaccess.ox.ac.uk): Wellcome Trust and Charity Open Access Fund (COAF) > Current Policy What do I do at Oxford? Oxford has received a Wellcome/COAF block grant for 1 October 2017 to 30 September 2018 to
Kevin Rue-Albrecht (03:28:22) (in thread): > @Federico Marini
Federico Marini (03:30:48) (in thread): > hello there:smile:
Aaron Lun (04:51:13): > I’m wondering whether we should just import the whole of shiny.
Aaron Lun (04:51:31): > I think someone mentioned this earlier, but now seems like the right time to re-examine this.
Kevin Rue-Albrecht (04:51:42): > what % of their functions do we have? ^^
Aaron Lun (04:51:50): > Not sure, but the improtFrom list is getting pretty big.
Kevin Rue-Albrecht (04:52:34): > From experience, the most notorious conflict I’ve encountered with that is thecolumnfunction that collides withAnnotationDbi,ensembldband their friends
Aaron Lun (04:52:51): > Hm.
Aaron Lun (04:52:57): > Well, we importcolumnanyway.
Kevin Rue-Albrecht (04:53:17): > Don’t quote me on this, but I believe our current long list ofimportFromis best practice
Kevin Rue-Albrecht (04:53:44): > i’m rushing to finish up stuff before a meeting in 5 min, so i’ll leave you guys to discuss it
Kevin Rue-Albrecht (04:53:53): > just wanted to throw my 2 cents in
Aaron Lun (04:54:10): > Well, that’s fine with me too.
Aaron Lun (05:56:36): > Note that, when creating the plotting commands to evaluate, use: > > sprintf("blah <- %s", deparse(some_arbitrary_string)) # good > sprintf("blah <- '%s', some_arbitrary_string) # not so good > > as the former will protect against existing quotes and escapes while the latter will not. This is only necessary in cases where the string is not known in advance - we can control the internally defined variables, so there’s no need to do this e.g. when building theaesobject.
Aaron Lun (05:57:11): > I’ve implemented this throughoutplotting.Rbut I may have missed some spots.
Aaron Lun (06:48:29): > Color-coding is done. A bit of persistent whitespace between rows, not entirely sure how to get rid of it.
Aaron Lun (06:49:03): > The coloring of the LHS boxes is also a bit funny as the “Width” label redners white for some reason.
Federico Marini (06:49:09): > probably too deep hard-coded in the box properties and organization?
Aaron Lun (06:49:20): > Perhaps.
Aaron Lun (06:49:31): > Play around with it and see what you can do.
Federico Marini (06:49:47): > I tend to like it with boxes
Federico Marini (06:50:03): > makes it this modern touch - which nowadays we see almost everywhere, but well
Kevin Rue-Albrecht (08:55:19) (in thread): > at the latest this weekend, hopefully tonight
Kevin Rue-Albrecht (08:56:02) (in thread): > everyone’s on edge here about a manuscript about to be submitted to Immunity:sleepy:
Kevin Rue-Albrecht (08:59:55) (in thread): > … i’ll probably have my tongue cut for even telling you that:wink:
Aaron Lun (09:00:29) (in thread): > lol
Aaron Lun (09:00:48) (in thread): > well, check out the latest UI for some stress release.
Kevin Rue-Albrecht (09:01:09) (in thread): > ok, i was installing the latest pull in the background
Kevin Rue-Albrecht (09:01:16) (in thread): > didn’t have igraph on the desktop
Kevin Rue-Albrecht (09:02:20) (in thread): > oh ffs: error during install: > > **** testing if installed package can be loaded > Error: package or namespace load failed for 'igraph' in dyn.load(file, DLLpath = DLLpath, ...): > unable to load shared object '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/igraph/libs/igraph.so': > dlopen(/Library/Frameworks/R.framework/Versions/3.5/Resources/library/igraph/libs/igraph.so, 6): Symbol not found: __ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEcm > Referenced from: /Library/Frameworks/R.framework/Versions/3.5/Resources/library/igraph/libs/igraph.so > Expected in: flat namespace >
Kevin Rue-Albrecht (09:02:33) (in thread): > i had a similar error trying to updatescaterrecently
Kevin Rue-Albrecht (09:03:07) (in thread): > Davis helped me: it was just about re-installing RHdf5Lib and beachmat
Kevin Rue-Albrecht (09:03:28) (in thread): > i’ll google this one again, except if you know already?
Aaron Lun (09:03:45) (in thread): > Sounds like you were running two installs at the same time.
Aaron Lun (09:03:55) (in thread): > Just re-install igraph, I guess.
Kevin Rue-Albrecht (09:04:10) (in thread): > weird, I just calledinstall.packages("igraph")
Kevin Rue-Albrecht (09:04:15) (in thread): > i’ll try again though
Aaron Lun (09:04:45) (in thread): > This’ll happen if you run install in one session and try to load the package in another befoer the install has finished.
Kevin Rue-Albrecht (09:05:14) (in thread): > running another install now, i’ll let you know
Kevin Rue-Albrecht (09:09:20) (in thread): > dammit. no it’s doing it again
Kevin Rue-Albrecht (09:09:40) (in thread): > i’ve update the bug report above with an extra line
Kevin Rue-Albrecht (09:10:00) (in thread): > extra 3 lines now
Kevin Rue-Albrecht (09:10:25) (in thread): > btw, I also tried in a fresh R session, after closing RStudio
Aaron Lun (09:13:12) (in thread): > I’m going to guess that you’re using conda or homebrew something like that.
Kevin Rue-Albrecht (09:13:33) (in thread): > mah, i’ll figure it out over the afternoon, might be calling the wrong gcc (/usr/local/bin/gcc-7) of something stupid like that
Aaron Lun (09:14:32) (in thread): > The error message indicates that it’s failing to link. This would be because the libraries it’s linking to were compiled with a different version of GCC compared to the one you’re using now.
Kevin Rue-Albrecht (09:14:47) (in thread): > lol nevermind: > I managed to install it with an R prompt in my console > …. just another RStudio quirk
Aaron Lun (09:59:47): > Okay, moving onto brushing memory. This shit is going to get real.
Kevin Rue-Albrecht (10:01:08): > just launched the app built from the latest master: it is beautiful
Kevin Rue-Albrecht (10:01:18): > can’t wait to get the colormap sorted
Kevin Rue-Albrecht (10:01:59): > i just found a reminder that I wrote last night: is there any step that we would like to add aprogressBarfor yet?
Kevin Rue-Albrecht (10:03:53): > shiny::withProgressfunction
Aaron Lun (10:03:54): > Not yet, not to my knowledge unless something is being ridiculously slow.
Aaron Lun (10:03:59): > Possibly the startup, but that’s about it.
Kevin Rue-Albrecht (10:04:45): > Ok then. I wouldn’t recommend one for the start up
Aaron Lun (10:24:22): > God, this brushing memory is painful. So many ways to go horribly wrong.
Kevin Rue-Albrecht (10:27:55): > “the higher the climb …” > I’ll just shut up ^^
Aaron Lun (10:54:32): > It seems easier to just leave zooming and brush transmission on by default; otherwise we would have to re-render whenever anyone presses those buttons.
Aaron Lun (10:54:48): > I hope that this won’t result in a big impact on performance - probably not, given that zoom is already on by default anwyay.
Kevin Rue-Albrecht (11:06:09): > @Federico Marinicorrection - Attachment: Attachment > @David Jenkins actual colors are at http://bioconductor.org/about/logo/
Kevin Rue-Albrecht (11:19:19): > <!channel>I informed Steve of our plans for the manuscript, and the fact that none of your group leaders have asked to be on the author list. Nevertheless, as I am a postdoc in his group, and have dedicated a significant portion of my recent daily work time to iSEE, he would like to request that his name appears as a co-author. > Now, he explicitly said that he is absolutely not going for the last author position. Aaron more than deserves it. > I get where he’s coming from: he has to justify the time I spend as one of his group members. I have tried to keep it to out-of-hours, but I have to admit that it escalated quickly when I attackedggplotaspects. > Thinking about this, I wonder whether you would all be ok placing him as semi-last author?
Aaron Lun (11:19:39): > What
Kevin Rue-Albrecht (11:20:05): > (don’t kill the messenger)
Kevin Rue-Albrecht (11:30:20): > I’m sorry for the situation. Call me naive, but I genuinely was not expecting that
Federico Marini (12:43:48) (in thread): > I do not have benchmarks, but I think there is not that muc h of extra overhead for transmitting the brush
Federico Marini (12:44:39) (in thread): > that said, I’d also leave it on automatically. It is the beauty of linking, and that could be ideally delivered b default. nothing happens in the other plots anyway, until the brush is sent over
Federico Marini (12:48:49) (in thread): > (if we take the check out, i’ll edit accordingly the tour:wink:)
Kevin Rue-Albrecht (14:33:50): > Ehm. > Actually, Steve just let me know that he does not need to be author anymore on this one, he apologises for the mis-communication.
Aaron Lun (15:01:07): > Oh. Good.
Federico Marini (15:02:20): > stepping in now. I also think it avoids “problems”, as this creature is kind of really our thing
Federico Marini (15:02:54): > one thing if say, the PIs had the same idea we had, and then put us to work
Federico Marini (15:12:22): > -> ideas for eventual DT::datatable pimping:
Federico Marini (15:12:23): > https://rstudio.github.io/DT/extensions.html
Federico Marini (15:12:35): > do you see something we could conveniently add?
Federico Marini (15:13:01): > I fancy the scroller and the button to export
Aaron Lun (15:17:06): > Hmm.
Kevin Rue-Albrecht (15:17:17): > i already don’t get the autofill. Nevermind, I didn’t realise immediately that it was a demonstration page: I found how to play with it:sweat_smile:… this thing with Steve today got me all upside down
Aaron Lun (15:17:37): > The scroller is nice but it will automatically reset to the top after re-rendering.
Aaron Lun (15:17:46): > That’s maybe not so bad, but something to keep in mind.
Aaron Lun (15:18:01): > The export button… well, might be useful for data hosted on a server.
Aaron Lun (15:18:27): > Okay, I’m going offline for dinner.
Federico Marini (15:21:23): > export I started adding it to all my reports with datatables in it
Kevin Rue-Albrecht (15:22:06) (in thread): > Exactly. Aside from the ‘who would go last’ thing that would arise, involving PI would slow down thing soooo much, I’m sure
Kevin Rue-Albrecht (15:23:48) (in thread): > They might as well be users (know enough about genomics to use it, don’t have enough time to contribute code), and feed back through issues like everyone else.
Kevin Rue-Albrecht (15:35:45): > btw, even though it would only cause a 1-line conflict: i’m taking the initiative to subsitutestopifnot(inherits(se, "SingleCellExperiment"))byse <- as(se, "SingleCellExperiment")on a new branch
Kevin Rue-Albrecht (15:36:06): > branch that i’ll use to fix the last colormap bugs
Aaron Lun (16:09:27): > I would suggest regularly merging withhardened, just to keep abreast of the brushing memory changes; as these will touch pretty much everything.
Aaron Lun (16:09:53): > Full support for brushing memory will not be possible untilhttps://github.com/rstudio/shiny/issues/1456 is available. - Attachment (GitHub): Set ggplot2’s plot brush programmatically Shiny · Issue #1456 · rstudio/shiny > So there is a blue semi-transparent brush/rectangle that you can drag and then move on ggplot2 with your mouse, you can get its xmin, xmax, ymin and ymax by observing the brush. How can I do the ot…
Aaron Lun (16:10:19): > However, we can do a poor man’s approximation that should be satisfactory until that comes along.
Kevin Rue-Albrecht (16:10:21): > thanks for the reminder, about merging, forgot to check how many open branches
Kevin Rue-Albrecht (17:16:43): > i’ll have to think more carefully about the hierarchy of colormaps this weekend. it’s a bit more work than it looks
Aaron Lun (17:22:00): > Wasn’t theassays/all_assays/globalidea good enough?
Aaron Lun (17:23:39): > GivenassayColorMap(ecm, i), if you can’t findiinassaysby character or integer, tryall_assays; and if no continuous/factor has been specified there, tryglobal; and if still you find nothing, use the defaults.
Aaron Lun (17:24:31): > It would be nice if you could allow expressions as list elements, to avoid having code chunks with an expandedviridis::viridis(100)
Kevin Rue-Albrecht (17:28:43): > maybe i’m just tired, but why do we need anall_assays? why not straight todefault?
Aaron Lun (17:29:18): > If you want different colors for assays and coldata, for example.
Aaron Lun (17:29:24): > Different default colors.
Kevin Rue-Albrecht (17:29:40): > also, you got me worried thinking that there may be non-numeric assays, so the default can’t just be a continuous scale anymore
Aaron Lun (17:29:48): > Well, that’s entirely possible.
Aaron Lun (17:29:56): > And it’s totally fine.
Aaron Lun (17:30:18): > Just make sure the validation method (on the ECM + the SE) handles it properly.
Aaron Lun (17:32:40): > And you don’t even need to check whether something is discrete or not; I’ll handle that.
Kevin Rue-Albrecht (17:32:43): > yep - i was thinking about implementing this one after fixing the current bugs, but maybe implementing the validation method first will save me some glitches downstream
Kevin Rue-Albrecht (17:32:50): > ahhhh you will?
Aaron Lun (17:32:53): > Yes.
Aaron Lun (17:33:00): > That’s part of the refactoring inplotting.R.
Aaron Lun (17:33:28): > You just need to give meassayColorMap(..., discrete=TRUE)or something.
Kevin Rue-Albrecht (17:34:11): > that’s what was splitting my head: i didn’t get how to handle discrete/continuous anymore when fetching the colormap, as the function is not aware of that information anymore
Kevin Rue-Albrecht (17:34:17): > ok, i’ll have a better look
Aaron Lun (17:34:51): > See thecolor_FUNargument passed out ofprocess_colorby_choice.
Aaron Lun (17:35:26): > Well, the discrete/continuous information wouldn’t have been available for the col data anyway.
Kevin Rue-Albrecht (17:38:08): > ok, i saw thecolor_FUN(and your earlier message about it), but it didn’t sink it yet, getting there
Kevin Rue-Albrecht (17:39:47): > ahhhhhhhhhhhhhh !
Kevin Rue-Albrecht (17:40:18): > i saw you were returning a function (obviously) but i just got the power of it now!
Kevin Rue-Albrecht (17:41:06): > those good looks will take you far, my friend
Aaron Lun (17:41:34): > your goddam right they will
Kevin Rue-Albrecht (18:24:28): > what’s you difference betweenall_assays/global?
Aaron Lun (18:27:09): > I was thinking if someone wanted to define their own continuous/discrete colors for all the assays, but only the assays; and use the built-in defaults for the colData. The same applies forall_coldata.
Kevin Rue-Albrecht (18:28:06): > That’s what I was getting at, theall_coldata. As I started picturing your thought process for assayData, I realised that you implied it for the others too
Kevin Rue-Albrecht (18:28:54): > isn’t that overkill? i get the point, but i wonder how many users will go to that depth
Aaron Lun (18:29:05): > Maybe, maybe not. I don’t really know.
Aaron Lun (18:29:21): > Power users would want it, though.
Aaron Lun (18:29:44): > And if you’re going to set up a hierarchy anyway, it’s just anotherelse ifstep.
Kevin Rue-Albrecht (18:30:08): > i know that once defined, the colormaps will be objects that can be saved and reloaded, it’s only a one-off cost
Aaron Lun (18:30:49): > I reckon it would be worth the effort to just do it now; avoid being asked for it later.
Aaron Lun (18:30:58): > I’m sure I
Aaron Lun (18:31:21): > ’m not theonly one who would think “gee, I want something in the middle between defining something for each assay/col data and everything.”
Kevin Rue-Albrecht (18:31:35): > indeed
Aaron Lun (18:32:16): > And you can just reuse the validity checks onglobalforall_assaysandall_coldata, because it’ll be the same sort of thing (a list withcontinuousanddiscrete.
Aaron Lun (18:32:17): > )
Kevin Rue-Albrecht (18:33:11): > ok, and thenglobalis a sort of user-defined default (one that they can leave toNULL), while the actual default is something that we hard-code as an ultimate failsafe?
Aaron Lun (18:33:16): > Yes.
Aaron Lun (18:36:33): > Note that every element ofassayswould need to be a list of two elements (withcontinuousand/ordiscretespecified), as the ECM doesn’t know the nature of these thingies.
Kevin Rue-Albrecht (18:37:03): > Ok, last barrier then is for discrete covariates: Even if we supply the defaultcolors , the colormap has no way of knowing how manylevels need to be given a color, nor what the name of those levels is
Aaron Lun (18:37:40): > Well, that relies on the user not giving us 4 colors when he/she has 5 levels in the data.
Aaron Lun (18:38:08): > I mean, how were we doing it before?
Kevin Rue-Albrecht (18:38:16): > i wasn’t thinking about this, i’m saying when they don’t supply any colormap
Aaron Lun (18:38:35): > Pass a function rather than an actualized vector of colors.
Kevin Rue-Albrecht (18:38:53): > in that case, I was not writing anyggplotinstruction, to let the ggplot defaults take over
Aaron Lun (18:38:58): > i.e., a function that accepts a number of levels and returns a number of colors.
Kevin Rue-Albrecht (18:39:15): > yeah you’re right - a sort of colorRamp
Aaron Lun (18:39:43): > It has to be deparsable, so keep that in mind.
Kevin Rue-Albrecht (18:39:52): > i’ll figure something out
Aaron Lun (18:40:27): > Arguably we should do something similar for the continuous colors, avoiding printing out all elements of a 100-color vector.
Kevin Rue-Albrecht (18:40:37): > all i’m saying is that continuous colormaps are naturally easier: you only need to give a vector colours and it will interpolate between the breaks
Kevin Rue-Albrecht (18:40:54): > categorical colormaps require the levels to which each colors are assigned
Aaron Lun (18:41:16): > Does it need the name of the levels? Or just the number of levels?
Kevin Rue-Albrecht (18:41:53): > well, if you supply a colormap, it needs to have the name of the levels (named vector: value = colour, name = level)
Aaron Lun (18:42:28): > So this is aggplot requirement? Sounds like a pain.
Aaron Lun (18:42:36): > Doable but a real pain.
Aaron Lun (18:42:46): > Well, a minor pain.
Aaron Lun (18:42:57): > Have to replacecolor_discretewith the actual vector of colors
Aaron Lun (18:43:15): > Then have to deparse it to create a vector definition that is fed into the colorRamp.
Kevin Rue-Albrecht (18:45:19): > what i was thinking was maybe let users define a loooong vector of colors, let’s call itglobal_discrete. When appropriate, the colormap would return that one, but it would then be the responsibility of the calling function to take the first N colors as needed, put the labels/levels on them, and use that as colormap
Kevin Rue-Albrecht (18:47:36): > actually, you’re right again: > > library(ggplot2) > mycolors <- c("blue", "red") > df <- data.frame(x = letters[1:2], y = 1:2) > ggplot(df) + geom_point(aes(x,y,colour=x)) + scale_color_manual(values = mycolors) > ggplot(df) + geom_point(aes(x,y,colour=x)) + scale_color_manual(values = rev(mycolors)) >
Aaron Lun (18:48:02): > Yeah, happens every now and then.
Kevin Rue-Albrecht (18:48:11): > you can give unlabelled colours, and it will just assign them to levels in arbitrary order (levels/alphabetical)
Kevin Rue-Albrecht (18:49:26): > thanks to iSEE, i’m learning again how to handle defaults. I mean I usually build my ggplot calls manually, so I don’t leave much to defaults
Kevin Rue-Albrecht (18:51:59): > OK.. just remains the scenario where none of the discrete colormaps have enough colors to cover all levels. But I already set a cutoff on the number of levels before coercing the covariate to numeric, 24. I’ll just figure out a good discrete colormap with that many colours.
Aaron Lun (18:54:16): > If you require functions rather than color vectors, that shouldn’t be a problem…
Aaron Lun (18:54:41): > heat.colors,topo.colors, etc.
Kevin Rue-Albrecht (18:54:55): > yeah - i was also bothered by the long vectors of colors in the modal script
Kevin Rue-Albrecht (18:55:48): > I took that ugly shortcut to implement a proof of concept, because i was really short on time this week
Aaron Lun (18:56:17): > Technically they’d need to passexpression(heat.colors).
Kevin Rue-Albrecht (18:56:26): > big day tomorrow too, so I’ll ask you all to bear with me til this weekend, before I push anything
Aaron Lun (18:58:04): > Though we could just define the colormap at the top of the reproduced code, and do something likeassayColorMap(color_map, "counts", discrete=TRUE)(10)without the need to try to preserve the original name of the function (which might not even exist).
Aaron Lun (18:58:53): > This would require an explicit ECM object for the default settings, but that’s probably easy.
Kevin Rue-Albrecht (19:00:05): > Glad we discussed it. I’d have gone into a thousand deadends tonight
Kevin Rue-Albrecht (19:00:38): > It’ll just require some proper planning. I think it’ll be the last big hurdle before the (first) finish line
Kevin Rue-Albrecht (19:01:59): > …or do you still have some brushing to figure out? can’t remember if your branch is ready
Aaron Lun (19:02:37): > I aim to get the brushing memory finished tomorrow in one big push.
Aaron Lun (19:02:53): > Laid the groundwork for it today, so everything is more-or-less ready to go.
Kevin Rue-Albrecht (19:04:59): > Alright then. I’ll go get some sleep, make people happy at work tomorrow, and earn my Saturday to finish off the colormap.
2018-01-26
Kevin Rue-Albrecht (02:30:18): > If anyone is interested, I’ll be using the following example to develop and test the colormap fixes: > > library(SummarizedExperiment) > ?SummarizedExperiment > > nrows <- 200; ncols <- 6 > counts <- matrix(runif(nrows * ncols, 1, 1e4), nrows) > rowRanges <- GRanges(rep(c("chr1", "chr2"), c(50, 150)), > IRanges(floor(runif(200, 1e5, 1e6)), width=100), > strand=sample(c("+", "-"), 200, TRUE), > feature_id=sprintf("ID%03d", 1:200)) > colData <- DataFrame(Treatment=rep(c("ChIP", "Input"), 3), > row.names=LETTERS[1:6]) > rse <- SummarizedExperiment( > assays=SimpleList( > counts=counts, > counts, > logcounts=counts, > counts, > counts > ), > rowRanges=rowRanges, colData=colData > ) > > > assayNames(rse) > [1] "counts" "" "logcounts" "" "" >
Kevin Rue-Albrecht (02:35:29): > It focuses on the mix of named/unnamed assays > Feel free to suggests other scenarios (discrete assays, etc.) if you can think of example use cases
Federico Marini (04:03:33): > -> graph of linked plots now available ondrafting_vignette
Kevin Rue-Albrecht (04:03:59): > can’t wait to see it
Aaron Lun (04:10:09): > That’s a pretty sadistic example, but fair enough.
Kevin Rue-Albrecht (04:10:34): > hehehe
Aaron Lun (04:10:35): > It would be a good idea to put your tests intests/testthatas you go along, to save yourself having to rewrite it all later.
Kevin Rue-Albrecht (04:11:38): > True, i’ve been thinking about that too. > To be fair, this example is entirely taken from?SummarizedExperiment+ some sadistic extra copy-pasting
Kevin Rue-Albrecht (04:11:49): > wouldn’t take long to regenerate ^^
Aaron Lun (04:12:10): > Okay, I’m going to finish the brushing memory - will not be responsive for a while.
Aaron Lun (04:36:41): > Okay, I’m back - how do I draw a rectangle with given coordinates in ggplot?
Charlotte Soneson (04:37:47): > I think it should work withgeom_rect(aes(xmin = , xmax = , ymin = , ymax = ))
Kevin Rue-Albrecht (04:38:36): > actually one correction: put all the min/max specificationsoutside the aes
Kevin Rue-Albrecht (04:39:06): > the aes is only useful if you supply a data.frame where each row should draw a rectangle
Charlotte Soneson (04:39:07): > ah yes, if you want to give the actual numbers
Kevin Rue-Albrecht (04:39:29): > it just depends what you want to do
Aaron Lun (04:43:34): > Okay; I presume this is possible withgeom_rect(aes(..), data)?
Kevin Rue-Albrecht (04:43:50): > exactly
Kevin Rue-Albrecht (04:44:06): > the values in youraescall then refer to columns indata
Aaron Lun (04:44:08): > okay, cool.
Aaron Lun (05:03:22): > Ugh. > > sprintf("geom_rect(aes(xmin = 'xmin', xmax = 'xmax', ymin = 'ymin', ymax = 'ymax'), data=data.frame(xmin = %.5g, xmax=%.5g, ymin = %.5g, ymax = %.5g)) +" > current$xmin, current$xmax, current$ymin, current$ymax) >
Kevin Rue-Albrecht (05:03:53): > ?
Aaron Lun (05:04:58): > Pretty damn long.
Aaron Lun (05:05:24): > Does it draw it with fill, or is it automatically empty?
Kevin Rue-Albrecht (05:06:45): > you need to add fill
Kevin Rue-Albrecht (05:07:25): > then again, iffillis a covariate, you’ll need to stash it indata, and refer toFillByin youraescall
Aaron Lun (05:07:31): > God
Aaron Lun (05:07:43): > Well, not having fill is fine for now, I’m just drawing brush limits.
Federico Marini (05:07:51): > I think fill can be some transparent shade
Federico Marini (05:07:54): > if at all
Kevin Rue-Albrecht (05:08:02): > it’s the price to pay for controlling aesthetics:wink:
Federico Marini (05:08:10): > if your aim is to just mark it on the source plot
Kevin Rue-Albrecht (05:09:18) (in thread): > Still better than the base plotting functions if you ask me. > At least with ggplot, you can break up the different aesthetics in multiple, smaller, calls, rather than one call with ten thousand arguments
Aaron Lun (05:12:42): > Hm. Above line is complaining aboutXnot found.
Kevin Rue-Albrecht (05:13:38): > arf
Kevin Rue-Albrecht (05:14:12): > it shouldn’t be a required aesthetic forgeom_rect, but let me refresh my memories
Aaron Lun (05:15:00): > I just want to draw a damn rectangle
Kevin Rue-Albrecht (05:15:20): > omg.. unrelated but we may be able to draw heat maps efficiently withggplot: “geom_raster is a high performance special case for when all the tiles are the same size.”
Kevin Rue-Albrecht (05:15:44): > it wasn’t there last time i checked, or I’ve never properly read the man page
Aaron Lun (05:16:14): > Maybe I need to get rid of the quotes.
Kevin Rue-Albrecht (05:16:19): > I’m sure you’ve read the man page, but here goes: > “geom_rect and geom_tile do the same thing, but are parameterised differently: geom_rect uses the locations of the four corners (xmin, xmax, ymin and ymax), while geom_tile uses the center of the tile and its size (x, y, width, height)”
Kevin Rue-Albrecht (05:16:36): > ahhhh yes
Kevin Rue-Albrecht (05:16:40): > i didn’t spot those
Aaron Lun (05:17:00): > Nope, that didn’t help.
Kevin Rue-Albrecht (05:17:13): > are you serious?! dammit
Kevin Rue-Albrecht (05:17:51): > Still from the man page > > # If you want to draw arbitrary rectangles, use geom_tile() or geom_rect() > df <- data.frame( > x = rep(c(2, 5, 7, 9, 12), 2), > y = rep(c(1, 2), each = 5), > z = factor(rep(1:5, each = 2)), > w = rep(diff(c(0, 4, 6, 8, 10, 14)), 2) > ) > ggplot(df, aes(xmin = x - w / 2, xmax = x + w / 2, ymin = y, ymax = y + 1)) + > geom_rect(aes(fill = z, width = w), colour = "grey50") >
Aaron Lun (05:18:37): > Okay, let’s have a look at what is causing the issue. > > plot.data <- data.frame(Y = colData(se)[,"NALIGNED"], row.names=colnames(se)); > plot.data$X <- factor(integer(ncol(se))) > plot.data$X <- as.factor(plot.data$X); > > # Defining the plot boundaries > ybounds <- range(plot.data$Y, na.rm = TRUE); > > # Setting up the data points > plot.data <- subset(plot.data, (X) & (Y)); > set.seed(100); > plot.data$jitteredX <- vipor::offsetX(plot.data$Y, > x=plot.data$X, width=0.4, varwidth=FALSE, adjust=1, > method='quasirandom', nbins=NULL) + as.integer(plot.data$X); > plot.data$GroupBy <- plot.data$X; > > # Generating the plot > ggplot(plot.data, aes(x = X, y = Y, group = GroupBy)) + > geom_violin(alpha = 0.2, scale = 'width') + > geom_point(aes(y = Y, x = jitteredX), alpha = 0.6, size = 1) + > labs(x = "", y = "NALIGNED") + > coord_cartesian(xlim = NULL, ylim = ybounds, expand = TRUE) + > # geom_rect(aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax), data=data.frame(xmin = 0.94554, xmax=1.1043, ymin = 1.4775e+07, ymax = 3.7462e+07)) + > scale_x_discrete(drop = FALSE) + > theme_bw() + > theme(legend.position = 'bottom') > > # Saving for brush transmission > all.coordinates[['colDataPlot1']] <- plot.data >
Aaron Lun (05:19:17): > Doesn’t like it when that line is uncommented.
Aaron Lun (05:19:34): > Does it need to be earlier? Does the order matter?
Kevin Rue-Albrecht (05:19:59): > order doesn’t matter, only if you’re concerned about showing a layer on top of another
Kevin Rue-Albrecht (05:20:15): > ahh wait
Charlotte Soneson (05:20:26): > my guess: it is looking for the X that you specify in the ggplot call
Kevin Rue-Albrecht (05:20:28): > try puttinginherit=FALSE
Kevin Rue-Albrecht (05:20:32): > exactly
Kevin Rue-Albrecht (05:20:52): > ```
Kevin Rue-Albrecht (05:20:55): > geom_rect(mapping = NULL, data = NULL, stat = “identity”, > position = “identity”, …, na.rm = FALSE, show.legend = NA, > inherit.aes = TRUE) > ```
Kevin Rue-Albrecht (05:20:59): > inherit.aes
Aaron Lun (05:21:04): > Yep, that’s better.
Aaron Lun (05:21:11): > Though it still autofills, I wonder how to turn that off.
Kevin Rue-Albrecht (05:21:48): > fill = NULLmaybe, outside the aes
Kevin Rue-Albrecht (05:21:55): > i can’t remember the value to turn it off
Charlotte Soneson (05:22:01): > maybe cheating, butalpha = 0, color = "black"
Charlotte Soneson (05:22:36): > outside theaes
Aaron Lun (05:26:28): > God, it’s beautiful.
Kevin Rue-Albrecht (05:27:35): > talk to yourself much?:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (05:31:16): > when’s the big push for us to see? (kid before Xmas)
Kevin Rue-Albrecht (06:15:46): > damn it’s beautiful indeed
Federico Marini (06:20:49): > Envy-boost
Federico Marini (06:20:59): > soon comes the pimped-up graph too:slightly_smiling_face:
Federico Marini (06:24:32): > @Federico Mariniuploaded a file:graph-y - File (PNG): graph-y
Federico Marini (06:24:59): > colors taken from the plot types itself
Aaron Lun (06:25:54): > The latest commit defines explicit colors as well - make sure these go somewhere centralized. It’s a shame thatboxdoesn’t allow us to specify arbitrary colors, so we just have to play along with their color scheme.
Federico Marini (06:30:55): > where would be the best place?constants.RI’d say
Aaron Lun (06:31:19): > Yes, probably.
Kevin Rue-Albrecht (06:31:27): > yes, if they’re shared across scripts
Aaron Lun (06:31:30): > Anyway, brushing memory is done. Try to break it.
Kevin Rue-Albrecht (06:31:41): > otherwise, inside the script that uses them is probably best
Aaron Lun (06:32:29): > Dealing with restriction across great grandchildren was particularly problematic, so be sure to test out complex networks.
Kevin Rue-Albrecht (06:33:20): > i still have plans to set up ‘modes’ such as FACS, that would test this kind of situation
Charlotte Soneson (07:35:13): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-26 at 13.30.34.png and commented: Trying to break the brushing memory, I ran into the following: > - Set Column data plot 1 to Receive brush from Reduced dimension plot 1 - effect Restrict > - Brush in Reduced dimension plot 1 > - Brush in Column data plot 1 (although nothing is receiving) > - Change in Column data plot1 to Receive brush from Column data plot 1 - effect Color > This gives the attached plot - File (PNG): Screen Shot 2018-01-26 at 13.30.34.png
Charlotte Soneson (07:35:17): > Shouldn’t this be the same as: > - Set Column data plot 1 to Receive brush from Column data plot 1 - effect Color > - Brush in Column data plot 1 > If not, perhaps we need to indicate the whole sequence of brushes for Column data plot 1 (that it remembers what was done before, even if you change the Receive brush from)
Aaron Lun (07:40:36): > Should just be a case of retriggering the plot when updating the selection - let me see.
Aaron Lun (08:19:07): > Okay, not seeing this on my version, but I changed half a dozen things.
Charlotte Soneson (08:27:15): > Weird, it still happens for me. Things work fine if you move the Column data plot 1 brush slightly after setting Column data plot 1 to receive its own brush, it just doesn’t seem to detect the brush if it is already there. And it is only for going from Restrict -> Color, if I go via Transparent it works…:thinking_face:
Charlotte Soneson (08:30:46): > Actually, the problem rather seems to be when you change “Receive the brush from” of Column data plot 1, from Reduced dimension plot 1 to Column data plot 1, if the Brush effect is Restrict, it seems to intersect the two brushes, while for the other two effects, it changes to the one from Column data plot 1
Aaron Lun (08:44:27): > Intriuging.
Aaron Lun (08:45:03): > Refactoring another thing right now, but can you just spell out how to reproduce it? I’ll get onto it.
Charlotte Soneson (08:49:27): > 1. Coldata1: Receive brush from Reddim1, effect = Restrict > 2. Brush some points in Reddim1 > 3. Brush some points in Coldata1 > 4. Change for Coldata1: Receive brush from Coldata1 > This selects only the intersection of the brushes in 2 and 3. > If another effect is used, step 4 selects the Coldata1 brush only.
Aaron Lun (08:55:14): > Okay. There’s a lot more holes in the brushing memory than I thought, so this will take a while.
Aaron Lun (10:10:31): > Hm. I think I understand the origin of the bug - it’s because the points stored inall.coordinatesare restricted. The plot needs to be regenerated to update the stored points, which it does, but not before (incorrectly) plotting the previously restricted set.
Aaron Lun (10:11:34): > I think we need to reorder our plotting commands to delay brushing until just before plot generation. This would solve this problem and also help a lot otherwise - check out the painful loops in the reported code to handle self-brushing.
Kevin Rue-Albrecht (10:17:36): > argh - i think we owe you quite a few pints when all of this is over!
Aaron Lun (10:24:25): > Don’t touchplotting.Rfor a while, this is going to be crazy again.
Kevin Rue-Albrecht (10:24:50): > ah - good that i delayed the colormap to this weekend then?
Aaron Lun (10:25:04): > Probably.
Kevin Rue-Albrecht (10:26:09): > I know most of it is going to happen inAllClasses.R, but I have a feeling I’ll have to tweak a thing or two inplotting.R
Davis McCarthy (12:42:08): > Hey gang - I’m ju
Davis McCarthy (12:42:34): > I’ve just submitted a PR adding a dockerfile and install.R script to be able to build a working docker image for the package
Davis McCarthy (12:43:21): > I’ve added it to my builds onquay.io , so you can play around with it if you’d like:quay.io/davismcc/isee:dockerfile
Davis McCarthy (12:44:03): > if you have docker installed: > > docker pull[quay.io/davismcc/isee:dockerfile](http://quay.io/davismcc/isee:dockerfile)docker run -it[quay.io/davismcc/isee:dockerfile](http://quay.io/davismcc/isee:dockerfile)/bin/bash >
Davis McCarthy (12:50:05): > Is there a way to specify a port for the shiny app?
Kevin Rue-Albrecht (13:19:11): > not this one, yet, but it is possible withrunApp(..., port=...)
Kevin Rue-Albrecht (13:23:05): > I can switch that
Kevin Rue-Albrecht (13:36:52): > pushing in 3…2…1
Kevin Rue-Albrecht (13:47:44) (in thread): > I’ve enabled that now: > > > iSEE(sce, port = 9999) > > Listening on[http://127.0.0.1:9999](http://127.0.0.1:9999) >
Kevin Rue-Albrecht (13:49:44) (in thread): > nowiSEEsupports all the arguments thatshiny::runAppdoes. I don’t see another use for the ellipsis iniSEEanyway
Aaron Lun (14:25:48): > Just got back from a mad party
Aaron Lun (14:26:32): > This docker stuff won’t interfere with the rest of the package, right?
Federico Marini (14:42:07): > don’t know exactly
Federico Marini (14:42:20): > are there alternative locations to place it?
Federico Marini (14:45:48): > <!channel>charlotte and I are done with the bulk of the vignette. some parts need to be slightly edited, after the ui edits. and the use case where we showcsase iSEE with CyTOF data will follow too. > What is missing as of now: > - finalize the bibliography > - the screenshots (placeholders are there []) > - (slightly related: do you know how to open up the vignette from a shiny app?)
Federico Marini (14:46:06): > If you have nothing against it, I’d merge intomaster
Aaron Lun (14:46:23): > Do we need screenshots? I mean, it would be a pain to have to update things.
Federico Marini (14:46:27): > there is also the work on plotting the linked-plots graph
Federico Marini (14:46:36): > I know
Aaron Lun (14:46:43): > People should be able to just run the code and see what it looks like for themselves, right?
Federico Marini (14:46:50): > We could try to keep it general
Aaron Lun (14:48:28): > Yeah
Federico Marini (14:48:36): > You know what, let me merge it (unless big changes are expected?) and you can judge better whether you see the strict need of it
Aaron Lun (14:49:39): > Okay. Currently no screenshots, right?
Federico Marini (14:49:43): > nope
Federico Marini (14:49:53): > only [] placeholders []
Aaron Lun (14:53:21): > cool.
Federico Marini (14:59:50): > merged now intomaster
Kevin Rue-Albrecht (15:12:46) (in thread): > to place what?
Kevin Rue-Albrecht (15:12:51) (in thread): > @Federico Marini
Federico Marini (15:14:43) (in thread): > to place the`Dockerfile``
Kevin Rue-Albrecht (15:16:42) (in thread): > @Federico Marini, wanna see ifvignette("iSEE_vignette")can be launched from a button in-app?
Kevin Rue-Albrecht (15:17:21) (in thread): > there’s a chance that it won’t trigger until the app’s closed and the prompt is available again
Federico Marini (15:17:52) (in thread): > I think I tried that already. Nothing happened
Federico Marini (15:18:19) (in thread): > hence the “weirder” choice of calling it
Federico Marini (15:18:24) (in thread): > I’ll search a little more
Kevin Rue-Albrecht (15:19:38) (in thread): > ah:confused:
Aaron Lun (15:58:16): > se <- as(se, "SummarizedExperiment")????
Aaron Lun (16:00:59): > I think@Charlotte Soneson’s bug is fixed in the latest commit tohardened. Boy, was this epic.
Aaron Lun (16:02:06): > Wait, haven’t actually committed it yet.
Federico Marini (16:02:28): > git commitor it didn’t happen
Aaron Lun (16:03:01): > Done.
Kevin Rue-Albrecht (16:12:58): > heya
Kevin Rue-Albrecht (16:13:20): > aaargh
Kevin Rue-Albrecht (16:13:27): > i just realised how stupid
Kevin Rue-Albrecht (16:13:32): > omg
Aaron Lun (16:14:34): > ggplot team, note that the boundaries of the violin plots do not render properly when no points are available. This can be easily seen from the latest version, by (1) gettingcolDataPlot1to receive fromredDimPlot1, (2) setting upcolDataPlot1to restrict, and (3) brushing somewhere without points inredDimPlot1. Axis information completely vanishes fromcolDataPlot1- however, it is very important that it doesn’t do that.
Kevin Rue-Albrecht (16:15:13): > i meanas(se, 'SummarizedExperiment')does make sense to bring inExpressionSetas well
Aaron Lun (16:15:33): > This is fixed in the incoming PR.
Aaron Lun (16:18:02): > There is still one sort-of outstanding issue with the brush memory; selecting a new gene from a linked table will cause double rendering of the plot, because two updates are triggered. Not quite sure how to stop this.
Kevin Rue-Albrecht (16:18:58): > do you have at least an idea of what the two causes of the updates are?reqcan sometimes help
Aaron Lun (16:19:22): > Yes, I do.
Aaron Lun (16:19:30): > I know exactly why.
Aaron Lun (16:19:43): > But I can’t stop it without changing a whole bunch of other things.
Aaron Lun (16:20:00): > Because I have to intercept the messages passed around theShiny networks
Kevin Rue-Albrecht (16:20:10): > Basically, it’s avalidatewithout an error message. Used with caution, you can block an update until something is valid
Aaron Lun (16:21:27): > Hm…
Kevin Rue-Albrecht (16:21:48): > typically, i use it for the startup of an app, because the inputs being initialised with a slight time offset can cause a plot to be rendered multiple times as all its inputs are being initialised
Kevin Rue-Albrecht (16:21:56): > dunno if that can help in this case
Aaron Lun (16:22:22): > I will think about it.
Aaron Lun (16:22:38): > Meanwhile, see if you can fix the violin plot axis problem.
Aaron Lun (16:22:43): > Probably affects the boxplots as well.
Aaron Lun (16:26:03): > Going home now.
Aaron Lun (16:50:50): > I think the only solution is to build a parallel network for the table->plot linkages. Which we can do. I guess that’ll be my job tomorrow.
Aaron Lun (16:51:05): > Anticipated changes toplotting.Rshould be minimal, so go ahead with the colormap.
Kevin Rue-Albrecht (16:51:19): > ok
Aaron Lun (16:51:38): > Do people enjoy the color-coding on the brush as well?
Aaron Lun (16:52:18): > Stroke of inspiration, that.
Kevin Rue-Albrecht (16:52:45): > color-coded brush was such a cool surprise: it really adds a thoughtful touch
Kevin Rue-Albrecht (16:55:24): > vignette team: i think there’s no shame hiding some of the warnings (e.g. Quick start section, awarning=FALSEwouldn’t hurt), just for the sake of not distracting the reader
Kevin Rue-Albrecht (16:57:10): > also, about the introJS tour, considering that you explain how to use it, I noticed a cool thing: if one wants to jump to any step without having to go through all the earlier ones, one can click on the circles that represent the individual steps.
Kevin Rue-Albrecht (16:59:29): > Good job writing it: i’ll continue reading later, though. I’ll try to get at least a base of colormap done tonight
Aaron Lun (17:04:40) (in thread): > HERE.
Aaron Lun (17:41:52): > Gene expression plots are currently a bit funky with respect to unnecessary re-renderings when an axis parameter is changed. This will be fixed tomorrow.
Kevin Rue-Albrecht (17:56:35): > ok. immediate question just opening the app: redDim plot doesn’t display on startup: any default changed?
Kevin Rue-Albrecht (17:57:45): > maybe it’s just a glitch on my side, re-installing now
Kevin Rue-Albrecht (17:58:18): > yep. nevermind
Aaron Lun (18:00:53): > Yes, it was caused by theas(..., "SummarizedExperiment")that I mentioned earlier.
Aaron Lun (18:01:13): > That wipes out the reduced dim slot and replaces it with an empty one.
Kevin Rue-Albrecht (18:07:37): > Makes sense. My bad. Friday night brain short-circuit:sweat_smile:
2018-01-27
Charlotte Soneson (03:34:45): > The latest commit indeed fixes the problem I had:+1:Now, for the violins, perhaps the “safest” way that I can think of to keep the axes is to create an additional data frame, containing all the points, and add ageom_blank(data = full.data)to theggplotcall. This will keep also the x-axis even if there are no points. However, it might look weird to people when it is not necessary (which I suppose will be always when people export the code, I doubt anyone will export the code for an empty plot…) Other options might be to addexpand_limits(y = ybounds), which keeps the y-axis (forces inclusion of the largest and smallest values), but does not show the0on the x-axis.
Kevin Rue-Albrecht (04:48:45): > last extreme: we could always add avalidate(need(...))no?
Kevin Rue-Albrecht (04:52:56): > although i’m not sure whether that would preserved the desired effect on the tracker script and the downstream linked plots
Aaron Lun (06:46:29): > I like thegeom_blankidea.
Kevin Rue-Albrecht (06:46:41): > sounds cleaner too
Aaron Lun (07:11:31): > Though by the time it gets togeom_blank, the full data set does not exist anymore.
Aaron Lun (07:12:16): > It would not be so painful to get it to exist… but it would still be a little painful.
Kevin Rue-Albrecht (07:13:35): > in comparison to the pain you went through for brushing ^^
Aaron Lun (07:44:12): > If we havegeom_blank, does that mean we don’t needscale_x_discreteetc. anymore?
Kevin Rue-Albrecht (07:47:04): > you don’t need to worry about scales i thinkgeom_blankclears everything
Kevin Rue-Albrecht (07:48:06): > oh no, sorry, I was thinkingtheme_void
Kevin Rue-Albrecht (07:59:55): > I’ve finally just got around to test the situation. I’m fairly sure the X-tick disappearing cannot be fixed by ggplot anymore. It’s because the data.frame is empty and therefore X does not have levels anymore. I imagine the fix would be to addplot.data$X < factor(character(), "0")to force a factor of length 0 that has a level defined - Attachment: Attachment > ggplot team, note that the boundaries of the violin plots do not render properly when no points are available. This can be easily seen from the latest version, by (1) getting colDataPlot1 to receive from redDimPlot1, (2) setting up colDataPlot1 to restrict, and (3) brushing somewhere without points in redDimPlot1. Axis information completely vanishes from colDataPlot1 - however, it is very important that it doesn’t do that.
Aaron Lun (08:00:19): > ? An empty data frame should stil have levels, surely?
Kevin Rue-Albrecht (08:01:02): > i’d expect it to, but maybe some of theplot.dataprocessing appliesdroplevels()implicitly (?)
Aaron Lun (08:01:15): > Only subset would do this.
Aaron Lun (08:01:27): > but it already hasdrop=FALSEby default.
Kevin Rue-Albrecht (08:02:45): > i’ll have to run the output script to see what goes on, and maybe throw in a couple of debug print statements to have runtime information
Aaron Lun (08:03:23): > Yep.
Aaron Lun (08:03:43): > Just give me a MWE for how to get it to work, and we can figure it out from there.
Kevin Rue-Albrecht (08:04:00): > Ok.
Kevin Rue-Albrecht (08:06:39): > Hm.. as we expected the length-0 factor has a level. So itis ggplot dark magic at work > > > plot.data$X > factor(0) > Levels: 0 >
Kevin Rue-Albrecht (08:06:58): > (the resulting violin having no X axis tick in this case)
Kevin Rue-Albrecht (08:22:40): > I may be homing in on a fix: > > discrete_scale(..., na.translate = TRUE, na.value = NA, drop = TRUE ...) >
Kevin Rue-Albrecht (08:23:56): > basically, violin don’t like empty tables:discrete_scale(drop=FALSE)only seems to work is one data pointis present
Kevin Rue-Albrecht (08:24:34): > I can sort of rescue that aspect byif(nrow(plot.table)==0){rbind(NA)}(invalid code to give you the jist)
Kevin Rue-Albrecht (08:25:16): > but then, I get an extra tick forNAlevel because of ourdrop=FALSEwish
Kevin Rue-Albrecht (08:25:57): > now, we can’t translateNAto the0level.. otherwise we’d be creating a level….. or not, because Yis NA!
Kevin Rue-Albrecht (08:26:01): > let me try that
Aaron Lun (08:27:17): > Good grief. ggplot makes hard things simple and simple things hard, so it seems.
Kevin Rue-Albrecht (08:28:24): > always the same story in softare dev: 90% of feature in 10% of the time and vice versa
Kevin Rue-Albrecht (08:28:42): > damn corner cases
Kevin Rue-Albrecht (08:32:13): > damn, can’t get rid of the damnNAnow. Feel free to play with > > plot.data <- data.frame( > Y = numeric(), > X = factor(levels = "0"), > BrushBy = logical(), > GroupBy = factor(levels = "0"), > jitteredX = numeric() > ) > if (identical(nrow(plot.data), 0L)){ > dummy <- matrix(NA, nrow = 1, ncol = ncol(plot.data), dimnames = list(character(), colnames(plot.data))) > dummy <- as.data.frame(dummy) > dummy$X <- factor(dummy$X, levels(plot.data$X)) > plot.data <- rbind(plot.data, dummy) > } > > # Creating the plot > ggplot(plot.data, aes(x = X, y = Y, group = GroupBy)) + > geom_violin(data = plot.data, alpha = 0.2, scale = 'width', na.rm = TRUE) + > geom_point(aes(y = Y, x = jitteredX), plot.data) + > labs(x = "", y = "NREADS") + > coord_cartesian(xlim = NULL, ylim = ybounds, expand = TRUE) + > scale_x_discrete(drop = FALSE, na.translate = TRUE, na.value = 0) + > theme_bw() + > theme(legend.position = 'bottom') >
Kevin Rue-Albrecht (08:32:54): > … after picking the rest of the script from the scenario that I copy pasted from Aaron’s earlier script a few messages above
Kevin Rue-Albrecht (08:33:20): > ah no wait, I can edit to give a MWE without the hassle
Kevin Rue-Albrecht (08:35:10): > here you go
Kevin Rue-Albrecht (08:37:34): > there’s a funny ‘NA -’ that appears in the bottom left corner though, with this approach
Aaron Lun (08:37:54): > Well, I’m still dealing with the table thing I mentioned yesterday,…
Aaron Lun (08:39:37): > real knockdown dragout battle
Kevin Rue-Albrecht (08:45:42): > I think I got it. Thanks to@Charlotte Soneson(again!)
Kevin Rue-Albrecht (08:47:45): > It’s all in thegeom_blank, and a extrafull.data
Charlotte Soneson (09:01:00): > Maybe we don’t even need to create two data frames.ggplot’sgeom_*can apparently take as adataobject a function, which takes as single argument the top-level data frame, and returns a data frame that will be used for plotting. So if we provide the full data frame, with a columnBrushBy, a function that subsets by this column can be used as thedataargument for all parts of theggplotcommand that should use only the brushed points. And I guess we no longer need thecoord_cartesian(...)andscale_x_discrete()stuff.
Charlotte Soneson (09:01:07): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-27 at 15.00.41.png - File (PNG): Screen Shot 2018-01-27 at 15.00.41.png
Aaron Lun (09:01:37): > Note thatcoord_cartesianis still necessary for zooming, andcoord_flipis still necessary for flipping.
Charlotte Soneson (09:01:54): > Aha, ok
Aaron Lun (09:24:27): > This table thing is hellish beyond words
Kevin Rue-Albrecht (09:25:06): > have you started drawing dependency trees on your house walls? ^^
Aaron Lun (09:25:22): > I can see them when I close my eyes.
Aaron Lun (09:54:46): > Okay, I feel I’m halfway there. I have purged the reactives in the plotting code, which now only depends on memories inpObjects. This makes it a lot easier to control and test.
Aaron Lun (09:55:05): > Now I just need to make sure that the tables update properly when removing panels.
Kevin Rue-Albrecht (09:55:11): > any major impact on the colormaps?
Aaron Lun (09:55:30): > I don’t think so, but have a look at the changes inplotting.R.
Aaron Lun (09:55:44): > Long story short,inputdoes not exist inplotting.Rany more (besides in the validation messages).
Aaron Lun (10:25:39): > God, it’s over.
Kevin Rue-Albrecht (10:27:45): > Well done. I’ve only truly started getting into the colormap defaults 10 min ago. I’m applying the (1) assay name/index, (2) all_assay, (3) global, (4) hard-coded default strategy. I’ll keep you posted.
Kevin Rue-Albrecht (10:28:37): > would you expect support for an index-based colData colormap, in addition to anames(colData)one?
Kevin Rue-Albrecht (10:28:49) (in thread): > i hope you take a break:wink:
Aaron Lun (10:29:09): > No, coldata should have unique names I think.
Kevin Rue-Albrecht (10:30:00): > I think so too: i foresee hell of a mess with the large number of columns in that slot.
Kevin Rue-Albrecht (10:33:11): > Peek on theshowmethod: > > > ecm > Class: ExperimentColorMap > assays(0): > colData(0): > rowData(0): > all(1): assays > global: TRUE >
Kevin Rue-Albrecht (10:33:51): > allis a slot that may contain one colormap per slot
Aaron Lun (10:33:58): > what’s with theTRUE? As in whether it exists or not?
Aaron Lun (10:34:04): > set or not, I mean.
Kevin Rue-Albrecht (10:34:15): > yes.. i’ve been thinking for 5 min what to show for it
Kevin Rue-Albrecht (10:34:58): > hence my post for feedback before I push something that I know is not ideal
Kevin Rue-Albrecht (10:35:23): > For theallslot, theshowmethod list the slot names for which a shared fallback colormap was defined
Kevin Rue-Albrecht (10:37:14) (in thread): > Is that just weird? > > > ecm > Class: ExperimentColorMap > assays(0): > colData(0): > rowData(0): > all(1): assays > global >
Kevin Rue-Albrecht (10:37:30) (in thread): > Or maybe > > > ecm > Class: ExperimentColorMap > assays(0): > colData(0): > rowData(0): > all(1): assays > global(1) >
Kevin Rue-Albrecht (10:37:45) (in thread): > for homogeneity
Aaron Lun (10:38:21): > Note that we’ll need discrete vs continuous options for everything.
Kevin Rue-Albrecht (10:38:55): > yeah - that’s the part that keep giving me headaches
Kevin Rue-Albrecht (10:39:51): > I mean, once past the individual colormaps, can’t we ask users to supply functions that have adiscrete=TRUE|FALSEargument?
Aaron Lun (10:39:51): > Oh? Just have each list with a discrete or continuous sublist.
Aaron Lun (10:40:10): > Not really.
Aaron Lun (10:40:32): > That would prevent the direct use ofviridis::viridis, etc.
Aaron Lun (10:40:53): > You would basically force everyone to define a function wrapper around built-ins.
Kevin Rue-Albrecht (10:41:13): > yeah. i really need to thinkbefore i write here
Kevin Rue-Albrecht (10:49:51): > in the happy news, I’ve found a generator function for any number of discrete colors:https://stackoverflow.com/questions/8197559/emulate-ggplot2-default-color-palette - Attachment (stackoverflow.com): Emulate ggplot2 default color palette > What function can I use to emulate ggplot2’s default color palette for a desired number of colors. For example, an input of 3 would produce a character vector of HEX colors with these colors:
Kevin Rue-Albrecht (12:40:24): > thisExperimentColorMapwas either a moment of inspiration, or a one-way ticket to hell:sweat_smile:
Aaron Lun (12:40:51): > I’m all done on my end, waiting for CI to clear for merge.
Aaron Lun (12:42:41): > @Federico MariniTable links are now available inpObjects$table_links. These should probably be added to the network.
Kevin Rue-Albrecht (12:44:17): > I was thinking about a different approach:scale_color_manual(values=assayColorMap(map, assay_choice))(10) +written as such in the script
Kevin Rue-Albrecht (12:44:46): > (almost as such, substituteassay_choice)
Aaron Lun (12:45:05): > That is one alternative.
Kevin Rue-Albrecht (12:45:12): > implying thatassayColorMap(map, assay_choice))return a simplefunction(n)
Kevin Rue-Albrecht (12:45:19): > where n is the number of colors requested
Aaron Lun (12:45:31): > I’m fine with that, though you will need to modifycodetrackto define the map at the top.
Kevin Rue-Albrecht (12:46:10): > yep - i was tracking down all thecolor_FUNto delete as well, considering that there’s a lot less to compute before writing the script
Kevin Rue-Albrecht (12:46:49): > there are just a lot of ellipses that appeared throughoutplotting.Rrecently ….:smirk:
Kevin Rue-Albrecht (12:47:31): > (maybe not a lot, but it’s taking me time to figure out what passes where)
Aaron Lun (12:47:51): > Wait - why are you deletingcolor_FUN?
Kevin Rue-Albrecht (12:48:29): > well i did, but then i’ve reverted all the code back because i’m pretty sure it was a mistake too
Aaron Lun (12:48:44): > Your functions don’t seem to take a discreteness argument, so you needcolor_FUNto make that choice.
Kevin Rue-Albrecht (12:50:05): > yep, don’t worry, I was toying around to catch up on the latest updates, but i realised i was making a mess and i’ve reverted back to a clean slate 10 min ago
Kevin Rue-Albrecht (12:50:23): > i just kept the new structure of ECM that I;ve described earlier
Aaron Lun (13:34:37) (in thread): > I would also suggest moving the code to generate the graph intocodetrack.R, to reduce the size ofiSEE.R, which is already over 500 lines.
Aaron Lun (13:41:35): > A whole bunch of things are merged. vignette team: note some minor changes in the interface.
Kevin Rue-Albrecht (13:41:57): > Cool. Well done, seriously.
Kevin Rue-Albrecht (13:42:46): > If my battle plan is correct, I’m close to writing the correct command for the scatter plot, which should then generalise fairly easily to the others
Kevin Rue-Albrecht (13:43:25): > i’ll have to go out for a bit though before I have time to test
Kevin Rue-Albrecht (13:43:40): > your color_FUN is still there Aaron:wink:
Aaron Lun (13:44:16): > Excellent.
Aaron Lun (14:01:12): > Done for the day, bugging out.
Federico Marini (14:23:48) (in thread): > on it soon
Federico Marini (14:24:07) (in thread): > I am not so fmiliar with igraph but it should not be rocket science
Federico Marini (14:37:00): > wow, the ui gets pimped up once again:slightly_smiling_face:
Federico Marini (14:38:00): > one thing to be kept in mind: I can not access the ui elements from the modals for the introjs tour - at least, with my actual knowledges
Federico Marini (14:39:31): > and a comment on the trash icon: I’d keep it somewhat more highlighted
Federico Marini (14:40:15): > not that users mis-click it? thinking of fitts’ law
Federico Marini (14:49:39): > I can not open more than one gene stats table now
Federico Marini (14:51:04): > fails with Warning: Error in strsplit: non-character argument > No stack trace available
Federico Marini (15:22:37): > lost in list:stuck_out_tongue:
Kevin Rue-Albrecht (15:22:44): > ?
Federico Marini (15:27:56): > trying to figure out how to tidily extract the table-plot relationships, and put them in the graph
Kevin Rue-Albrecht (15:28:05): > ahh
Kevin Rue-Albrecht (15:29:05): > i’m personally lost in inception: because of this damn discrete/continuous ambivalence, i’m currently returning a function that returns one of two functions depending on its argument:dizzy_face:
Federico Marini (15:34:12): > oh
Kevin Rue-Albrecht (15:48:30): > weird.. it seems thatcolor_discretealways evaluates toFALSEeven with clearly discrete factors
Kevin Rue-Albrecht (15:48:34): > investigating
Kevin Rue-Albrecht (15:49:07): > I just put aprint(coloring)aftercoloring <- eval_env$plot.data$ColorByinplotting.R
Charlotte Soneson (15:58:31): > I just pushed thefixaxisbranch, which (I think) contains a fix for the disappearing axis when the brush is empty. Let me know if there are any weird things happening… In the end, I created another data frame with all the values, and supplied that togeom_blank. The other option would be to subset within the plot commands, as well as when adding toall_coordinates.
Kevin Rue-Albrecht (16:03:35): > Ok. I’m starting to wonder whether something got broken in.create_plotwhen checking if something is groupable
Kevin Rue-Albrecht (16:04:00): > coloring <- eval_env$plot.data$ColorByseems to constantly returnNULL
Kevin Rue-Albrecht (16:06:38): > i’ll merge in the latest updates to master, to see if that does any good
Federico Marini (16:14:28): > do you guys also have the same issue of creating extra gene tables?
Kevin Rue-Albrecht (16:14:58): > i haven’t tried yet. I’m tracking where on earth the commands that createColorByhave gone
Kevin Rue-Albrecht (16:16:58): > i think i’ve got the logic in place to supply all kinds of colormaps, but somehow every colData covariate is now detected as continuous, even though i haven’t touched that: colormap selection is downstream of that detection
Charlotte Soneson (16:17:22): > Yes, I have the same problems with additional gene tables. It is created, then the app crashes
Kevin Rue-Albrecht (16:17:58): > i’m adding print statements all over the place to track down where things go wrong
Federico Marini (16:18:50): > ok - “good” to know it is not just me
Federico Marini (16:20:14): > @Federico Mariniuploaded a file:Screen Shot 2018-01-27 at 22.19.51.png - File (PNG): Screen Shot 2018-01-27 at 22.19.51.png
Federico Marini (16:20:30): > rudimental (the code) but yet seems to be working
Federico Marini (16:20:37): > at least with 1 tbl
Federico Marini (16:21:07): > I feel I am using the most stupid way possible the igraph functions, i.e. brutal for loops
Federico Marini (16:21:16): > but they seem to do the job
Kevin Rue-Albrecht (16:24:52): > anyone knows abouteval_env$plot.data, or was it Aaron who refactored it? > i think there are some naming issues that lead toNULLduring evaluation
Federico Marini (16:25:10): > wasntme
Kevin Rue-Albrecht (16:25:35): > “but she caught me on the counter”:notes:
Kevin Rue-Albrecht (16:25:56) (in thread): > https://www.youtube.com/watch?v=ZyeOvBjVlfI - Attachment (YouTube): SHAGGY-LICIOUS - It Wasn’t Me (Shaggy Cover)
Federico Marini (16:26:22): > I am torn between saying again “wasntme” and not contributing to reaching so fast the 10k messages limit
Kevin Rue-Albrecht (16:26:31): > hehehe
Kevin Rue-Albrecht (16:26:40): > i think we boosted that upward recently
Kevin Rue-Albrecht (16:27:52): > anyway, I think I understand now, I’m pretty sure all theeval_env$plot.data$...things are evaluated asNULL
Charlotte Soneson (16:28:32): > Maybe the problem with the new table is that no other plots are using it, so it doesn’t have any children, andiSEE.Rline 747 goes crazy when trying to split NULL
Federico Marini (16:30:02): > i did not track it yet but you might be very right
Federico Marini (16:31:21): > from the .split_encoded call
Aaron Lun (16:31:52): > Wait wait wait
Aaron Lun (16:31:57): > wHat’s going on.
Kevin Rue-Albrecht (16:32:03): > I think my issue comes from a misplace command betweendata_cmdsandsetup_cmds
Kevin Rue-Albrecht (16:32:10): > now i can see stuff
Kevin Rue-Albrecht (16:32:32): > :sob:
Aaron Lun (16:34:31) (in thread): > Subsetting within the plot commands is not easy, because the calculations for the quasi random scatter are done on the subset. It’s not impsosible, but we’d have to subset it; do the calculations; and then restore the full dimensions back again.
Aaron Lun (16:34:55) (in thread): > This might be something with.split_encoded.
Federico Marini (16:35:17) (in thread): > yep
Federico Marini (16:35:27) (in thread): > charlotte helped spotting it out
Charlotte Soneson (16:35:44) (in thread): > Do the scatter calculations need to be done on the subset, or could they be done before and then everything subsetted?
Federico Marini (16:35:52) (in thread): > I have the grpah ready also with the table
Federico Marini (16:35:58) (in thread): > tables, I could not test yet
Federico Marini (16:36:06) (in thread): > but I think it might be robust
Charlotte Soneson (16:36:16) (in thread): > That would also place the subsetted scattered points in the same place as they are in the full data
Aaron Lun (16:37:02) (in thread): > My guess is that.split_encodeddoes not like being fedNULL.
Federico Marini (16:38:05) (in thread): > which would be the case whrn the new table gets created and noone cares about her
Kevin Rue-Albrecht (16:38:36): > setup_cmds[["color"]] <- color_out$cmdshould bedata_cmds[["color"]] <- color_out$cmd
Kevin Rue-Albrecht (16:38:58): > otherwise,ColorBydoes not exist in time to be checked for ‘groupability’ in.create_plot
Aaron Lun (16:39:31): > Be careful about shuffling things around.
Kevin Rue-Albrecht (16:40:18): > well, I’ve been _not_shuffling around things for the last few hours, and was wondering whycoloringwas constantlyNULL
Aaron Lun (16:40:31): > Where are you putting your colorby command?
Aaron Lun (16:40:51): > It should be going intoplot_cmds, right?
Kevin Rue-Albrecht (16:41:07): > i think we’re not talking about the same commands
Kevin Rue-Albrecht (16:41:41): > i’m talking about the command that creates theColorBycolumn, not the one(s) that uses it to color layers
Kevin Rue-Albrecht (16:42:17): > In.create_plotyou have a line that sayscoloring <- eval_env$plot.data$ColorBy
Aaron Lun (16:42:24): > Yes
Aaron Lun (16:42:25): > Right
Aaron Lun (16:42:36): > Hm. This is troublesome.
Aaron Lun (16:42:53): > It needs to go intosetup_cmds, otherwise the distinctions between the groups of commands fall apart.
Kevin Rue-Albrecht (16:42:55): > however, I’ve printedeval_env$plot.data, and that shows no command about setting upColorByeven when colors are selected
Kevin Rue-Albrecht (16:43:19): > thereby,eval_env$plot.data$ColorBysystematically returnsNULL
Aaron Lun (16:43:38): > Yes, I understand the problem.
Kevin Rue-Albrecht (16:44:01): > … which leads tocolor_discretesystematicallyFALSE, and the script always expecting a continuous color map even when I send a discrete one
Kevin Rue-Albrecht (16:45:09): > what’s the reason for storing theColorBycommand insetup_cmdsrather thandata_cmds?
Aaron Lun (16:45:26): > Because it’s part of the plot setup rather than the plot data.
Aaron Lun (16:45:41): > But I guess wecould put it into data.
Kevin Rue-Albrecht (16:45:55): > hm.. i’d have to navigate the code again to understand why that is
Aaron Lun (16:45:59): > It sort of is data. But not really.
Aaron Lun (16:46:29): > Hm.
Aaron Lun (16:46:38): > Well, maybe it’s the lesser of two evils.
Kevin Rue-Albrecht (16:46:43): > well, i’m not sure what you consider data then. My immediate intuition is that any column inplot.datais data
Aaron Lun (16:46:44): > We’ll just pretend that it’s data.
Aaron Lun (16:47:01): > Well, the idea was to distinguish from X, Y and (possibly ColorBy)
Aaron Lun (16:47:10): > from jitteredX, jitteredY and other weird crap
Aaron Lun (16:47:19): > that gets plonked in to make the plot.
Kevin Rue-Albrecht (16:47:20): > i see what you mean
Aaron Lun (16:47:34): > And also to distinguish from brushBy, which causes headaches of its own due to circularity.
Aaron Lun (16:48:46): > Okay. I would suggest changing all color fromsetup_cmdstodata_cmds.
Kevin Rue-Albrecht (16:48:46): > I get where you’re coming from. However, thecoloring <- eval_env$plot.data$ColorByreally needsColorByto be already in place. Therefore, it’ll have to be data in the curren setup
Aaron Lun (16:48:58): > This is the lesser of the two evils.
Aaron Lun (16:49:10): > Given that brushBy is already stored.
Aaron Lun (16:49:22): > So X, Y, colorBy and brushBy represents the “data”
Aaron Lun (16:49:37): > and the jittered X/Y columns are everything else in the set up.
Kevin Rue-Albrecht (16:49:51): > Exactly. I don’t mind if you do another refactoring from hell later, but after spending hours on end today tracking that ghost down, I’m not ashamed from taking the shortest path here
Aaron Lun (16:50:05): > This also means that we no longer need explicitsetup_cmdsargument in.create_plot().
Aaron Lun (16:50:35): > Well, the reorganization was important to avoid needing to create loops in the code tracker.
Kevin Rue-Albrecht (16:50:39): > yep, I spotted that you seem to have createdsetup_cmdsspecifically forColorBy
Kevin Rue-Albrecht (16:56:31): > when i retire, screw puzzles, i’m gonna assemble the dependency tree of this app
Aaron Lun (16:58:34): > Well, it’s a lot simpler now, because everything is fully contained withiniSEE.R. Notinputinplotting.Ras there were before.
Aaron Lun (16:58:59) (in thread): > Yes, the scatter calculations do need to be done on the subset.
Aaron Lun (16:59:31) (in thread): > Otherwise we get densities in the scatter that don’t really make sense.
Kevin Rue-Albrecht (16:59:56): > I do appreciate how you’ve organised the infrastructure.
Aaron Lun (16:59:56) (in thread): > It is also important to subset it so that brushing by restriction makes sense on its dependent plots.
Kevin Rue-Albrecht (17:01:49): > Alright. Nearly there. I just need to deal with the discrete colormaps, differentiating the ‘static’ ones that give the same vector of (named) colors, from the ‘dynamic’ ones that deliver N (unnamed) colors upon request.
Aaron Lun (17:02:02) (in thread): > If you like, you can put anifclause in.create_plotafter the second evaluation ofeval_env$plot.data, asking whether there are non-zero rows. If so, you can remove the commands insetup_cmdsandplot_cmdsthat you added, to avoid cluttering the code tracker with unused code.
Charlotte Soneson (17:02:34) (in thread): > Yeah, I’ll do that
Aaron Lun (17:02:35): > Just have a function that doesn’t care about its arguments.
Kevin Rue-Albrecht (17:02:37): > The difference between the user controlling which color is assigned to which level, and a colormap supplying N colors arbitrarily assigned to the N levels of a factor.
Aaron Lun (17:02:57) (in thread): > Add some pretty clear comments, though.
Charlotte Soneson (17:03:03) (in thread): > Yep
Kevin Rue-Albrecht (17:03:11): > I have this function(s) already, actually.
Aaron Lun (17:04:11): > Well, the user-controlled levels are only going to be useful in the top-levelassays, right.
Kevin Rue-Albrecht (17:04:16): > It’s just supplying the N, that’s slowing me down. Needs a bit of evaluation to find it
Kevin Rue-Albrecht (17:04:23): > yep
Aaron Lun (17:04:47): > Yes, you will need to extract this information at thecolor_discretestage.
Kevin Rue-Albrecht (17:05:02): > it’s when you get down to the ‘shared’ colormaps that you need to dynamically compute the number of colours that you need the colormap to return
Aaron Lun (17:05:10): > I’m okay with changing that argument to just report the number of levels.
Aaron Lun (17:05:19): > I mean, we’ll have this information in.create_plot.
Aaron Lun (17:05:49): > So just change it ton_colors, set toInfif it was not groupable, and the number of levels otherwise.
Aaron Lun (17:06:03): > I think there was an old.nlevelsfunction somewhere…
Aaron Lun (17:06:15): > Maybe it got refactored away.
Kevin Rue-Albrecht (17:06:37): > hehehe, I sense in the “…” that you know that I wrote that function myself:wink:
Kevin Rue-Albrecht (17:06:43): > no no, it’s still there
Aaron Lun (17:07:01): > And then change the function-generating function to take the number of colors rather thanis_discrete.
Kevin Rue-Albrecht (17:09:55): > thanks for the advice: at the moment I was working directly in.scatter_plot,where you were applying thecolor_FUNfunction, but indeed, I could see in my code that it was going to be redundantly coy pasted in the other plotting functions
Federico Marini (17:50:57): > merge intomastercomplete for the graphs including also tables, plus hotfix on the crash when opening up new table (thx to Aaron)
2018-01-28
Charlotte Soneson (02:40:20) (in thread): > I added the check in.create_plotand removed the commands ifeval_env$plot.datahas zero rows. It is perhaps not optimal since the creation ofplot.data.allis done in thebrush_cmds(has to be done before the subsetting), soeval_envwill have aplot.data.allin any case. Thus, if we at some point useplot.data.allto affect any other variable up until thebrushcommands, the code that we return will no longer run. What’s your opinion, should we leave it there? Or, add anifto thebrush$fullcommand, so thatplot.data.allwill only be created if no points are brushed? The only completely “clean” way I see, where it will only be evaluated if needed, is to split thebrushcommands in two: brushing and subsetting.
Kevin Rue-Albrecht (04:37:22): > Merge into master completed last night for support of custom maps.
Kevin Rue-Albrecht (04:38:06): > As ECM do not need to contain the same number of maps as there are assays, they must be named
Kevin Rue-Albrecht (04:49:06): > I mean for use within the app
Kevin Rue-Albrecht (04:49:26): > outside, users can access them by index as long as they know what they’re doing
Kevin Rue-Albrecht (04:51:58): > within the app, as the assay choice is stored as the index (not the name), my only choice is to fetch the name of the i-th assay, and look for that name in theExperimentColorMap
Kevin Rue-Albrecht (04:54:18): > the only way to access both theassayand thecolormapby index would be ifcolormapwas stored in a slot of theSCE, that would allow to keep the two coordinated (e.g. fill the ECM withNULLcolormaps where undefined, to keep the number of assays/colormaps the same)
Kevin Rue-Albrecht (05:19:38): > Btw, i haven’t fully documented the features yet. That’s as far as I got last night. and i have also on my to-do list: validity method for the ECM class, check against the SE as soon as the app launched. I’ll look into this as soon as possible, but can’t guarantee a time due to local duties
Aaron Lun (07:15:55) (in thread): > We can catch this by simply destroyingplot.data.allin.create_plotin the sameifclause. This will ensure that if we ever did add some code depending onplot.data.allwhen it doesn’t exist, it would immediately cause errors, thus allowing us to easily debug it.
Aaron Lun (07:17:00) (in thread): > This seems like the best approach. We can do this automatically withiniSEE(), so the user only needs to fill by name.
Aaron Lun (07:18:24): > Nice work. Your refactoring ofplotting.Rwas a bit odd, though - it would have been easier to redefine the function in.process_colorby_choice()to handle all the different types of color inputs.
Aaron Lun (07:18:49): > This would avoid the need to double-check the color inputs in.create_plot().
Kevin Rue-Albrecht (07:19:46): > Yeah - honestly, I wasn’t thinking very clearly anymore at the end. Just wanted to push in something that worked, and leave the clean refactoring for another day.
Aaron Lun (07:19:52): > I’ll handle it.
Kevin Rue-Albrecht (07:20:09): > thanks.
Kevin Rue-Albrecht (07:22:12) (in thread): > I don’t want to rush into things, but I keep wondering whether the core Bioc team would approve aColoredExperimentthat has an extra slot to store anExperimentColorMap
Kevin Rue-Albrecht (07:23:12) (in thread): > At least we’re offering a proof of concept iniSEE, and they can then decide what’s best
Aaron Lun (07:32:35) (in thread): > I doubt they would… we’d have to do it ourselves.
Aaron Lun (07:47:37) (in thread): > But if we do havegeom_blank, maybe that might allow us to get rid of thescale_x_discrete, etc.?
Aaron Lun (07:48:01) (in thread): > coord_cartesianand friends would still need to be there, but only for zooming.
Charlotte Soneson (07:48:07) (in thread): > I guess destroyingplot.data.allwill work as long as the new code is not in thebrush_cmds, right? But that might be good enough:slightly_smiling_face:
Charlotte Soneson (07:48:30) (in thread): > And yes, I would say that we don’t need thescalethings.
Kevin Rue-Albrecht (07:54:15): > I’ll be out for the rest of today. If anyone could merge my PR#100when it’s finished updating, that’d save me updating it again later. Cheers!
Aaron Lun (07:54:29): > Does that fix the out-of-bounds error withassayColorMap?
Aaron Lun (07:54:35): > Currently finished refactoring.
Kevin Rue-Albrecht (07:54:46): > ow.. can’t remember testing that
Aaron Lun (07:55:12): > ecmin?iSEEdoesn’t have an entry forlogcounts.
Aaron Lun (07:55:37): > Giving me:
assayColorMap (ecm,6 )in x@ assays[[i]] : subscript out of bounds
Kevin Rue-Albrecht (07:55:37): > otherwise, just apply the logic implemented inSummarizedExperiment-class.R: > > setMethod("assay", c("SummarizedExperiment", "numeric"), > function(x, i, ...) > { > tryCatch({ > assays(x, ...)[[i]] > }, error=function(err) { > stop("'assay(<", class(x), ">, i=\"numeric\", ...)' ", > "invalid subscript 'i'\n", conditionMessage(err)) > }) > }) >
Kevin Rue-Albrecht (07:55:56): > except that instead ofstop, returnNULL
Aaron Lun (07:56:13): > Shouldn’t it go up the hierarchy of defaults?
Kevin Rue-Albrecht (07:56:17) (in thread): > interactively, i’d expect an error
Aaron Lun (07:57:05): > Otherwise a user would have to specify every color for every assay/coldata; that’s quite a burden.
Kevin Rue-Albrecht (07:57:06) (in thread): > but then, i think the chunk of code that I’ve just sent fromSummarizedExperiment-class.Rshould help the accessor go down in the hierarchy
Kevin Rue-Albrecht (07:57:54) (in thread): > thetryCatchreturningNULLwill avoid the out of bound error
Aaron Lun (07:58:25) (in thread): > I’d expect the default color map.
Kevin Rue-Albrecht (07:58:36) (in thread): > and in my code, theif(is.null)will understand that as a signal to go down the hierarchy
Kevin Rue-Albrecht (07:58:41) (in thread): > got it?
Kevin Rue-Albrecht (07:58:57) (in thread): > i can’t implemented it now as we’re dressing up to leave now
Aaron Lun (07:59:07) (in thread): > I’ll have a look at it.
Kevin Rue-Albrecht (07:59:10) (in thread): > cool
Kevin Rue-Albrecht (07:59:15) (in thread): > catch up later
Kevin Rue-Albrecht (08:55:27) (in thread): > Btw: I hope you’ve noticed that an ellipsis passes arguments down to the individual colormap functions, including ‘discrete’. Which mutes the need for a parallel hierarchy of discrete and continuous colormaps.
Kevin Rue-Albrecht (08:57:10) (in thread): > I got the inspiration of how your refactoring of ‘plotting.R’ passes arguments picked up as needed
Aaron Lun (09:11:34) (in thread): > Shouldn’t you have a discrete/continuous choice in.colDataAllColorMapand.assayAllColorMap?
Federico Marini (09:12:20): > bummer, I can not anchor introjs tour steps to dropdown menus and their contents
Federico Marini (09:12:38): > there is no possibility to pass theidparameter
Aaron Lun (09:14:29): > Don’t worry about it; those things aren’t necessary for the interface itself anyway.
Aaron Lun (09:14:45): > They’re more like add-ons, really.
Aaron Lun (09:15:09): > And we need to keep the LHS panel light, otherwise there will be much scrolling.
Federico Marini (09:15:24): > k
Federico Marini (09:15:44): > maybe it is somehow possible to override the dropdown menu function
Federico Marini (09:15:48): > if we define our own
Aaron Lun (09:16:07): > Well, we’d have to define our own that is able to interact with shiny’sinput.
Federico Marini (09:16:32): > which just mimics the functionality as in shinydashboard, and then “just add that part”
Federico Marini (09:16:54): > ok, low priority:slightly_smiling_face:
Aaron Lun (09:17:16): > The problem is that this would need to be done at the JS level.
Aaron Lun (09:17:27): > which was a real pain for the collapsible boxes.
Federico Marini (09:17:32): > likely:disappointed:
Aaron Lun (09:17:36): > I still don’t really know how I got that to work.
Federico Marini (09:17:46): > serendipity?:smile:
Aaron Lun (09:23:37) (in thread): > Frustratingly… we still do need thescalethings, otherwise empty levels are just dropped in the violin plots. Moreover, we stillcoord_cartesian, because otherwise the plot will expand to put more space between the edge of the plot and the brushing box we draw withgeom_rect. FFS,ggplot is too smart for its own good.
Charlotte Soneson (09:26:19) (in thread): > Mhm… But it should only drop levels that are empty in the entire original data set, no?
Charlotte Soneson (09:26:31) (in thread): > Or that only correspond to NA values of the things we plot.
Aaron Lun (09:26:41) (in thread): > Apparently not.
Charlotte Soneson (09:26:45) (in thread): > Interesting
Aaron Lun (09:35:02) (in thread): > Also, we need to think of a better x-axis label than “0” when there are no groupings on the violin plots.
Aaron Lun (09:35:23) (in thread): > What about “all”?
Kevin Rue-Albrecht (09:37:50) (in thread): > Haven’t tested it yet, but maybe it’ll just need to be explicitly coded as in: > ’.globalColorMap <- function(x, …, discrete = FALSE){‘
Kevin Rue-Albrecht (09:38:06) (in thread): > (No backtick on the phone)
Charlotte Soneson (09:48:43) (in thread): > Or nothing?
Aaron Lun (09:52:10) (in thread): > Think “All cells” might be better.
Charlotte Soneson (09:53:58) (in thread): > I’m thinking two things: 1. what if the data is not single cell (so the columns are something else), 2. if the data are brushed, it is not actually all cells…
Aaron Lun (09:55:09) (in thread): > Hm.
Aaron Lun (09:55:20) (in thread): > Can we just turn off the label - give it an empty string?
Charlotte Soneson (09:55:53) (in thread): > The axis label is already an empty string, but if we can somehow getaxis.text.x = element_blank()into the theme, that will turn of the tick mark label
Aaron Lun (09:56:22) (in thread): > Was it an empty string? I think I was seeing zeroes.
Aaron Lun (09:56:32) (in thread): > The tick mark is not so bad.
Charlotte Soneson (09:56:45) (in thread): > The tick mark will be kept, just the label of the tick will be gone
Charlotte Soneson (09:57:09) (in thread): > The 0 is because all the values in theXcolumns are 0
Aaron Lun (09:57:26) (in thread): > Okay, we can change theXto an empty string.
Charlotte Soneson (09:59:11) (in thread): > Yes, settingplot.data$X <- ""seems to work as well.
Aaron Lun (10:01:27) (in thread): > Could you do that torefactor?
Charlotte Soneson (10:02:46) (in thread): > Yes
Charlotte Soneson (10:06:36) (in thread): > Pushed
Charlotte Soneson (10:07:10) (in thread): > Just changedinteger(ncol(se))tocharacter(ncol(se))
Aaron Lun (10:07:45) (in thread): > Thx
Aaron Lun (11:06:08) (in thread): > Looking at your class definition, you could just change it to: > > setClass("ExperimentColorMap", > contains="Vector", > representation( > # each slot has a list of closures > assays="list", > colData="list", > rowData="list", > all_discrete="list", > all_continuous="list", > global_discrete="function_or_NULL" > global_continuous="function_or_NULL" > ), > prototype( > all_discrete=list(assays=NULL, colData=NULL, rowData=NULL), > all_continuous=list(assays=NULL, colData=NULL, rowData=NULL), > global_discrete=NULL, > global_continuous=NULL > ), > validity = .valid.Colormap > ) >
Aaron Lun (11:06:36): > which is still pretty clean.
Aaron Lun (11:07:30): > And perhaps theglobalcan be set to the hard-coded defaults in the ECM constructor, to avoid the need to define (and document!) the union class.
Aaron Lun (11:56:14): > iSEE that people enjoy sticking all their sentences onto a single line in the vignette.
Charlotte Soneson (11:57:27): > Ehm, sorry about that. I can split it by sentence.
Aaron Lun (11:57:55): > It’s okay, I’m going through and doing that already.
Aaron Lun (13:14:29): > I’ve reviewed and edited half of it up to the use cases. but I’m pretty pooped now. My major suggestion with the use cases would be to break up the instructions into explicit steps, i.e., 1. Do something, 2. Do some more stuff, 3. Etc.
Aaron Lun (13:16:03): > I also commented out the section discussing theiSEEarguments, as this seemed to be regurgitation of the stuff in?iSEE. While some advice on how to set it up on a remote/Shiny server would be useful, I would put this at the end of the vignette.
Aaron Lun (13:57:45): > Apparently browsers don’t allow opening of local files:https://stackoverflow.com/questions/5246292/open-local-folder-from-link - Attachment (stackoverflow.com): Open local folder from link > How can I open a local folder view by clicking on any link? I tried many options like Open folder or Open folder…
Aaron Lun (13:57:53): > So we’ll just have the web vignette then.
Aaron Lun (13:58:24): > Well, it’s not impossible, just not viaon.click.
Aaron Lun (14:05:24): > Now enabled viabrowseURL. Note that this requires your installation to actually have the vignette; it seemsR CMD INSTALLwon’t build the vignette, you’ll have to BUILD then INSTALL if you’re working on the command line. Don’t know what Rstudio does, maybe it does it both automatically.
Federico Marini (14:11:23): > nice workaround
Federico Marini (14:12:00): > are you about to merge soon into master or is more coming?
Aaron Lun (14:12:10): > Nope, will merge once CI clear.
Aaron Lun (14:12:11): > s
Federico Marini (14:12:50): > okiedokie. I’ll probably stick then to this latest version
Aaron Lun (14:27:25): > merged. Check everything is fine for your presentation tomorrow.
Aaron Lun (14:27:29): > I’m going home now.
Federico Marini (14:36:04): > will do
Federico Marini (14:36:19): > but I am quite sure that’ll be fine
Federico Marini (16:10:50): > I saw this back then once
Federico Marini (16:10:58): > but for our effort it is well deserved
Federico Marini (16:10:59): > http://starlogs.net/#csoneson/iSEE
Kevin Rue-Albrecht (17:04:08): > devtools::install(build_vignettes = TRUE) - Attachment: Attachment > Now enabled via browseURL. Note that this requires your installation to actually have the vignette; it seems R CMD INSTALL won’t build the vignette, you’ll have to BUILD then INSTALL if you’re working on the command line. Don’t know what Rstudio does, maybe it does it both automatically.
Kevin Rue-Albrecht (17:11:22): > RStudio doesn’t build the vignette with the build and reload button. But autocomplete makes the above command fairly quick to type and it does the job
Kevin Rue-Albrecht (18:29:03): > Hi all. Just to let you know that i’m aware of the the out-of-bound error about the assay color maps.
2018-01-29
Kevin Rue-Albrecht (02:55:08): > Hi@Federico MariniA far as I can see, I’ve solved all things about theExperimentColorMapand just merged the result tomaster
Kevin Rue-Albrecht (02:56:08): > The version from last night still had issues with it if theECMdidn’t have as many assay colormaps as theSCEhas assays
Kevin Rue-Albrecht (03:30:51): > I suppose you were going to avoid invalid choices in your presentation, or use an older commit. But I think this one may simplify the situation.
Kevin Rue-Albrecht (03:31:23): > Do keep track of an older, stable, that you can revert back to, just in case I missed something
Federico Marini (03:33:58): > I have quite a short time so I would probably focus more on the core functionality, and grab the code charlotte wrote in the vignette to show you can use your fav colors
Kevin Rue-Albrecht (03:42:12): > Ok
Kevin Rue-Albrecht (03:43:49): > I just thought that core functionalities include the redDim coloured by gene expression, and that was breaking easily if the you select assay #3 when the ECM only has 2 defined, for instance
Kevin Rue-Albrecht (03:48:14): > Maybe it’s clearer if you stick with the old vignette, but it might be misleading. Correct me if I’m wrong: the vignette as you still see it returns the actual vector of colours
Kevin Rue-Albrecht (03:55:40): > While the recent updates since yesterday return a function(n) where n is the number of colours requested
Kevin Rue-Albrecht (03:56:27): > (Constant colormaps take the n argument but don’t care about it)
Aaron Lun (04:15:10): > Good work
Aaron Lun (04:19:31): > We’ve hit 500 commits onmaster.
Kevin Rue-Albrecht (04:21:36): > Let me add Good work everyone as well.
Kevin Rue-Albrecht (04:22:29): > I’ve coded a bit late last night, so I appreciate any feedback, and issue reporting, if I missed anything
Charlotte Soneson (04:23:09): > Cool:slightly_smiling_face:It’s looking really nice
Charlotte Soneson (04:24:25): > I was clicking through the app just now and noticed a couple of unexpected behaviours in various parts. I’ll open some issues in the GitHub repo.
Kevin Rue-Albrecht (04:24:42): > awesome, please do
Aaron Lun (04:34:33): > Also, I’m getting two color bars for coldata in the violin plots.
Aaron Lun (04:34:49): > One is the default continuous, the other is viridis.
Kevin Rue-Albrecht (04:34:54): > screenshot?
Aaron Lun (04:35:44): > @Aaron Lunuploaded a file:image.png - File (PNG): image.png
Kevin Rue-Albrecht (04:36:27): > ok, indeed, that sounded like the effect of having two layers linked to different color aesthetics
Kevin Rue-Albrecht (04:37:43): > let me boot up the add to that state and look at the script, i’m pretty sure someaescalls were not in the optimal place yet
Kevin Rue-Albrecht (04:38:51): > you’ve probably figured by now that anaes()call in theggplot() call gets applied to all layers (assuming one left the defaultinherit.aes=TRUE)
Kevin Rue-Albrecht (04:39:15): > and that each layer can override the globalaesor at least add stuff to it
Kevin Rue-Albrecht (04:39:51): > so in this case, it’s a matter of two layers defining different color aesthetics
Aaron Lun (04:40:06): > Um.
Aaron Lun (04:40:08): > If you say so.
Aaron Lun (04:40:18): > I don’t really have a handle on how this ggplot business works.
Kevin Rue-Albrecht (04:42:04): > No worries. I’ll get to it this evening.
Federico Marini (04:43:22): > Checking on the TODO list we set ourselves last week, I guess we pretty much covered it all. The vignette needs some extra pulp in the use case for the CyTOF
Federico Marini (04:43:41): > but for the rest we are at the final polishing
Federico Marini (04:43:49): > or did I oversee something major?
Charlotte Soneson (04:44:04): > I’m on to the CyTOF part, just need to get some data first:slightly_smiling_face:
Aaron Lun (05:42:14): > Bugfix branch can be merged as soon as it passes CI.
Aaron Lun (05:54:41): > I did some modifications to reduce the amount of unnecessary replotting when the brush options update. I’ve checked it out and I’m pretty sure that it works, but everyone should give it some stress tests.
Kevin Rue-Albrecht (07:19:17): > @Charlotte Sonesonwhat is involved in the “CyTOF” part? Vignette and data set? Or are you also looking into a wrapper aroundiSEE()that would preset a good default set of panels and brushing connections?
Charlotte Soneson (07:20:25): > I haven’t started with any actual wrapper. My idea was to define a set of defaults that made sense and start the app from those settings.
Kevin Rue-Albrecht (07:20:55): > ok, i think we’re talking about the same thing, actually
Kevin Rue-Albrecht (07:23:45): > Indeed, we need to define a set ofredDimArgs = ..., colDataArgs = ..., geneExprArgs = ...that’s adequate for startup. > By wrapper, that I mean is that I was thinking about aiSEE_CyTOF(...)function that sets up those defaults, and then returnsiSEE()using those defaults
Kevin Rue-Albrecht (07:28:01): > Nice thing about a wrapper is that one can also preset brushing links across the various plot types
Charlotte Soneson (07:37:04): > Is there a way we could easily export the “current” set of defaults from the app? I mean, if you have a setup that you like and you would like to start the app like that next time (not just make the plots).
Aaron Lun (07:37:22): > This is possible, but the app will stop once you do so.
Aaron Lun (07:37:42): > This is done by returning something from therunAppcall.
Charlotte Soneson (07:38:21): > Ok. It is not accessible in a way that it could be printed out like the plot commands then?
Aaron Lun (07:38:29): > No.
Kevin Rue-Albrecht (07:38:33): > they’re tables, bit tougher
Aaron Lun (07:38:34): > Too many bits and pieces.
Aaron Lun (07:38:52): > Some of the columns are nested lists
Aaron Lun (07:38:59): > and other weird crap e.g. with the brushes.
Kevin Rue-Albrecht (07:39:50): > i’ve been testing the brushing, not exhaustively, but I absolutely love chained brushes. Gotta try ‘branched’ chains soon
Kevin Rue-Albrecht (07:40:03): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-01-29 12.38.17.png - File (PNG): Screenshot 2018-01-29 12.38.17.png
Kevin Rue-Albrecht (07:49:01): > and the panel chart is absolutely amazing looking
Kevin Rue-Albrecht (07:55:57): > I love the branched stuff too. > I guess we’ll have to crowdsource the ‘stress test’ to real users soon. Too many combos to test:laughing:
Aaron Lun (07:59:25): > I’ll put together a draft of an email to BioC-devel, once I finish cleaning upscater ’s vignette.
Aaron Lun (07:59:31): > Ugh.
Aaron Lun (09:09:08): > See draft here:https://docs.google.com/document/d/19OlHIlBS-e3RGjaEUEFptKRY1u3sdvyw0agH8rZvieA/edit?usp=sharing
Aaron Lun (09:09:25): > You can comment if you like.
Federico Marini (09:15:20): > added a couple of notes
Federico Marini (09:15:38): > it was already good anyway:slightly_smiling_face:
Charlotte Soneson (09:18:23): > Looks good to me too.
Charlotte Soneson (09:18:55): > Only thing I was wondering was whether we want a broader focus (’omics instead of single-cell ’omics) in the first paragraph
Federico Marini (09:19:21): > yep
Federico Marini (09:19:34): > we should also decide what the official acronym will be for
Federico Marini (09:19:55): > SummarizedExperiment/SCE would catch them all
Aaron Lun (09:55:59): > Just put it as SummarizedExperiment, I think.
Kevin Rue-Albrecht (09:59:00): > yep I kept struggling with theiSCEEacronym that anSCEshould make
Kevin Rue-Albrecht (10:06:39): > Great draft email. I’ve offered a handful of superfluous edits.
Aaron Lun (10:29:55): > Okay, everyone have a look through before we go through with it.
Aaron Lun (10:30:09): > Also check that the latest release from github is working!
Aaron Lun (10:30:20): > I wonder whether we should label it.
Aaron Lun (10:31:48): > Have a look at the “release” section on GItub.
Kevin Rue-Albrecht (10:33:43): > yep - I think it’s worth branding this release either alpha or beta
Kevin Rue-Albrecht (10:34:17): > A couple more comments on the email (my PhD supervisor taught me well how to be pedantic about details)
Aaron Lun (10:36:27): > You know what, I don’t think the feature/gene thing matters.
Kevin Rue-Albrecht (10:36:33): > I think I’m good with the email now. Considering it’s Bioc-devel, I’m sure “feature selection” will be clear to most people
Aaron Lun (10:37:00): > Because we have hardcoded “gene statistics table” anyway.
Aaron Lun (10:37:17): > And being general at this point would just confuse people.
Kevin Rue-Albrecht (10:37:35): > hm you’re right about that
Kevin Rue-Albrecht (10:38:28): > maybe just describing it with a use case would be clear than trying to cover it with an all-encompassing description
Aaron Lun (10:38:51): > For an email? Don’t worry about it.
Aaron Lun (10:45:32): > I won’t send this out until the double color bar gets fixed, I’ve just seen a bug myself.
Charlotte Soneson (10:57:03): > The double color bars seem to be one for color (viridis), one for fill (default).@Kevin Rue-Albrecht, would addingscale_fill_*in addition (and equal) toscale_color_*destroy anything else?
Kevin Rue-Albrecht (10:58:26): > Arf yes ok. Good to remind me. Can’t do it at work today, too much to do. If no one fixes it before, I’ll look at it tonight. Best I can do
Kevin Rue-Albrecht (10:59:42) (in thread): > not sure what you mean by “destroy”, but my intuition would be to solve this by adding afillscale equal to thecolourscale indeed
Kevin Rue-Albrecht (10:59:51) (in thread): > ah i get what you meant now
Kevin Rue-Albrecht (10:59:59) (in thread): > dunno if it would destroy anything else
Charlotte Soneson (11:00:54) (in thread): > I didn’t look through the rest of the code yet to see if anything else would be affected with regards to the colormaps, or if there was a reason not to setscale_fill_*
Charlotte Soneson (11:01:13) (in thread): > Was just wondering if anything struck you immediately
Kevin Rue-Albrecht (11:01:14) (in thread): > I can’t say unless I look at the code. I could only guess that I’m not expecting to break anything, because colour and fillshould be coordinated
Kevin Rue-Albrecht (11:01:51) (in thread): > as far as I’m aware we haven’t implemented any situation where fill and colour can be controlled separately
Charlotte Soneson (11:12:55): > Just addingscale_fill_*seems to work. I’ll push this to thecharlotte_fixesbranch and merge it
Kevin Rue-Albrecht (11:13:05): > cool, thanks for taking care of it
Federico Marini (11:35:35): > Hi, quickly noting down the feedback after todays presentation
Kevin Rue-Albrecht (11:36:00): > in the channel?
Kevin Rue-Albrecht (11:36:20): > you may want to use the “+” button on the left to write a “Post”
Federico Marini (11:36:35): > relevant for next development, I got not that much - apart from the question from my mentor nr. 2 “who will take care of the maintainance and or implement new features”
Kevin Rue-Albrecht (11:36:37): > that’ll make it easier to pin it and find it again
Federico Marini (11:36:39): > uh
Federico Marini (11:36:52): > too bad it is not that much much to be worth a post:stuck_out_tongue:
Kevin Rue-Albrecht (11:37:04): > ahhh
Kevin Rue-Albrecht (11:37:12): > in that case, go Davis-style:wink:
Federico Marini (11:37:15): > more like “how do i install it” - they have some data, and like to try it out.
Aaron Lun (11:37:28): > Just ask them “Do you even R?”
Federico Marini (11:37:44): > some nice jawdropping after they notice that the plots are exactly reproduced
Kevin Rue-Albrecht (11:37:49): > hahaha
Federico Marini (11:38:03): > (which made me innerlyand
Kevin Rue-Albrecht (11:38:11): > i’m still amazed myself by the feature to be honest
Federico Marini (11:38:57): > I told them to visit the repo and open an issue if they come across some interesting feature request
Federico Marini (11:39:25): > and then the procedure would be something like reaching a critical mass of users that want it, or having us already wanting it
Federico Marini (11:40:37): > apart from that, mostly people presenting project ideas for a coordinated grant proposal, but at its nascent state still
Kevin Rue-Albrecht (11:40:58): > Fair enough. Maybe we’ll have to think about sub-labels for issues, to deal with them efficiently.
Federico Marini (11:41:32): > Let them come first:slightly_smiling_face:I saw a lot of time people telling you “yes I’ll do this and that” and then not doing this and that
Aaron Lun (11:41:43): > PR or nothing.
Federico Marini (11:42:23): > hopefully suggestions will not come only on the[whatever stupid error generated by me not knowing R] basis
Federico Marini (11:43:07): > but anyway, I felt good in giving the overview. Works, scaled well on the TCGA dataset to have quite immediate re-plot
Federico Marini (11:43:48): > one interesting thing I heard of
Federico Marini (11:44:06): > a professor in CS here is developing some ‘mapping’-ish algorithm
Federico Marini (11:44:34): > that seems to work ~10 times faster than kallisto, and with better precision-recall than rapmap2
Federico Marini (11:44:42): > no preprint
Federico Marini (11:44:44): > no repo
Federico Marini (11:44:47): > yet, at least
Federico Marini (11:45:09): > it is probably in need of further benchmarking
Aaron Lun (12:01:16): > For some inexplicable reason, changing the color from or to “None” destroys the brush.
Aaron Lun (12:01:19): > in violin plots.
Aaron Lun (12:02:08): > And only in violin plots, which is weird.
Aaron Lun (12:11:11): > For reasons I don’t understand, adding a legend to the violin plot seems to destroy the brush.
Aaron Lun (12:15:54): > Same with removing the legend.
Kevin Rue-Albrecht (12:16:19): > removing what legend?
Aaron Lun (12:16:27): > for the color on the violin plot.
Kevin Rue-Albrecht (12:16:53): > ah i get you now
Aaron Lun (12:16:54): > Try opening it upon the allen dataset; brushing somewhere on the coldata plot; and then changing thec olor by to something else.
Aaron Lun (12:17:29): > The problem is at theinputlevel; something with the creation of the violin plot is clearing the brush.
Aaron Lun (12:19:05): > The annoying this is that it does actually get plotted correctly - and then it gets destroyed.
Kevin Rue-Albrecht (12:21:13): > makes me wonder if we should add adebug=FALSE|1|2|3|...option to the app, that wouldmessage()out different levels of info at runtime
Kevin Rue-Albrecht (12:22:19): > things like names of functions being called, orclearing brush for plot %smight come in handy
Kevin Rue-Albrecht (12:23:22): > The connection are still (relatively) clear in our mind right now, but i’m thinking in 6 month time, and when we add new features
Aaron Lun (12:24:19): > Well, that’s fine and all, but that’s not the problem right now.
Aaron Lun (12:24:38): > How is this possibles?
Kevin Rue-Albrecht (12:24:54): > not the immediate problem, but my point was that it might have helped you track down the source of the bug
Aaron Lun (12:33:37): > This is fully attributable to the violin plots - it doesn’t happen for anything else.
Charlotte Soneson (12:48:07): > I can reproduce this, but I am wondering if it is actually plotting the correct thing first. To me it looks like only the outer rectangle is retained, not the actual brushing (the filled thing), and the outer rectangle is lagging a bit also if you drag the brush around.
Aaron Lun (12:48:43): > The lag is to be expected, it’s replotting the outer box every time, remember.
Charlotte Soneson (12:49:34): > Yes, what I meant was that I am not sure it is actually plotting the brushing correctly when you change the coloring, to me it looks like it is just the outer box remaining
Charlotte Soneson (12:49:56): > And then, since the brush is gone, also the outer box goes
Aaron Lun (12:50:15): > The behaviour for the reduced dimension plots is the correct one; both the inner and outer boxes stay.
Aaron Lun (12:51:29): > Finally, I got it to stay; something to do with the aes in the initialggplotcall.
Aaron Lun (12:51:57): > Now let me restore the commands I deleted and see if it helps.
Aaron Lun (13:00:45): > Well, I don’t know how I fixed it, but I did.
Aaron Lun (13:01:05): > Seems like the initialggplot(plot.data, aes)was causing the problem.
Aaron Lun (13:01:22): > Why?shrug
Kevin Rue-Albrecht (13:01:54): > I won’t try to guess that one
Aaron Lun (13:04:45): > Here’s a wild guess; when we ran the original command,ggplot actually makes two plots, which have completely different coordinate systems.shiny tries to apply the brush on the first plot, but because the coordinate system is so different, the brush just gets wiped out (as it doesn’t lie within the x/y-axis limits). Upon creation of the final plot, the brush no longer exists.
Kevin Rue-Albrecht (13:05:06): > ah
Aaron Lun (13:05:11): > Completely wild guess.
Aaron Lun (13:05:26): > Now, by removing the initialaes, we avoid any hypothetical double-plotting.
Aaron Lun (13:05:57): > God.
Kevin Rue-Albrecht (13:07:04): > I like that explanation. Wonder how many other Easter eggs like this we’ll have to chase in future plots
Aaron Lun (13:07:25): > Anyway, do we need to setFillBy?
Aaron Lun (13:15:47): > And is data always inherited down the chain of ggplot commands?
Charlotte Soneson (13:18:16): > From theggplot(), not from onegeomto another
Aaron Lun (13:18:25): > Hm.
Aaron Lun (13:18:32): > Okay, well, I’m surprised that it works then.
Aaron Lun (13:18:51): > Oh, actually, it’s fine.
Aaron Lun (13:19:02): > The data is passed correctly, it’s just the aes that I removed.
Aaron Lun (13:37:56): > Okay, bugfixes in a PR.
Aaron Lun (13:39:53): > I won’t be at my computer tomorrow, so could you guys fiddle with the patched version and see if it breaks?
Charlotte Soneson (13:52:09): > Sure
Kevin Rue-Albrecht (13:54:33): > I was actually asked tonight to write the instructions to install iSEE for a few earlier adopters in Oxford. Will be nice to have extra hands on deck for testing
Aaron Lun (13:54:57): > Charlotte, did we manage to get an app on a shiny server?
Aaron Lun (13:55:04): > As in, an outward-facing server?
Aaron Lun (13:55:09): > Or is yours still internal?
Charlotte Soneson (13:55:51): > Our is still on the internal one. Do we still want one (I thought it was mainly for Federico’s presentation, which would have been too tight)?
Aaron Lun (13:56:25): > Hm. Well, any presentations I do will have to be on my mac, and it creaks at the seams when I ask it to runiSEE.
Aaron Lun (13:56:55): > I mean, during our latest skype, everything stopped and I could only hear your voices.
Kevin Rue-Albrecht (13:57:13): > Is it too big forshinyapps.io free plan?
Charlotte Soneson (13:57:17): > Ok. I wonder how difficult it would be to put it onshinyapps.io
Charlotte Soneson (13:57:19): > Exactly:slightly_smiling_face:
Aaron Lun (13:57:41): > What’s the limitations?
Kevin Rue-Albrecht (13:57:44): > I’m on my way home. Let me try after dinner
Kevin Rue-Albrecht (13:57:53): > App size and number of CPU
Aaron Lun (13:58:20): > Ha, someone could just run our app for >25 hours per month.
Aaron Lun (13:58:25): > and it would shut down.
Aaron Lun (13:58:28): > Lol
Kevin Rue-Albrecht (13:58:34): > Arghh forgot that one
Charlotte Soneson (13:58:47): > Yes…but if we use it for demonstrations we don’t have to give away the address:slightly_smiling_face:
Aaron Lun (13:58:51): > Yes, that’s true.
Kevin Rue-Albrecht (13:58:55): > Well. Exactly
Aaron Lun (13:59:08): > Well, we’d have to change the address every time, because they would see it on the screen, rigth?
Aaron Lun (13:59:16): > :slightly_smiling_face:
Charlotte Soneson (13:59:35): > But they could anyway only explore the Allen data set or whatever we choose to deploy it with…
Kevin Rue-Albrecht (13:59:40): > How about embedding in an iframe for presentations?
Aaron Lun (13:59:58): > Okay… this all seems a bit too much. I’ll see how far I can go on the mac.
Kevin Rue-Albrecht (14:00:51): > Anyway. I don’t think anyone would play much with theshinyapps.io much
Charlotte Soneson (14:01:06): > No, me neither, not if they can’t look at their own data
Aaron Lun (14:01:48): > I’m thinking of the guy who will use play with it one hour per day, just to bring it down.
Aaron Lun (14:01:49): > I would.
Kevin Rue-Albrecht (14:01:52): > Plus there’s a number of concurrent users too I think
Kevin Rue-Albrecht (14:02:13): > BiocTroll package?
Aaron Lun (14:04:48): > I have thought about writing a function that effectively deletes the home directory, and sneaking it into.onLoad.
Aaron Lun (14:04:55): > Many lols to be had.
Kevin Rue-Albrecht (14:08:48): > “You’re home directory has been deprecated”
Kevin Rue-Albrecht (14:11:52): > Just crossed my mind: we’re already four developers/maintainers. Pretty sure we can host ashinyapps.io each to increase the chances having one not trolled empty at any time
Federico Marini (14:15:53) (in thread): > you can get far away enough:slightly_smiling_face:
Federico Marini (14:16:05) (in thread): > but I have to say, I also am using my private one
Federico Marini (14:16:21) (in thread): > we get some crappy other stuff as default for the institute
Federico Marini (14:17:19): > check out the evilR package for that@Aaron Lun
Federico Marini (14:17:27): > plenty of nasty ideas
Federico Marini (14:17:31): > let me check the link
Federico Marini (14:18:02): > https://romain.rbind.io/blog/2017/05/28/turn-r-users-insane-with-evil/ - Attachment (romain.rbind.io): Turn R users insane with evil > Romain François. Consulting Datactive
Charlotte Soneson (14:21:46): > Seems you can set passwords for apps onshinyapps.io :slightly_smiling_face:
Federico Marini (15:49:57): > -> done editing the introjs steps, removed the references to the now gone buttons
Federico Marini (15:50:30): > now merging intomaster
Kevin Rue-Albrecht (16:19:08): > There are a few missing importFrom things btw. I’m adding them in now (e.g.scale_color_manual).
Aaron Lun (17:19:45): > I notice that my commit to fix the violin plots also wiped out the use offill_set. Is settingFillByandfill_setnecessary for violin plots?
Aaron Lun (17:20:16): > I mean, the fill color would only occur in very rare circumstances - when all the points in a given group have the same colour.
Aaron Lun (17:21:25): > I can’t imagine that this would occur enough to warrant using the fill.
Aaron Lun (17:21:42): > But if people disagree, then you’ll have to manually change thefill = FALSEback tofill = fill_set.
Kevin Rue-Albrecht (17:24:48): > hmm.. let me play with app. For the sake of argument i immediately wonder about people who would colour and group thing by the same factor. Just to have a ‘cute’ simultaneous violin and jitter. Is that what you’re thinking about?
Kevin Rue-Albrecht (17:25:59): > ah ok, i see what you mean. at the moment violins are not filled at all. ok
Kevin Rue-Albrecht (17:26:44): > They look good like this. Users can grab the code and fill it themselves I guess. We’ll giv’em pencils and tell them not to colour outside the lines.
Aaron Lun (17:33:43): > Okay. I suggest getting rid of all incidences offill_set, then, to avoid confusion later.
Aaron Lun (17:34:35): > In addition, now that all arguments are inparam_choices, we can refactorplotting.Ragain for greater simplicity.
Aaron Lun (17:35:06): > This will eliminate the need for the wackycolor_FUNbusiness that was previously constraining us.
2018-01-30
Kevin Rue-Albrecht (03:58:30): > I’ll have to update the ECM validation function tonight. I just realised that I’m catching invalid colormaps (not functions) at runtime only
Kevin Rue-Albrecht (04:01:01): > And for the alpha I don’t think we need this, but I’ll probably add some setters to the ECM so that to edit a colormap one doesn’t have to create a new one from scratch or directly fiddle with the slots
Aaron Lun (04:17:44): > Fix your bug and we’ll proceed with the allpha; don’t worry about new features at this point.
Kevin Rue-Albrecht (08:55:02): > Quiet day it seems. Is this the last bug/issue that we’re aware of?
Aaron Lun (08:55:38): > Well, I haven’t been messing with it, can’t do anything on the mac.
Kevin Rue-Albrecht (08:55:53): > If anyone is interested, I’d appreciate feedback on theExperimentColorMapin terms of usability and intuitiveness at the command line .
Kevin Rue-Albrecht (08:56:15): > I’m biased into using it ‘correctly’ as I know what it expects
Kevin Rue-Albrecht (08:56:35): > let me know if more informative error messages are needed for instance
Kevin Rue-Albrecht (08:57:11): > for example, at the moment, I put a simplestopifnotif a colormap function… isn’t a function
Kevin Rue-Albrecht (08:57:52): > although that’s a poor example as I will move this to check systematically all colormaps in the class validity method tonight
Aaron Lun (08:58:30): > I can have a look at it tonight. Push back alpha to tomorrow, then?
Aaron Lun (08:58:39): > Give everyone a chance to break the interface.
Kevin Rue-Albrecht (08:59:05): > Yep. I’m putting together dozens of figures in Illustrator for a report today:sleepy:
Kevin Rue-Albrecht (09:00:14): > But I’m also scribbling my to-do list and roadmap for fixing up the ECM tonight
Federico Marini (09:01:20): > no need to come up with the logo yet, I assume?:slightly_smiling_face:
Kevin Rue-Albrecht (09:01:57): > what happened to the draft logo that you sent? what kind of quality/filetype is it
Kevin Rue-Albrecht (09:02:15): > found it again, png
Federico Marini (09:02:22): > uh that was a combo of screenshots, very primordial
Federico Marini (09:02:49): > so i guess png as a final screenshot
Federico Marini (09:02:55): > but I have the “source” in ppt
Federico Marini (09:02:56): > :smile:
Kevin Rue-Albrecht (09:03:04): > hm.. i liked the left lense already, really nice tSNE layout of the TCGA
Kevin Rue-Albrecht (09:03:31): > ideally, i’d rather have a proof of the logo in vector graphics than PPT/PNG
Kevin Rue-Albrecht (09:05:24): > The right lens of the glasses needs updating, we’ve changed the appearance of the plot since then. Would you like to play with the app, find a good-looking plot, and then fiddle with the output script to generate some high-quality PDFs?
Kevin Rue-Albrecht (09:06:42): > I remember Aaron knows Inkscape well. I’ve got Illustrator over here. We can get the different items together in a high-res logo pretty quick I think
Federico Marini (09:27:23): > I’ll play around with some plots later tonight
Federico Marini (09:27:44): > and then have the code saved for reproducibility:slightly_smiling_face:
Kevin Rue-Albrecht (09:29:45): > This app is a dream seriously. So glad the group of us got on this together. Wouldn’t have been the same without any of us.
Aaron Lun (09:29:55): > Agreed.
Aaron Lun (09:30:16): > Also, as I said before, I think the logo should go into the “About iSEE” modal. Plenty of space there so you can make it as pretty as you want.
Aaron Lun (09:30:44): > I also think we should allow users to modify the title, so that people will know what they’re looking at for hosted apps on servers.
Kevin Rue-Albrecht (09:31:07): > Over and beyond: we could make it a GIF in the modal with data moving around, or brushing
Kevin Rue-Albrecht (09:31:50): > But a good static one is a perfect start, obviously
Aaron Lun (13:55:00): > Holy shit, I’m almost at 400 commits this month.
Aaron Lun (14:33:35): > I’ve run through a whole bunch of clicking and I’m happy with the interface. At some point we should draw up a list of expected outcomes from user actions - I’m not sure the vignette is detailed enough for this (and it probably shouldn’t be, as we’ll be getting VERY technical).
Aaron Lun (14:35:01): > There are only two issues; one is that for some reason, the brushing select-by-plot options don’t open at the right place, but this is pretty hard to reproduce so I’ve just left it.
Aaron Lun (14:35:31): > The other is that the color legend changes upon subset by restrict, if some colors no longer exist in the subset. Not sure whether this is desirable or not.
Kevin Rue-Albrecht (14:42:55) (in thread): > ‘Not sure whether this is desirable or not’ either, but it would be very simple to fix thanks toggplot. As long as the level is not dropped from the factor, allscale_*take the optiondrop=FALSEto preserve the legend in this kind of scenarios
Aaron Lun (14:43:58) (in thread): > Hm. Okay. Well, we’ll let it go for now, and see if anyone complains. They probably will.
Charlotte Soneson (14:54:09): > I just got an example CyTOF data set from Lukas for the vignette (basically the one Gosia and he used in the BioC workflow), together with all the code to generate it from the raw data (also mostly from the workflow). Below is a screen shot of loading a subset into iSEE (5000 cells, so I could calculate t-SNE quickly). Does anyone have an opinion of how to provide the data for the vignette (in the package, to download from somewhere)? I looked through BioC packages and I couldn’t really find one with a good, real example CyTOF data set that we could use as easily as the Allen scRNA-seq data. The .rds file of this subset is 1.4MB.
Charlotte Soneson (14:54:22): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-30 at 20.40.38.png - File (PNG): Screen Shot 2018-01-30 at 20.40.38.png
Federico Marini (14:55:35): > looks nice:slightly_smiling_face:
Federico Marini (14:55:48): > host it on google drive?
Federico Marini (14:55:52): > provided in the pkg?
Federico Marini (14:56:17): > separate pkg? or get it from the workflow? or even better, via ExperimentHub as recipe?
Charlotte Soneson (14:56:59): > Yeah, I was thinking about a “CyTOF example data set package” but the problem is that I guess it will take too long:slightly_smiling_face:
Charlotte Soneson (14:57:08): > It will not be available on BioC when we submit
Aaron Lun (14:57:24): > The problem is that cytof data doesn’t have a repository ala GEO or ArrayExpress
Aaron Lun (14:57:27): > Cytobank doesn’t count.
Charlotte Soneson (14:58:11): > I don’t know abouthttp://flowrepository.org/ - Attachment (flowrepository.org): FlowRepository > FlowRepository is a public database of flow cytometry experiments where you can query and download data collected and annotated according to the MIFlowCyt standard. It supports storage, annotation, analysis, and sharing of flow cytometry datasets.
Federico Marini (14:59:12): > is the construction possible from the workflow?
Federico Marini (14:59:18): > if following instructions
Charlotte Soneson (14:59:46): > Lukas put the code here:https://github.com/lmweber/CyTOF-example-data but yes, it is basically code from the workflow - Attachment (GitHub): lmweber/CyTOF-example-data > CyTOF-example-data - CyTOF example data sets: BCR-XL
Aaron Lun (15:02:08) (in thread): > Interesting. But we would ideally have something backed by a big institution e.g. NIH or EMBL.
Aaron Lun (15:02:20) (in thread): > Small repositories have a habit of disappearing when the money runs out.
Charlotte Soneson (15:02:28) (in thread): > yes, true
Federico Marini (15:02:50): > I’d say we pack it in?
Federico Marini (15:03:03): > or we link externally to the googledrive file?
Aaron Lun (15:04:42): > Pack it intoiSEE ? God no.
Aaron Lun (15:04:51): > Easier to make an ExperimentHub package.
Aaron Lun (15:04:57): > Provided it’s not too big.
Aaron Lun (15:06:20): > Are there any cytometry datasets already in experiment packages?
Aaron Lun (15:06:31): > Surely there must be something being used to test the flowCore stuff.
Aaron Lun (15:06:52): > EvenflowCoreitself had a small dataset inside it, if I remember correctly.
Charlotte Soneson (15:08:29): > Yeah, there is something there, and something incytofkit. I was mostly trying to link to the workflow and to a data set where we know something about the populations (manual annotation etc)
Aaron Lun (15:11:24): > Okay. But whatever you decide to do: I’ll just really not like to put any data iniSEE.
Aaron Lun (15:11:51): > I’m going home now, but rack up any bug reports on GH if you can’t deal with them yourself.
Aaron Lun (15:12:08): > If we’re all green, let’s go for the alpha tomorrow.
Kevin Rue-Albrecht (15:27:18): > Coming in a bit late: i looked for “cytof” in theexperimenthublast week and didn’t come up with anything in there. I would be in favour of uploading it in there, it’s good experience for whoever of us does it, and guarantees Bioc support
Federico Marini (15:27:31): > BTW, working on the two lenses of the glasses now
Federico Marini (15:27:38): > yes I also favor the EH way
Federico Marini (15:28:25): > I indirectly contributed to the EH1 recipe back then. I sent in some code to Sonali, who then took care of the whole rest. So i do not know the exact steps taken later
Kevin Rue-Albrecht (15:29:06): > I’ll get on theECMaspects now
Kevin Rue-Albrecht (15:31:17) (in thread): > well, the HTML vignette of the ehub seems pretty helpfulhttp://bioconductor.org/packages/release/bioc/vignettes/ExperimentHub/inst/doc/CreateAnExperimentHubPackage.html Makes clear that a Bioc member needs to be contacted, apparently Lori now
Federico Marini (15:35:30) (in thread): > Then Lori it will be:slightly_smiling_face:
Federico Marini (15:35:38) (in thread): > practical we met her in Cambridge
Kevin Rue-Albrecht (15:37:03) (in thread): > yeah it’s always nice to put a face on a name, and meet in person
Federico Marini (15:44:16) (in thread): > Glasses: to be found here:
Federico Marini (15:44:17) (in thread): > https://pixabay.com/en/fashion-glasses-hipster-optics-1295985/ - Attachment (pixabay.com): Free Image on Pixabay - Fashion, Glasses, Hipster, Optics > Download free pictures about Fashion, Glasses, Hipster, Optics from Pixabay’s library of over 1,300,000 public domain photos, illustrations and vectors - 1295985
Federico Marini (15:44:27) (in thread): > with CC0 license
Charlotte Soneson (16:07:41): > Maybe I should check with Lukas for the ExperimentHub thing since he prepared the data. But in any case it will not be up before the alpha release, so perhaps we still have to put it somewhere (like Google Drive) for now.
Kevin Rue-Albrecht (16:12:10): > Google drive or Dropbox should be fine indeed. It’s all documented in section ‘Using online file sharing systems’ of the vignette
Kevin Rue-Albrecht (16:15:38): > btw, I haven’t documented something that will probably confuse many users as it did for me: “copy Dropbox link” puts in the clipboard something likehttps://www.dropbox.com/ …?dl=0. > To programmatically fetch the data at that link, the finaldl=0should be edited todl=1.
Kevin Rue-Albrecht (16:16:12): > Didn’t know how to write that clearly at the time. But maybe just the above message would be enough. Opinions?
Kevin Rue-Albrecht (16:21:50): > On a separate note, for anytestthatenthusiast out there: it seems that theexpect_*functions cannot handle comparison of functions when called from thecovrpackage: > I was testing the colormaps returned in various scenarios against the hard-coded default values and: > - tests were passing when run at the prompt in RStudio > - tests passed when called through thetestthatpackage: > > > test_dir("tests/testthat") > ✔ | OK F W S | Context > ⠼ | 5 | 0 > ══ Results ══════════════════════════════════════════════════════════════════════════════════════════════ > Duration: 0.2 s > > OK: 5 > Failed: 0 > Warnings: 0 > Skipped: 0 > > - tests were failing if called bycovr::package_coverage():target, current do not match when deparsed
Kevin Rue-Albrecht (16:24:13) (in thread): > maybe something to do withhttps://github.com/r-lib/covr/issues/248 - Attachment (GitHub): Problem using covr in testthat testcases · Issue #248 · r-lib/covr > Hi, As part of my package’s testthat test suite, I would like to include testcases for code coverage (covr version 2.2.2), e.g. that the average code coverage is over 90%. I have written a simple f…
2018-01-31
Charlotte Soneson (03:56:29): > I was just trying to set the starting configuration for the CyTOF example, and everything works fine until I try to define the brushing relationships. The problem is in.sanitize_memory, but I’m afraid that I’m not exactly following what is being checked there (l. 153-157,misc.R).:disappointed:I’m most likely missing something obvious, but why is it “bad” if the specified plot is inbrush_names?
Charlotte Soneson (03:56:38): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-01-31 at 09.11.39.png - File (PNG): Screen Shot 2018-01-31 at 09.11.39.png
Kevin Rue-Albrecht (04:22:39): > heyyyy - i got that same error too!
Kevin Rue-Albrecht (04:22:51): > except it was 20 rows to replace 10
Kevin Rue-Albrecht (04:22:58): > so it’s a problem of doubling somewhere
Kevin Rue-Albrecht (04:23:26): > i tried placing lots ofprintdebug around the place, yet couldn’t find where it happens
Kevin Rue-Albrecht (04:25:00): > let me find my config, although I got too bold and tried to define some values dynamically. I thought I was the problem when I got the error
Kevin Rue-Albrecht (04:27:54): > here is mine: > > iSEE_CyTOF <- function( > se, > redDimMax = 1, > colDataMax = 1, > geneExprMax = min(10, nrow(se)/2), > geneStatMax = min(10, nrow(se)/2), > initialPanels = DataFrame( > Name = c( > sprintf("Gene expression plot %i", seq(1,min(3,geneExprMax),1)), > sprintf("Gene statistics table %i", seq(1,min(3,geneStatMax),1)), > sprintf("Reduced dimension plot %i", 1), > sprintf("Column data plot %i", 1) > ), > Width = c( > rep(3, min(3,geneExprMax) + min(3,geneStatMax)), > 4,4 > ) > ), > ... > ){ > message("redDimMax: ", redDimMax) > message("colDataMax: ", colDataMax) > message("geneExprMax: ", geneExprMax) > message("geneStatMax: ", geneStatMax) > print(initialPanels) > > redDimArgs <- redDimPlotDefaults(se, redDimMax) > colDataArgs <- colDataPlotDefaults(se, colDataMax) > geneExprArgs <- geneExprPlotDefaults(se, geneExprMax) > geneStatArgs <- geneStatTableDefaults(se, geneStatMax) > > # prepare geneExprArgs > geneExprArgs$XAxis <- "Gene text" > geneExprArgs$YAxis <- "Gene text" > geneExprArgs$XAxisGeneText <- rownames(se)[seq(1,nrow(geneExprArgs)*2,2)] > geneExprArgs$YAxisGeneText <- rownames(se)[seq(2,nrow(geneExprArgs)*2,2)] > geneExprArgs$BrushByPlot <- c("", sprintf("Gene expression plot %i", seq(1,2,1)), rep("",nrow(geneExprArgs)-3)) > geneExprArgs$BrushEffect <- c("","Restrict", "Color", rep("",nrow(geneExprArgs)-3)) > print(geneExprArgs) > print(dim(geneExprArgs)) > iSEE( > se=se, > redDimArgs=redDimArgs, > colDataArgs=colDataArgs, > geneExprArgs=geneExprArgs, > geneStatArgs=geneStatArgs, > redDimMax=redDimMax, > colDataMax=colDataMax, > geneExprMax=geneExprMax, > geneStatMax=geneStatMax, > initialPanels=initialPanels, > ... > ) > } > > iSEE_CyTOF(sce) >
Aaron Lun (04:29:08) (in thread): > Suggest testing evaluation of the functions for a few inputs, e.g., N=1, 3, 5, 10
Aaron Lun (04:36:05): > Bugfix up.
Aaron Lun (04:44:21): > Okay, now actually fixed.
Aaron Lun (04:45:54): > It should have removed names that were not amongst the existing panels.
Charlotte Soneson (04:50:04): > Yes, that does indeed make more sense:slightly_smiling_face:I can create the panels and brushing relations now:+1:
Aaron Lun (05:08:23): > Are we otherwise good to go?
Charlotte Soneson (05:09:11): > I’m extending the vignette, but otherwise I would say yes
Aaron Lun (05:10:21): > Will you be done soonish, or should we just do it?
Charlotte Soneson (05:11:37): > I plan to be done today, but I have a meeting in a little while and I don’t know if I’ll be done before that. Maybe we can just do it now, it is easy for people to reinstall later…
Aaron Lun (05:12:15): > Let me review the current status of the vignette…
Aaron Lun (05:13:37): > I will clean up the allen formatting and will go ahead.
Aaron Lun (05:13:57): > Also, I noticed this:sce1@assays@.xData$data$exprs. Urgh.
Aaron Lun (05:14:13): > because clearly,exprs(sce1)doesn’t look smart enough.
Kevin Rue-Albrecht (05:29:45): > i’m happy with the state in which I brought things last night, and you all seem to have deep-tested the various kinds of interactions between brushes. > The CyTOF section stills stops in the middle of a sentence, Are we extending it or removing it until we have resolved the data set availability?
Aaron Lun (05:32:14): > Suggest commenting it out for the time being.
Charlotte Soneson (05:33:07): > Fine with me
Aaron Lun (05:36:18): > When you compile your vignettes, doesinteractive()actually work? My vigentte compilation freezes, presumably because it actually tries to open the app.
Aaron Lun (05:39:28): > Also, the ECM fails when not all rowData, colData, and assays are specified inall_continuous.
Aaron Lun (05:39:32): > Don’t even know what rowData is.
Aaron Lun (05:39:48): > ah, actulaly nvm.
Kevin Rue-Albrecht (05:47:51) (in thread): > how do you compile them?
Kevin Rue-Albrecht (05:48:27) (in thread): > i usedevtools::install(build_vignettes=TRUE)and never had a problem
Kevin Rue-Albrecht (05:49:20) (in thread): > can you elaborate on your “ah, actulaly nvm.“?
Kevin Rue-Albrecht (05:50:07) (in thread): > is there still a bug? do I throw an invalid error? or does it make sense in the end?
Kevin Rue-Albrecht (05:55:52) (in thread): > oh noo. it is indeed a bug
Kevin Rue-Albrecht (05:56:04) (in thread): > i thought I had this one solved last night
Kevin Rue-Albrecht (05:56:06) (in thread): > damn
Aaron Lun (05:57:08) (in thread): > Oh, I thought it was okay.
Aaron Lun (05:57:29) (in thread): > Just came back from the toilet.
Kevin Rue-Albrecht (05:58:00) (in thread): > well, the whole point of.sanitize_controlled_colormapswas to set missing color maps inall*
Aaron Lun (05:58:02) (in thread): > R CMD build
Kevin Rue-Albrecht (05:58:57) (in thread): > I think I pinpointed the issue
Kevin Rue-Albrecht (05:59:06) (in thread): > hang on
Kevin Rue-Albrecht (05:59:28) (in thread): > Yep: bug only happens when callingnew
Kevin Rue-Albrecht (06:00:02) (in thread): > if you call theExperimentColorMapconstructor, it handles it the way I meant
Kevin Rue-Albrecht (06:01:52) (in thread): > ahhh i think i know how to handle that
Kevin Rue-Albrecht (06:02:12) (in thread): > ah. no.
Aaron Lun (06:02:52) (in thread): > Well, NVM. People callingnewshould know what they’re doing.
Kevin Rue-Albrecht (06:03:05) (in thread): > I guess we should demonstrate theExperimentColorMapconstructor in the help page rather than thenewmethod
Aaron Lun (06:03:56) (in thread): > Yes, I’ve changed this in the vignette.
Kevin Rue-Albrecht (06:04:21) (in thread): > do you wanna do man pages too?iSEEandECM-class?
Kevin Rue-Albrecht (06:04:36) (in thread): > I was about to start but i need to get going for other stuff now
Aaron Lun (06:04:59) (in thread): > I can… but I would like to reduce the size of the ECM example in?iSEE. Seems better to have a comprehensive example in the ECM man.
Aaron Lun (06:15:33): > I’ve noticed that the text keeps on getting contracted to 80 lines. Why is this?
Kevin Rue-Albrecht (06:17:09) (in thread): > absolutely
Kevin Rue-Albrecht (06:19:01) (in thread): > I think the one in iSEE doesn’t need all theassaysthings anyway: one would be plenty enough
Charlotte Soneson (06:19:04): > which text? 80 lines or lines of length 80? The latter is what BiocCheck wants, right:wink:I don’t think I have anything setup to automatically change the line length though.
Aaron Lun (06:19:15): > Lines of width 80.
Aaron Lun (06:19:23): > And BiocCheck is stupid on this.
Aaron Lun (06:19:34): > Who has screens that small?
Aaron Lun (06:19:54): > Text is… okay. But code doesn’t make any sense when split across lines.
Charlotte Soneson (06:20:05): > Haha, I don’t know. But you have to fit several panels next to each other in RStudio…
Kevin Rue-Albrecht (06:20:12): > It might be my RStudio. I eventtually manage to tell it ‘2-tab indents’, but I don’t remember touching the line width, so it might still be at 80
Aaron Lun (06:20:16): > Don’t get me started on Rstudio.
Kevin Rue-Albrecht (06:21:22): > If I remember correctly, the 80 char thing is historical. Something about the first screens only showing that many. But yeah.. I’d pity anyone who still has a screen like that
Aaron Lun (06:22:13) (in thread): > I fleshed out the ECM explanation in the vignette so I removed the ECM definitions in?iSEEcompletely.
Aaron Lun (06:24:01): > Why do we need to import viridisLite? Shouldn’t viridis be sufficient?
Aaron Lun (06:25:23): > Also, it doesn’t seem like RColorBrewer is actually used in the package itself, suggest moving it to Suggests.
Kevin Rue-Albrecht (06:28:52): > Good question aboutviridisLite.R CMD checkcomplained about not finding theviridisfunction until I addedviridisLite. Weirdly theviridisfunction is exported by both.
Aaron Lun (06:32:36): > Hm. Well, I’ll moveviridisto Suggests andviridisLiteto Imports.
Kevin Rue-Albrecht (06:34:27): > Fair enough. I haven’t tried all combinations, maybe that one will work. Note thatcheckmay complain if you import from a package inSuggests. I importviridisfrom both in theNAMESPACE
Kevin Rue-Albrecht (06:34:44): > anyway, i’m sure you’ll figure it out:wink:
Aaron Lun (06:36:02): > <!channel>Alpha going out before lunch. Final comments.
Kevin Rue-Albrecht (06:44:55): > good point removing the ECM altogether in ?iSEE
Aaron Lun (06:46:51): > In the ECM man page, does"undefined"refer toall_discreteorall_continuous? This would be dangerous if there is actually a field named"undefined"; suggest putting a separate flag or accessor, e.g.,undefined=TRUEorcolDataUndefinedColorMap.
Aaron Lun (06:47:11): > I have had numerous problems with cleaning upscater ’s code because of issues like this.
Kevin Rue-Albrecht (06:47:34): > No sorry, misleading then: it was meant to be an example of try to access a color not defined in the ECM
Kevin Rue-Albrecht (06:48:03): > to demonstrate how it falls back through the hierarchy rather than crashing
Aaron Lun (06:48:16): > okay.
Kevin Rue-Albrecht (06:48:20): > should probably be prefaced by a comment then
Aaron Lun (06:48:33): > Yeah
Aaron Lun (06:48:46): > Or name it something silly, like"blah"so people can’t get confused.
Kevin Rue-Albrecht (06:49:36): > well.. that was the point of"undefined"… before we introduced theall_*slots
Kevin Rue-Albrecht (06:50:04): > there will always be confused people, unfortunately, one way or another
Aaron Lun (06:53:04): > I’m happy with deployment, once the latest PR merges. Though I have a couple of cosmetic issues that I’ll post on GH.
Kevin Rue-Albrecht (06:54:34): > Just found: open the vignette from within the app crashes if the vignette wasn’t built. Error message:Warning: Error in browseURL: 'url' must be a non-empty character string
Kevin Rue-Albrecht (06:54:52): > We could probably catch that up with avalidate
Aaron Lun (06:55:06): > Yes, we could; but all installations should have the vignette installed.
Aaron Lun (06:55:14): > As in, all official insallations via bioclite.
Aaron Lun (06:55:26): > I mean, we can do it (raise a Notification)
Kevin Rue-Albrecht (06:55:29): > My thought as well. Just thought i’d throw it out there anyway.
Aaron Lun (06:59:45): > Okay, protection has been added.
Aaron Lun (07:02:10): > I will merge the PR and write the email after lunch.
Federico Marini (07:28:22): > sorry for joining later, kids were sick and I had <<< time to get busy otherwise
Federico Marini (07:29:08): > I did not have time to check any more bugs, but I am sure we were good in catching already
Federico Marini (07:29:18): > how much do we want the logo for today’s release?
Kevin Rue-Albrecht (07:30:49): > To me, it would only be the cherry on top. I’d rather wait longer to see something as final as possible, than immediately include a work in progress
Federico Marini (07:35:33): > I could not find out yesterday what lens 2 could have been. Although I kind of liked Charlotte’s CD3 vs CD4 abundance
Federico Marini (07:35:57): > for lens 1, I’d have the TCGA ninja-tSNE-glyph
Kevin Rue-Albrecht (07:38:19): > i like that shuriken:grin:
Kevin Rue-Albrecht (07:39:32): > i’ll think about it over lunch and group meeting this afternon
Aaron Lun (07:51:57): > Sending in 10 minutes, unless I’m stopped.
Aaron Lun (07:57:18): > Okay, I need to go, so I’m just going to send it now.
Charlotte Soneson (07:58:07): > Sounds good!
Federico Marini (08:12:01): > feel free to add us in cc so that we get notifications?
Aaron Lun (09:59:57): > I think we should all go out for a drink.
Aaron Lun (10:00:13): > And each of us can imagine the others there.
Charlotte Soneson (10:08:25): > I guess that will work as long as we don’t start talking to the “virtual us”…:slightly_smiling_face:but yes, really good job!
Aaron Lun (10:10:08) (in thread): > Sent it just after C’s reply, so… nope.
Federico Marini (10:13:41): > hey, virtual “colleagues”. proud to be contributing in this
Federico Marini (10:14:18): > the more we drink, the more bugs will be filed in - but this is what we want i.e. outsourcing the debugging:smile:
Federico Marini (10:14:34) (in thread): > :stuck_out_tongue:
Aaron Lun (10:15:45): > We can play a drinking game.
Aaron Lun (10:15:58): > Where the person who wrote the buggy line of code has to buy the next round.
Aaron Lun (10:16:09): > Weighted by the total number of lines contributed, of course.
Federico Marini (10:19:07): > eheheh
Federico Marini (10:19:11): > I’m in
Charlotte Soneson (10:20:19): > Ehm, sure. Drinks are probably most expensive in Switzerland anyway:wink:
Aaron Lun (10:22:47): > I’ll put it on the agenda at the next Bioc meet. Got to wait for those bugs to build up to avoid sampling error, you know.
Federico Marini (10:23:16): > munich prices are not the lowest either but hey
Aaron Lun (12:39:28): > How to turn off geom_violin line color?
Kevin Rue-Albrecht (12:40:02): > aes(colour=...))I believe
Kevin Rue-Albrecht (12:40:14): > as opposed toaes(fill=...))
Aaron Lun (12:40:37): > right.
Aaron Lun (12:40:50): > colour=FALSE in .build_aes?
Aaron Lun (12:40:55): > Let’s see if that works..
Kevin Rue-Albrecht (12:41:09): > yep
Kevin Rue-Albrecht (12:42:24): > I wish I had.build_aesearlier… can’t tell you how often I type what that function returns
Aaron Lun (12:42:56): > Allright, fill in the violin plots has been destroyed.
Aaron Lun (12:43:06): > Now for how to avoid dropping levels from the legend.
Kevin Rue-Albrecht (12:43:25): > scale_*(..., drop=FALSE)
Aaron Lun (13:06:22): > Is this only necessary for manual? Or also for gradientn? And also for fill?
Kevin Rue-Albrecht (13:06:54): > immediate intuition is that it’s for all
Aaron Lun (13:06:57): > Also I remember the reason for fill was to avoid two color legends - how does this work?
Kevin Rue-Albrecht (13:08:29): > i’m pretty sure you needscale_color_manual(..., drop=FALSE)to preserve all you colours if one is absent from the data
Aaron Lun (13:08:46): > Nope, didn’t work.
Kevin Rue-Albrecht (13:09:13): > the other way around i’m not sure how ggplot may complain ifscale_*_manualdoes not provide a colour for every level
Kevin Rue-Albrecht (13:09:18): > uh
Aaron Lun (13:10:12): > Continuous was also very unhappy aboutdrop=FALSE.
Kevin Rue-Albrecht (13:10:19) (in thread): > color``fillandshapeinteract as factors into creategroup(hence thegroupaesthetic to override it if desired
Aaron Lun (13:10:37) (in thread): > jesus.
Kevin Rue-Albrecht (13:10:40): > ah yeah that makes sense for continuous xD
Kevin Rue-Albrecht (13:10:49) (in thread): > but
Kevin Rue-Albrecht (13:11:14) (in thread): > aesthetics that are tied to the same covariate are collapsed into a single legend
Aaron Lun (13:11:37): > Let me rephrase: it works as long as there is at least one point.
Aaron Lun (13:11:46): > Once there’s no points, everything vanishes, which is not entirely desirable.
Kevin Rue-Albrecht (13:11:49) (in thread): > so we made the double legend disappear by making fill the same as colour
Kevin Rue-Albrecht (13:12:00): > argh
Aaron Lun (13:13:19): > Not sure how to fix that, unfortunately.
Kevin Rue-Albrecht (13:13:35) (in thread): > i’d have to look at the code myself - i’m a bit tired now, but almost all aesthetics can be specifiedoutside theaescall, if one wishes to set them to a constant value (typical forgeom_point(aes(...), size = ...))
Aaron Lun (13:14:52) (in thread): > Yeah, see if it can be cleaned up.
Kevin Rue-Albrecht (13:15:51): > Yeah. As you say ggplot makes simple things complicated
Kevin Rue-Albrecht (13:16:18): > it’s really smart at figuring out how to draw legends from data, so… take away the data…
Aaron Lun (13:17:52): > I wonder whether we could just add a dummy value toplot.data, somewhere outside the range of the plot.
Aaron Lun (13:18:02): > Trick ggplot into putting something.
Aaron Lun (13:28:05): > @Charlotte SonesonCan you help standardize the usage of transparencies whennot brushing? Some plots have transparencies by default ingeom_point, while others do not. I guess the simplest and most easily interpretable solution would be to turn transparencies off totally in the absence of a brush; otherwise users may be confused when brushing by transprency. Any suggestions?
Charlotte Soneson (14:15:11): > Right, yes. I was thinking about this earlier today for the CyTOF thing, that in a way it would be useful to have a transparency in the scatter plots there to get an indication about the density, but it would be hard to separate from the brushing effect. So yes, I guess the safest is to turn the transparencies off in absence of brushing.
Aaron Lun (14:19:20): > I’m happy with having the transparencies, but you’ll have to be careful about how they work.
Aaron Lun (14:20:21): > Upon brushing by color, all brushed points should get a solid fill? Or also transparency?
Aaron Lun (14:20:39): > Upon brushing by transparency, brushed points should lose their transparency; this is probably fair enough.
Charlotte Soneson (14:21:38): > But then the original plot and the plot you would get by brushing all points would look different, which I don’t know if it’s desirable
Aaron Lun (14:21:45): > Hm.
Aaron Lun (14:22:30): > Is it undesirable?
Charlotte Soneson (14:22:49): > I don’t know
Charlotte Soneson (14:26:24): > So in that case we’d have have to change the non-brushing transparency in the reduced dimension plots, and harmonize the ones in violins and rectangle plots.
Aaron Lun (14:34:32): > Yep.
Charlotte Soneson (14:39:18): > I’ll look into this.
Kevin Rue-Albrecht (14:39:40): > I was wondering about that too: whether brushing by ‘transparency’ should also apply a ‘restrict’ on the child plots. It’s both fair enough and a bit frustrating that transparent data in the parent plot get highlighted in the child plot
Charlotte Soneson (14:39:46): > I just pushed thecharlotte_fixesbranch, which contains an updated vignette (with the CyTOF example), if anyone wants to have a look
Charlotte Soneson (15:02:38): > Also pushed thetransparencybranch. There, alpha=0.6 everywhere except when receiving a transparency brush, in which case it is set by the brush settings.
Aaron Lun (15:49:57): > I was thinking further about this; we can fob off responsibility to the user, by giving an option to set the alpha in the colour parameters.
Aaron Lun (15:50:32): > This would default to 1, but it would allow the user to turn it down if they wanted.
Charlotte Soneson (15:50:41): > Yes. I guess we need to decide how much we want them to be able to change. What about point size for example?
Aaron Lun (15:51:49): > I did think about this, but the point size is probably no so important?
Charlotte Soneson (15:52:15): > Maybe depends on how many samples/cells you have. I could think that with large data sets our point size is too large
Charlotte Soneson (15:52:26): > And with a bulk data set it is probably too small:slightly_smiling_face:
Aaron Lun (15:52:54): > Probably settable in the command rather than anything else…
Aaron Lun (15:53:18): > Though we could change “Coloring parameters” to “Aesthetic parameters”
Kevin Rue-Albrecht (15:53:37): > I’m not against another button on the interface (in a modal I assume), but we could also add this as a bonus argument to iSEE() that would initialise it to the user’s favorite value (?)
Kevin Rue-Albrecht (15:54:55): > I personally already have my launcher scripts for a couple of data sets, this way I could just tweak those and never have to worry again about changing it in-app/per-app
Aaron Lun (15:58:43): > I think it should be set as an argument on the command line -point.args=list(), extra arguments to be passed togeom_point.
Aaron Lun (15:59:47): > This’ll get deparsed as required (excluding any suppliedalphawhen brushing for transparency). I hope it doesn’t throw up when you specify color here and thenscale_color_*later.
Kevin Rue-Albrecht (16:12:12): > if I follow your last bit, I don’t it would be a massive problem: general settings would applyoutside aescalls, whilescale_*applyto aesthetics
Kevin Rue-Albrecht (16:12:37): > i’d have to play around to see what happens when one sets both simultaneously, though
Kevin Rue-Albrecht (16:13:33): > but as a general rule, i’d expect the code to switch between the two types of calls
Aaron Lun (17:53:09): > Hm… upon further reflection, I’m not sure it would be wise to let the user tinker with stuff iniSEE().
Aaron Lun (17:53:25): > It would have to really affect visualization to warrant putting it in as an argument.
Aaron Lun (17:53:34): > I mean, what’s next; font size? label size?
Aaron Lun (18:14:32): > But on the other hand, these might be necessary…
Aaron Lun (18:14:42): > Dammit.
Aaron Lun (18:16:01): > Well, if anyone feels like this is necessary, give me an idea of how it woudl fit in with ggplot and what type of extra arguments would be needed iniSEE.
2018-02-01
Kevin Rue-Albrecht (03:39:21): > Another desirable feature that just came back to mind: upon brushing a message under the plot “n of N data points selected (x%)”
Kevin Rue-Albrecht (03:53:28): > @Aaron LunI’ll think more and wrote a proper suggestion about ggplot later but I’d say iSEE() should also accept a sort of global setting argument to store things valid across multiple plots of the same type, and maybe a last one across all plot types
Kevin Rue-Albrecht (03:54:53): > I can’t really really predict all the ggplot side though. If you set up the infrastructure as you previously did, I can pick it up and apply it. It’s worked so far:)
Kevin Rue-Albrecht (03:57:46): > I’m sure Charlotte will have some good idea about it too, if I miss something
Aaron Lun (04:25:41): > Can someone remind me whatscale_size_areadoes?
Aaron Lun (04:26:57): > I’m looking at it in the gridplot command, and I am not sure why it is there.
Charlotte Soneson (04:37:18): > The help saysscale_size_areaif you want 0 values to be mapped to points with size 0…otherwise I don’t know, it’s not something I’ve ever used
Aaron Lun (04:38:08): > ???
Aaron Lun (04:38:37): > In any case, I’ve found Yet Another ggplot Unexpected Behaviour (YAGUB for short).
Aaron Lun (04:39:17): > Will elaborate once I get back from a summons from the boss
Kevin Rue-Albrecht (04:55:17): > scale_size_areais part of the family withscale_sizeandscale_size_areaI remember suggesting it. but can’t remember exactly for what purpose now
Kevin Rue-Albrecht (04:55:39): > i think the size of the boxes
Kevin Rue-Albrecht (04:56:09): > i have do dash to a genomics forum in 2 min, but let me look at the code
Charlotte Soneson (05:00:44): > For the additionalggplotoptions, to me it seems that the problem is perhaps not to give it toiSEE(in the end, it will translate to additional columns of thegeneExprArgsetc, right?) We also don’t have much currently in thetheme, so there could be another function setting up that, using the provided defaults (it all has to be in the sametheme(...)I think, to avoid overwriting). For the integration, the risk is perhaps to clutter the interface, and that if you provide some ways of fiddling, people will start wondering why they can’t doall the things they want:wink:
Charlotte Soneson (05:02:10): > One thing that we could address if we do it is the overlapping of axis labels for e.g. the violin plot (if there are many/long group names), by allowing the user to rotate the labels
Aaron Lun (05:22:55): > Well, putting that aside, let’s fix this first:
Aaron Lun (05:23:14): > 1. Load theallendata set from?iSEE.
Aaron Lun (05:23:59): > 2. Set up the column data plot to group bydissection_s.
Aaron Lun (05:24:11): > 3. Set up the column data plot receive a brush from the reduced dim plot.
Aaron Lun (05:24:21): > 4. Set up to brushing to restrict in the column data plot.
Aaron Lun (05:24:43): > 5. Select the brush in the reduced dim plot, starting from (5, -5) and all the way down to the bottom right corner.
Aaron Lun (05:25:11): > You will notice that the widths of the violin plots expand beyond an acceptable range. The question is, how do we control the maximum width of the violins?
Charlotte Soneson (05:25:11): > Oh…
Charlotte Soneson (05:30:20): > Settingwidth=1ingeom_violin?
Charlotte Soneson (05:31:47): > or maybe slightly smaller
Charlotte Soneson (05:35:35): > seems default width is 0.9
Aaron Lun (05:37:51): > I don’t even see awidthargument ingeom_violin.
Charlotte Soneson (05:38:01): > There is one, but apparently it is not documented:slightly_smiling_face:
Aaron Lun (05:38:07): > Typical.
Charlotte Soneson (05:38:16): > Just addingwidth = 0.9togeom_violin()seems to do it
Aaron Lun (05:38:39): > The vipor offsets have width=0.4. So would width=0.8 be better?
Charlotte Soneson (05:39:57): > Yeah, maybe this is the reason that the violins were a bit outside the points in the TCGA data? On the other hand, for smaller data sets it looks like a part of the point may end up looking like it’s outside the violin if it is narrower than 0.9.
Aaron Lun (05:40:21): > Hm.
Aaron Lun (05:40:28): > Do we still needscale='width', then.?
Charlotte Soneson (05:40:37): > Yes, otherwise they will scale by area
Aaron Lun (05:40:43): > Okay.
Charlotte Soneson (05:47:44): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-01 at 11.47.02.png and commented: This is what happens withwidth=0.8. It agrees with the width of the point centers, but some part of the points fall outside - File (PNG): Screen Shot 2018-02-01 at 11.47.02.png
Aaron Lun (05:48:07): > width=0.9 is still not working for me with the commands I reported above. Are they working for you?
Charlotte Soneson (05:48:19): > Yes
Aaron Lun (05:48:49): > Dammit
Charlotte Soneson (05:48:53): > I changed lines 317-319 inplotting.Rto
Charlotte Soneson (05:48:58): > > plot_cmds[["violin"]] <- sprintf( > "geom_violin(%s, alpha = 0.2, data=plot.data, scale = 'width', width = 0.9) +", > .build_aes(color = FALSE, group = TRUE) >
Charlotte Soneson (05:49:02): > And that works
Aaron Lun (05:49:47): > Hm. Can you commit a fix on your end then? Do it frombugfix, as I’ve refactored everything (again).
Charlotte Soneson (05:51:33): > pushed tobugfix
Aaron Lun (05:53:18): > Hm. Don’t know how it worked, but it did.
Aaron Lun (05:54:53): > BTW, does the loading bar help at all for the cytof stuff?
Charlotte Soneson (06:02:29): > Right, I can’t see the loading bar…where should I look?
Charlotte Soneson (06:05:26): > Oh, now I see it:slightly_smiling_face:but not on startup, right?
Charlotte Soneson (06:06:08): > Doesn’t really make a big difference, rendering the plots is pretty fast
Aaron Lun (06:07:14): > Yeah, I thought so.
Aaron Lun (06:07:21): > It should be only present on startup.
Aaron Lun (06:07:28): > But I guess that was a waste of time.
Kevin Rue-Albrecht (06:09:46) (in thread): > @Charlotte Soneson@Aaron Lunthat’s the thing: i don’t think ‘global’ arguments should get neargeneExprArgs, as that would create unnecessary redundancy. As for the ECM, I would have expected a hierarchy of ‘fallback’ options, each stored in a different, progressively smaller, data frames
Aaron Lun (06:10:06) (in thread): > Yes, global arguments will go somewhere else.
Aaron Lun (06:10:23) (in thread): > But not via a fallback system, that would be a real pain in the ass.
Aaron Lun (06:10:30) (in thread): > Especially when things change dynamically.
Kevin Rue-Albrecht (06:10:57) (in thread): > yep, needs some careful planning before implementation
Kevin Rue-Albrecht (06:11:23) (in thread): > still only throwing ideas as my time is fairly still limited these days
Kevin Rue-Albrecht (06:15:10): > I think it will become more useful when we get to much bigger data sets that would require stuff as downsampling for instance
Aaron Lun (06:17:26): > The question is; would figuring out the points to downsample take as much time as plotting them anyway?
Aaron Lun (06:17:55): > Intelligent downsampling would need to be density-aware, requiring approximately NlogN time.
Aaron Lun (06:18:00): > Whereas just plotting them should be a linear operation.
Kevin Rue-Albrecht (06:21:35): > Well, to be honest, I wouldn’t considering ‘rendering’ as part of any progress bar in our app. That’s more ggplot’s problem, ours is to supply a decent amount of data. > Progress bars are more useful to display progress through analytical steps: e.g. ‘collating data’, ‘identifying axes range’, ‘downsampling’, ‘post-processing’, …
Kevin Rue-Albrecht (06:22:57): > not sure how much experience you have with progress bars, but inTVTB, I used that to increment the progress bars through 3-4 major data processing steps that could take some time, but never had to worry about the rendering. I think that’s our situation here too
Kevin Rue-Albrecht (06:24:13): > if you want to check it out:https://github.com/kevinrue/TVTB/blob/459b69f8211125427de6c66b034203fa47485110/inst/shinyApp/server.R#L450
Kevin Rue-Albrecht (06:24:40): > and have a look at theincProgresscommands
Aaron Lun (06:24:41): > The problem is that almost all of the evaluations are done in theeval.
Kevin Rue-Albrecht (06:24:57): > argh. yes. sorry. I keep forgettign that
Aaron Lun (06:25:17): > There’s only one place where insertion can be performed; after the firstevalto obtain X/Y/ColorBy.
Aaron Lun (06:25:24): > But I can’t imagine that would take particularly long…
Aaron Lun (06:25:46): > Unless we get into disk-backed operations, but that’s another story.
Kevin Rue-Albrecht (06:25:49): > … I’m curious to see if we can place theincProgressstatements in the evaluated command, and wrap all that in awithProgress
Aaron Lun (06:26:36): > We could, provided we passed in thesession. Not sure it’s worth it, though.
Aaron Lun (06:27:10): > Assuming that the major slowdowns will be in generatingplot.data, we should only needincProgressin between the twoevalblocks.
Federico Marini (06:30:05): > (side note: good catch on place 1 for the bioc devel daily digest, aaron:slightly_smiling_face:)
Aaron Lun (06:30:27): > oh - uh, yes, that was all part of my plan.
Federico Marini (06:30:31): > plus, we started some socialmedia-ads with out twitters
Federico Marini (06:30:42): > should we notify also the general channel here?
Aaron Lun (06:31:15): > Not sure that’s necessary… though if we were to do it, we should have done this yesterday.
Aaron Lun (06:31:31): > Now it just seems a bit weird, like an afterthought (which it is).
Federico Marini (06:34:23): > :stuck_out_tongue:
Federico Marini (06:34:56): > we say something like “instead of nothing, it’s better instead of”:smile:
Charlotte Soneson (07:50:20) (in thread): > Hmm…I see the point, but wouldn’t a hierarchy of data frames be more difficult to keep track of than just another bunch of columns in thegeneExprArgsDataFrame? Most of the rows there will be identical anyway (at least to start with), and it would be immediate to allow users to change the plots independently, or set a global default at startup. Or is the main issue then that you can’t make global changes while running the app?
Kevin Rue-Albrecht (07:58:54) (in thread): > Well, I think global changes would be possible, albeit not necessarily straightforward to implement. My motivation would rather for our own sanity, to keep separate things that are controlled in different ways; better said, to group things that are controlled in similar ways (e.g. individual plots, plot type, across plot types)
Charlotte Soneson (11:56:15): > Anyone has an opinion on the CyTOF example (in thecharlotte_fixesbranch)? I put the data file on Dropbox for now, we have started to put together a CyTOF data package but it will take some time. If you prefer we can keep it commented.
Federico Marini (11:56:53): > Did not check yet, I’ll take a look
Federico Marini (11:57:01): > is the EH way too complicated?
Charlotte Soneson (11:57:11): > No, that’s what we are doing
Charlotte Soneson (11:57:19): > But it still needs a package
Federico Marini (11:57:23): > ah ok
Federico Marini (11:57:37): > thought it could “simply be a recipe”
Charlotte Soneson (11:57:43): > I can also put the file on our server.
Charlotte Soneson (12:23:46): > BTW, now that it is released, we may want to start bumping the version number when we change things (and adding things to the NEWS file)?
Aaron Lun (12:24:22): > Meh.
Aaron Lun (12:24:36): > Feel free to knock up the version, as long as it’s below 0.99.0
Aaron Lun (12:25:03): > Also, suggest moving NEWS to inst.
Charlotte Soneson (12:26:32): > Just thinking that it might be easier to track bugs etc if not everything is from the same version…but maybe it’s unnecessary?
Aaron Lun (12:26:41): > Well, that may be true.
Aaron Lun (12:27:14): > Maybe any PR can result in a bump, though this would get us into pretty large numbers pretty quickly.
Kevin Rue-Albrecht (12:30:07): > Version bumps with PR is probably sensible, provided the PR do something notable (i.e., not a typo in a help page). Minor edits should probably be merged with only a new bullet point in theNEWS
Kevin Rue-Albrecht (12:30:53): > this way the minor edits get distributed as part of the next ‘notable’ update
Aaron Lun (12:32:26): > I mean, in practice, people should only be downloading the alpha release.
Aaron Lun (12:32:36): > And we would tag new releases as we go along.
Aaron Lun (12:32:51): > But I guess people would just godevtools::install_github("csoneson/iSEE")in reality.
Kevin Rue-Albrecht (12:33:38): > exactly, that would match better the Bioc way of propagating new releases (alpha, and later tags)
Kevin Rue-Albrecht (12:34:22): > the devtools way would be for us and people who don’t mind testing the latest code
Aaron Lun (15:18:22): > Looking at thescater code currently.
Aaron Lun (15:18:34): > If I see a long Roxygenated file with multiple user-level functions, I will scream
2018-02-02
Kevin Rue-Albrecht (06:07:57): > Hey all, a bit off topic here, but did anyone recently play with R=3.4? > I’m getting the following error: > > > biocLite("SingleCellExperiment") > BioC_mirror:[https://bioconductor.org](https://bioconductor.org)Using Bioconductor 3.5 (BiocInstaller 1.26.1), R 3.4.3 (2017-11-30). > Installing package(s) 'SingleCellExperiment' > Warning message: > package 'SingleCellExperiment' is not available (for R version 3.4.3) > > while the package should be available for that version of R:https://bioconductor.org/packages/release/bioc/html/SingleCellExperiment.html
Charlotte Soneson (06:09:22): > I think you need to update Bioconductor to 3.6 (the current release)
Kevin Rue-Albrecht (06:10:39): > arf.. that makes sense! i knew it was starting me in the face
Kevin Rue-Albrecht (06:10:45): > thanks!
Kevin Rue-Albrecht (06:12:51): > oh boy.. updating all the packages now:sweat_smile:thanks anyway! I really got used to being on the devel branch all the time that I forgot how to behave as a user
Federico Marini (06:14:38): > users will be users:slightly_smiling_face:
Federico Marini (09:52:04): > I’m not saying we should introduce bugs on purpose
Federico Marini (09:52:17): > but I kinda miss the conversations here on the channel
Kevin Rue-Albrecht (09:52:24): > hahahaha
Kevin Rue-Albrecht (09:53:55): > sorry I was being selfish kept a discussion aboutshinyapps.io on a direct conversation with Charlotte. I’ll be sure to share on the channel next time:stuck_out_tongue_winking_eye:
Charlotte Soneson (09:54:46): > Shall we start thinking about the paper?:upside_down_face:
Charlotte Soneson (09:55:11): > Just to have something to talk about:joy:
Kevin Rue-Albrecht (09:55:54): > Bottomline is that the free plan is likely too restricted to even host theallendemo. I’ve managed to deploy the app through a few hacks, but the app page displays at best an “out of memory” error. Charlotte found out that thediamonddemo data set of R already uses about 100 MB
Federico Marini (09:56:12): > wtf
Kevin Rue-Albrecht (09:56:23): > and the free plan offers at most 1GB of memory
Aaron Lun (09:57:05): > Deep in a dogfight with scater’s plotting functions right now.
Aaron Lun (09:57:11): > Won’ tbe responsive until it’s over
Federico Marini (09:57:14): > the only problem is R-devel - I also have a shiny server I could use for deployment
Federico Marini (09:57:52): > if I enter the sacrilegious territory of downgrading the requirement
Federico Marini (09:58:03): > we could be set
Federico Marini (09:58:19): > it is not the most powerful server, this I should say upfront
Kevin Rue-Albrecht (09:59:56): > Speaking of manuscript, I dug up the manuscript that I optimistically submitted as an application note to Oxford Bioinformatics a few years back, for GOexpress. We ended up publishing a more fleshed out version to BMC bioinformatics. Anyway. I made a few points about data viz and multifactorial experimental designs in there that could be reused to describe applications of iSEE.
Kevin Rue-Albrecht (10:00:11): > Shall I share the word file here as an attachment, or by email?
Charlotte Soneson (10:01:10): > If everyone is fine with writing in a Google Doc, it could be pasted directly there?
Federico Marini (10:01:29): > fine for me
Kevin Rue-Albrecht (10:01:29): > Not a bad idea indeed.
Charlotte Soneson (10:01:33): > Or Overleaf, if we are going LaTeX
Kevin Rue-Albrecht (10:02:28): > Good excuse for me to sign up for Overleaf. I’ve been curious about it
Charlotte Soneson (10:03:16): > I don’t know if it’s possible to see what other people are doing in reasonable real time on Overleaf, but it would be more practical for submission to Bioinformatics at least.
Kevin Rue-Albrecht (10:04:09): > Which reminds me, before I make any rushed assumption: are we decided for an application note (2 pages) to Oxford Bioinformatics?
Kevin Rue-Albrecht (10:04:31): > or do we want to flesh out something longer and/or another journal?
Charlotte Soneson (10:05:30): > I’m fine with anything, but it feels more like a short thing to me, unless we want to become philosophical:slightly_smiling_face:
Kevin Rue-Albrecht (10:06:10): > Yep. I personally don’t see how we could make a Original Research paper out of it, to be honest
Aaron Lun (10:06:26): > I would go with Google docs
Aaron Lun (10:06:32): > overleaf is a real pain in the ass
Aaron Lun (10:06:42): > Either that or a git latex
Charlotte Soneson (10:06:57): > I think git LaTeX would be a bit of a pain too
Aaron Lun (10:07:01): > really?
Aaron Lun (10:07:08): > It’s the best setup I’ve had for technical people.
Charlotte Soneson (10:07:17): > I like that you can see everyone’s cursor in a Google Doc, so you are not working on exactly the same place at the same time
Charlotte Soneson (10:07:35): > git works if one at the time is writing
Aaron Lun (10:07:44): > Well, how about we start drafting in docs and move to git once we’re closer to completion.
Charlotte Soneson (10:07:50): > Sounds good
Aaron Lun (10:07:56): > But in any case, we’re too far off- still need to implement proper tests.
Aaron Lun (10:08:06): > And actually submit to bIOc-devel
Federico Marini (10:08:46): > what we can start doing anyway would be to collect a couple of references - especially of our “concurrence”
Federico Marini (10:09:09): > which as of now mostly consists of commercial suites, anyway:smile:
Kevin Rue-Albrecht (10:10:46): > Well, for what it’s worth I’ll upload my old manuscript to Google Docs, just so that we can all look up the handful of (potentially) reusable ideas/phrases. Don’t make me say that it’s a scaffold for our manuscript, far from it:sweat_smile:Then, we can just start the real manuscript wherever is most convenient for us
Kevin Rue-Albrecht (10:13:59): > here goes:https://drive.google.com/open?id=1MF5FNdWx6QJ9xYBoNywe7_gkGSPUIH3v
Kevin Rue-Albrecht (10:14:45): > @Kevin Rue-Albrechtshared a file:RueAlbrecht_et_al.(Bioinformatics)_2014-10-09_FINAL.doc - File (Word Document): RueAlbrecht_et_al.(Bioinformatics)_2014-10-09_FINAL.doc
Aaron Lun (10:18:46): > Hm… I dunno… first author looks a bit suspicious, if you ask me.:slightly_smiling_face:
Kevin Rue-Albrecht (10:19:00): > maybe that’s why they refused it:wink:
Aaron Lun (10:19:07): > lol
Federico Marini (10:19:17): > dammit Kevin
Federico Marini (10:19:29): > you should keep the surname cut to Rue
Federico Marini (10:19:38): > look how much space it ate up with the extra line
Federico Marini (10:19:57): > HASHTAG APPLICATIONS NOTE RULES
Federico Marini (10:20:04): > :smile:
Kevin Rue-Albrecht (10:20:08): > First author isn’t enough for me, I want the full first line of the author list:stuck_out_tongue_winking_eye:
Federico Marini (10:20:29): > it could have even worked
Kevin Rue-Albrecht (10:20:32): > Wait until I put my middle name in there:stuck_out_tongue:
Federico Marini (10:21:36): > it is not Maximilian or some other longer ones right?
Kevin Rue-Albrecht (10:21:49): > close enough (to the length): Christophe
Kevin Rue-Albrecht (10:23:31): > http://www.historyrundown.com/top-5-people-with-the-longest-names/ Didn’t know Picasso’s full name was ”Pablo Diego José Francisco de Paula Juan Nepomuceno María de los Remedios Cipriano de la Santísima Trinidad Ruiz y Picasso”
Charlotte Soneson (10:24:35): > But he’s not on our paper, right?
Federico Marini (10:24:57): > well we could
Kevin Rue-Albrecht (10:25:06): > well… weare doing (data) visualisation
Charlotte Soneson (10:25:10): > I don’t think our plots are really his style
Federico Marini (10:25:14): > geom_cube
Charlotte Soneson (10:25:25): > Haha yes, maybe those
Federico Marini (10:25:26): > coming up in the next ggplot release
Kevin Rue-Albrecht (10:26:51): > we could always set the default random seed to Picasso’s birth year or something iniSEE ^^
Federico Marini (10:27:59): > for-ty-two! for-ty-two!
Kevin Rue-Albrecht (10:28:43): > “The Hitchhiker’s Guide to Genomics with iSEE”
Kevin Rue-Albrecht (10:30:24): > … could be a good title for the vignette, come to think of it:joy:
Federico Marini (10:33:06): > BTW, trying to test iSEE with some other data, did you already try to plug in the 10x neurons set?
Federico Marini (10:33:10): > at least the subset of it
Kevin Rue-Albrecht (10:34:36): > Not yet. I keep thinking about this density-density downsampling thing that Aaron mentioned before I dare approach the 1.3M data set
Federico Marini (10:35:19): > not that one, probably this would kill many machines
Federico Marini (10:35:45): > I was about to try this one here:
Federico Marini (10:35:45): > https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.3.0/1M_neurons
Federico Marini (10:36:01): > sorry wrong link
Federico Marini (10:36:02): > http://cf.10xgenomics.com/samples/cell-exp/1.3.0/1M_neurons/1M_neurons_neuron20k.h5
Kevin Rue-Albrecht (10:36:38): > ah. i thought there was a misunderstanding when I opened the first link:joy:
Kevin Rue-Albrecht (10:36:45): > ahh the 20k one
Kevin Rue-Albrecht (10:37:05): > yeah, that could be a cool progression from the 8k TCGA data set
Federico Marini (10:39:20): > we go “2x”. AH AH. oooh I need free time
Kevin Rue-Albrecht (10:41:29): > Btw, I just got asked today if iSEE handles microarray data sets. I’m sure that it can, but we should probably document an example. I’ll have a look at theExperimentHub, but suggestions welcome
Charlotte Soneson (10:42:30): > Shouldn’t be a problem I think, there are no count models or anything in there.
Federico Marini (11:28:28): > still not the neurons, still not scaling up. yet, testing it on 10x data
Federico Marini (11:29:05): > and trying to obtain similar stuff tohttp://cf.10xgenomics.com/supp/cell-exp/cellrangerrkit-PBMC-vignette-knitr-2.0.0.pdf
Federico Marini (11:29:26): > > download_sample(sample_name="pbmc3k",sample_dir="10xdata_pbmc3k", > host="[https://s3-us-west-2.amazonaws.com/10x.files/samples/cell/](https://s3-us-west-2.amazonaws.com/10x.files/samples/cell/)") > gbm <- load_cellranger_matrix("10xdata_pbmc3k") > # also compute the logcounts > scepbmc3k <- SingleCellExperiment(list(counts = as.matrix(exprs(gbm)), > logcounts = log2(1 + as.matrix(exprs(gbm))))) > scepbmc3k <- scater::runPCA(scepbmc3k, exprs_values = "counts") > irlba_out <- irlba(scepbmc3k@assays@.xData$data$counts) > tsne_out <- Rtsne(irlba_out$v, pca = FALSE, perplexity = 50, verbose = TRUE) > reducedDim(scepbmc3k, "TSNE") <- tsne_out$Y > colData(scepbmc3k) <- DataFrame(pData(gbm)) > library(org.Hs.eg.db) > genenames <- AnnotationDbi::mapIds(org.Hs.eg.db,keys = rownames(scepbmc3k),column = "SYMBOL",keytype = "ENSEMBL") > genenames <- scater:::uniquifyFeatureNames(ID = rownames(scepbmc3k), names = genenames) > rownames(scepbmc3k) <- genenames > > library("iSEE") > iSEE(scepbmc3k) >
Federico Marini (11:30:09): > this piece of code gets quite close to it (see fig. 2 in their vignette, they put some info on the markers, and I did not have anything for the 20k neurons to check)
Aaron Lun (11:30:40): > scepbmc3k@assays@.xData$data$counts???
Aaron Lun (11:30:51): > becausecounts(scepbmc3k)is too simple, obviously.
Federico Marini (11:30:58): > yop:stuck_out_tongue:
Federico Marini (11:31:20): > they had a complicated class structure
Federico Marini (11:31:38): > I had to get it like this in a previous version. sorry, did not change that line
Federico Marini (11:32:38): > if you see anything to tweak please let me know:wink:
Federico Marini (11:33:28): > for example, naive question: is t-SNE to be applied on the irlba-processed version of the raw counts?
Aaron Lun (11:33:46): > no.
Aaron Lun (11:34:08): > You’ll need to normalize, log-transform, select HVGs and transpose before feeding it into irlba
Aaron Lun (11:34:26): > Pretty sure also that V aren’t the PCs, you have to scale them by D or sqrt(D) (can’t remember which).
Federico Marini (11:35:36): > thanks. I asked because we had a similar chunk inhttps://github.com/csoneson/iSEE/blob/master/vignettes/iSEE_vignette.Rmd#use-case-iii-using-the-experimenthub - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Federico Marini (11:36:08): > and I am myself not so navigated in the preprocessing of these types of sets
Aaron Lun (11:36:51): > We can skip a lot of the stuff if we just want an example of how it works, though log-transforming and transposing are necessary.
Federico Marini (11:37:57): > Ok, _in my case I just wanted to see if the core findings from their vignette are also doable via iSEE
Kevin Rue-Albrecht (11:38:54): > eh.. that chunk’s my fault (I’ll drink to that tonight :P), once I got the irlba recommendation from Aaron, I didn’t bother to ask how to use it properly ^^
Federico Marini (11:39:08): > do you have a link/vignette for the documentation of the steps you mentioned? (asks while googling for the single cell workflow… )
Aaron Lun (11:40:00): > Yes,https://bioconductor.org/help/workflows/simpleSingleCell/intro
Aaron Lun (11:40:30): > Note thathttps://bioconductor.org/help/workflows/simpleSingleCell is an old version, I’ve asked them to re-direct if possible.
Federico Marini (11:40:59): > excellent, thank you + good to know
Federico Marini (11:57:38): > wow these are quite small bibles, thanks again:wink:
Charlotte Soneson (15:09:30): > Apparently Winston Chang was talking abouthttps://rstudio.github.io/shinytest/articles/shinytest.html at RStudioConf today. I guess the talk was streamed (https://www.youtube.com/watch?v=ogy7rHWlsQ8 ), might be useful to have a look if it is available afterwards - Attachment (YouTube): rstudio::conf 2018 | Official Live Stream
Aaron Lun (17:26:47): > shinytest looks pretty neat, though it’s not on CRAN, which makes it difficult to use for BioC testing.
Aaron Lun (17:27:20): > It also relies on a save state of expected results; for the number of tests we would need to do, that’s going to be pretty large.
Aaron Lun (17:27:45): > I would suggest testing all the internal functions viatestthat, and perhaps having a separate repo for shinytest if we need it.
Kevin Rue-Albrecht (17:28:57): > btw, anyone knows whethersetClasscan be tested? InTVTBI ended up marking them#nocovbecause they I just couldn’t figure out
Aaron Lun (17:29:31): > Are you trying to get 100% code cov?
Kevin Rue-Albrecht (17:29:36): > i suppose they’re run on package load/attach, but never at runtime
Aaron Lun (17:29:46): > We won’t get that anyway, the observers won’t trigger unless the app is launched.
Kevin Rue-Albrecht (17:30:29): > (in TVTB I hit 100%, not that I’m particularly proud of it, in retrospect)
Kevin Rue-Albrecht (17:30:55): > plus, I figured that covering all the lines doesn’t mean they’re covered in all scenarios anyway
Kevin Rue-Albrecht (17:31:45): > (a line that works with a positive value won’t necessarily work withNULL)
2018-02-03
Kevin Rue-Albrecht (07:07:07): > Are we aware of any outstanding bug fix or desirable feature that needs implementing this weekend? > I’m going out for a bit, but I’ll have a bit of time when I get back. > I think we’ve been mentioned things on the channel that aren’t listed as issues on GitHub, but I can’t remember. > - Among the open GitHub issues, there’s Davis’ “info on hover” that I’d like to see. I could get on that. > - I’m also thinking of a little reactivetextOutputthat would displayn of N (x%) data points brushed
Aaron Lun (07:07:38): > I think we need a feature freeze and testing of everything we have now.
Aaron Lun (07:07:50): > But if you are going to add new features, I would suggest thinking about heatmaps.
Aaron Lun (07:08:10): > I have some ideas but probably best to skype about it if you’re interested.
Aaron Lun (07:08:30): > There’s some interesting opportunities for linking to other plots, but it’s not as simple as one might think.
Kevin Rue-Albrecht (07:09:08): > I agree that a new Skype would be good at this point, to reflect on the alpha release and prioritise where to go next
Aaron Lun (07:10:05): > But extending the tests is my first iSEE-related priority for the upcoming week.
Aaron Lun (07:10:25): > I don’t suppose you could give shinytest a run and see if it’s any good?
Aaron Lun (07:10:39): > Anything we do with it would need to be in a separate repo though;
Aaron Lun (07:10:49): > the saved states look like they’d be pretty heavey.
Aaron Lun (07:11:53): > Okay, going to get some food, back in a bit.
Kevin Rue-Albrecht (07:12:07): > I could look at shinytest indeed. Although I should probably makes sure I cover as much of theAllClasses.Ras possible. > Which reminds me, I’ll see if I can sort out integration withcodecov.io to track coverage better
Kevin Rue-Albrecht (07:13:22): > For those who may not have used it, it reports coverage on a line-per-line basis.
Kevin Rue-Albrecht (07:17:32) (in thread): > @Charlotte Sonesoni think only the repo owner can set up the webhook between GitHub andcodecov.io
Kevin Rue-Albrecht (07:17:50) (in thread): > https://github.com/codecov/example-r - Attachment (GitHub): codecov/example-r > Contribute to example-r development by creating an account on GitHub.
Kevin Rue-Albrecht (07:18:27) (in thread): > something about a token to add in the settings
Charlotte Soneson (07:18:49) (in thread): > Ok. I’m just on my way out to get some food now as well (before the shops close here), but I’ll look into this when I get back.
Kevin Rue-Albrecht (07:19:20) (in thread): > no worries, we can catch up later!
Kevin Rue-Albrecht (08:21:28): > Seems like codecov integration worked:https://codecov.io/gh/csoneson/iSEE/tree/5d0bf6f4e44f7d2ac108cdb6d5fa6ddd13aa046e/R - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Charlotte Soneson (08:58:30) (in thread): > Does that mean I don’t have to do anything else, or does it still need some activation or something?
Kevin Rue-Albrecht (09:02:18) (in thread): > Apparently nothing else. I hadn’t set anything up in a while on codecov so I was a bit confused when it gave me a “token”
Charlotte Soneson (09:02:54) (in thread): > Ok:+1:
Aaron Lun (10:05:23): > FYI, finished 50% of the scater refactoring. It is… brutal.
Kevin Rue-Albrecht (10:10:51): > hahaha, out of curiosity i had a look at the latest commits, how new is theaesthargument toplotExpressionDefault? I don’t remember seeing it before, but again, I like doing my own ggplots
Aaron Lun (10:11:03): > Well, it’s going to be gone.
Kevin Rue-Albrecht (10:11:07): > lol
Aaron Lun (10:11:12): > Precisely because people can do their own damn ggplots
Aaron Lun (10:11:31): > and it was too much effort to expose all of the scater internals in a coherent way.
Kevin Rue-Albrecht (10:15:00): > speaking of which, I was half-tempted to expose theisColorMapCompatiblefunction that I’ve written as a check before launching theiSEEapp. Users may want to check compatibility outside the app (precisely if they want to use the ECM for their own plots). > However, 1) the function is still pretty raw and would need some cleanup, 2) it should also probably be given a better name and turned into a generic that could check againstExpressionSetfor instance
Kevin Rue-Albrecht (10:26:16): > Alright. codecov badge added. Let’s get that 6% up:slightly_smiling_face:
Kevin Rue-Albrecht (11:02:20): > i’ve come across this little gem, if anyone still has all the remote-tracking branches that have come and gone in the last few weeks:git fetch --prune
Aaron Lun (11:03:26): > sweet
Kevin Rue-Albrecht (11:03:53): > it’ll remove local references to all the branches that don’t exist on GitHub anymore
Kevin Rue-Albrecht (11:22:28): > one note about writing tests, that I just discovered myself:?test_dir > > There are four classes of .R files that have special behaviour: > - Test files start with "test" and are executed in alphabetical order. > - Helper files start with "helper" and are executed before tests are run and from devtools::load_all(). > - Setup files start with "setup" and are executed before tests, but not during devtools::load_all(). > - Teardown files start with "teardown" and are executed after the tests are run. > > I never paid attention to the last 3. We should probably move the definition of test values/objects in “setup_“, to avoid redefining them in individual “test_” scripts
Kevin Rue-Albrecht (11:23:15): > I’m thinking about things like loading or preprocessing anrdsfile
Aaron Lun (11:25:24): > Uh… why would we need to process an RDS file?
Aaron Lun (11:26:00): > It would be helpful to only have to constructsceonce, though.
Aaron Lun (11:26:12): > But a lot of the tests intest_thatwill be far below that level
Aaron Lun (11:26:25): > much more low-level, I would supect.
Kevin Rue-Albrecht (11:27:30): > it should be much lower level, trying to feed as much pre-digested arguments to the low-level functions as possible. > but if I remember correctly some things like the default-generating functions want anSCEto do their job
Aaron Lun (11:28:02): > Yes.
Kevin Rue-Albrecht (11:28:14): > not much preprocessing needed, but as you wrote, we’d rather prepare a single testSCEonly once, and share it across tests potentially distributed across multiple"test_"files
Aaron Lun (11:29:38): > what does devtools::load_all do?
Kevin Rue-Albrecht (11:29:48): > no clue
Aaron Lun (11:30:01): > Because it’s a pretty subtle distinction betwen ehlper and setup files.
Kevin Rue-Albrecht (11:30:01): > (yet)
Kevin Rue-Albrecht (11:31:57): > shameless copy-paste rom the doc: > “load_all loads a package. It roughly simulates what happens when a package is installed and loaded with library”
Aaron Lun (11:32:31): > Hm.
Kevin Rue-Albrecht (11:32:48): > I supposehelpercan rundata(allen)whilesetupcannot
Kevin Rue-Albrecht (11:45:34): > insce <- as(allen, "SingleCellExperiment"), where is theasmethod defined? > > > showMethods("as") > Function "as": > <not an S4 generic function> >
Kevin Rue-Albrecht (11:46:32): > i don’t get why the command only works oncescateris loaded, ifshowMethods("as")returns the above even afterscateris loaded
Aaron Lun (11:49:53): > Well, it should be in methods, right?
Aaron Lun (11:50:00): > SCE is a subclass of SEs, so it’s auto-defined.
Aaron Lun (11:50:05): > No need forsetAs.
Aaron Lun (11:50:09): > I think.
Aaron Lun (11:50:19): > It’s possible that we wrote one, but I don’t remember writing it.
Kevin Rue-Albrecht (11:50:31): > ahh.. autodefined for child classes.. makes sense
Kevin Rue-Albrecht (11:51:37): > nah, I guess you made the right point. I was just used to see something likeinherited from <parent class>, but I guess that only applies if the parent classdoes have a customsetAsdefinition
Kevin Rue-Albrecht (11:52:19): > in this case, it’s probably just the point of changing the class name, and adding a defaultreduceDimslot
Aaron Lun (11:55:37): > God, Davis writes his code with no defence at all.
Kevin Rue-Albrecht (11:55:53): > as in sanity checks?
Aaron Lun (11:55:56): > all hard-coded variables.
Aaron Lun (11:56:07): > none of which work now, because we changed stuff upstream.
Kevin Rue-Albrecht (11:56:43): > ah yeah. I kept meaning to fix the explanatory variable plot that was hard coded to qc fields*_controlswhilecalculateQCmetricsgenerated*_control
Kevin Rue-Albrecht (11:56:52): > drove our PhD student nuts for a day
Aaron Lun (11:57:03): > lol
Kevin Rue-Albrecht (12:50:47) (in thread): > up to 7.14% (100% of a file covered):tada:
Aaron Lun (12:55:42): > Feel free to put in a PR, I need all the help I can get on this.
Kevin Rue-Albrecht (12:56:17): > well, just looking back at it now, it seemed to be fixed
Aaron Lun (12:56:22): > Once I finish merging the ~50 commits inplot_refactor.
Kevin Rue-Albrecht (12:57:08): > but yeah, your earlier message seemed to indicate that things are going to move quite a bit, so i’d rather wait that you finish doing your magic before looking where I can help
Aaron Lun (12:57:49): > I mean, I had to clear out.Rhistoryfiles that somehow got committed.
Kevin Rue-Albrecht (12:58:23): > i’m kinda doing helping right now: re-compiling a full Rmarkdown website taking SmartSeq2 data from raw counts to manuscript figures using scran, scater and co.
Aaron Lun (12:59:14): > Down to 2 warnings in CHECK…
Kevin Rue-Albrecht (12:59:18): > (since I’m too lazy to switch back to Bioc-release, I’ve been doing this analysis using the bioc-devel version of the packages)
Kevin Rue-Albrecht (12:59:47): > for better and worse, considering I started it before the SCESet-SingleCellExperiment transition
Aaron Lun (13:06:48): > Down to 1 warning…
Kevin Rue-Albrecht (13:15:21): > alright, I’ve collected a bit of information from the Oxford Bioinformatics website here:https://docs.google.com/document/d/1WaiqAeYYg9bv5JK-lAGr8KhU3VezUKV7OWnr70wHG0c/edit?usp=sharing
Kevin Rue-Albrecht (13:15:26): > @Kevin Rue-Albrechtshared a file:iSEE to Oxford Bioinformatics - File (Google Docs): iSEE to Oxford Bioinformatics
Kevin Rue-Albrecht (13:15:55) (in thread): > mostly taken from:https://academic.oup.com/bioinformatics/pages/instructions_for_authors
Kevin Rue-Albrecht (13:16:43) (in thread): > We can add to it as we go along. For the time being I’m diving back into unit tests
Aaron Lun (13:20:14) (in thread): > What bit are you testing now?
Kevin Rue-Albrecht (13:20:48) (in thread): > I was about to look how to testplotting.R, to stick to my field of expertise for the time being
Kevin Rue-Albrecht (13:21:03) (in thread): > but I could also look into this shinytest thing
Aaron Lun (13:21:31) (in thread): > No, that’s good.
Aaron Lun (13:21:37) (in thread): > I was going to ask you to test plotting anyway.
Kevin Rue-Albrecht (13:21:55) (in thread): > I think that makes more sense for now.
Aaron Lun (13:22:10) (in thread): > It would be sufficient to test that the plots are generated without error with all possible specifications; I don’t see a decent way of callingexpect_equalon ggplot objects.
Aaron Lun (13:23:23) (in thread): > Though one way of doing it would be to evaluate the reported commands, to confirm that we get the same ggplot object…
Kevin Rue-Albrecht (13:24:19) (in thread): > That is true as well.
Kevin Rue-Albrecht (13:24:48) (in thread): > Makes me think that if we manage to set up shinytest, it will implicitly test plotting.R
Aaron Lun (13:24:58): > NO WARNINGS, NO ERRORS, SCATER PASSES CHECK!
Aaron Lun (13:25:00): > :sob:
Aaron Lun (13:25:07): > After 20 hours of working on this.
Kevin Rue-Albrecht (13:25:09) (in thread): > still, i think it’s better to testplotting.Routside of the app
Aaron Lun (13:25:36) (in thread): > Yes, we might as well.
Kevin Rue-Albrecht (13:25:37) (in thread): > could disambiguate where issues come from
Kevin Rue-Albrecht (13:25:55) (in thread): > (if it’s a runtime thing, or the functions themselves)
Kevin Rue-Albrecht (13:26:56): > it’s up to you, but that’s the kind of circumstances when i’d crack a beer open ^^
Aaron Lun (13:27:33): > Still need to flesh out the tests tomorrow, but I’m done with that for the day.
Aaron Lun (13:27:44): > Now, to the toilet, and onto a paper I need to review.
Kevin Rue-Albrecht (13:28:00): > bad omen for the paper:joy:
Aaron Lun (13:32:38) (in thread): > Keep me in the loop for how you’re going to set up yourexpect_equalcalls; because the central plotting returns theplot.dataand the commands, there’s a number of things we can check for sanity.
Aaron Lun (13:34:30) (in thread): > I will test the table and brush link updates function, but I might not get to it soon, as I forgot about another project and have to deal with it next week.
Kevin Rue-Albrecht (13:34:48) (in thread): > yeah.. i already hit a wall when I set up the first checks: seems like there is a double-standard between RStudio and a terminal session when testing functions for equality
Kevin Rue-Albrecht (13:36:55) (in thread): > apparentlyexpect_identicalwas happy in RStudio but failed in the terminal and CMD check for a weird reason, trying to reproduce it now
Kevin Rue-Albrecht (14:06:12): > just having a fight with roxygen… which seems to parse files in thetestsdirectory… and somehow complains thatobject 'allen' not found
Kevin Rue-Albrecht (14:07:34): > (after a call todata(allen), obviously, or I wouldn’t report it
Kevin Rue-Albrecht (14:08:55): > i mean, the package builds and works, butR CMD checkandroxygen2::documentfail because of that error
Aaron Lun (14:09:16): > ¯*(ツ)*/¯
Aaron Lun (14:09:34): > Why would CHECK fail?
Kevin Rue-Albrecht (14:09:37): > it’s on thekevin_testsbranch already, if anyone has time to waste on this
Kevin Rue-Albrecht (14:10:13) (in thread): > this one is leaving me even more speechless: > > * checking for file 'iSEE/DESCRIPTION' ... ERROR > Required field missing or empty: > 'Author' >
Kevin Rue-Albrecht (14:11:08) (in thread): > while i can guarantee you that DESCRIPTION hasn’t changed a symbol since master
Aaron Lun (14:12:12) (in thread): > lol
Kevin Rue-Albrecht (14:17:22): > ok… so i just checked outmasterand ranroxygen: no problem > checked outkevin_testsand didroxygen: above error (object 'allen' not found)
Aaron Lun (14:17:51): > Doesn’t likesetuporhelper, I would say.
Aaron Lun (14:17:55): > Try using the other one.
Aaron Lun (14:18:01): > As in,setup.
Kevin Rue-Albrecht (14:18:54): > well.. I’ve been usinghelperonly so far, as it’s supposed to be run in an environment whereiSEEis loaded
Kevin Rue-Albrecht (14:19:01): > but i’ll play around report back
Kevin Rue-Albrecht (14:22:06): > ….:expressionless:…
Kevin Rue-Albrecht (14:22:26): > its frustrating when you’re right all the time, you know that?:stuck_out_tongue_winking_eye:
Aaron Lun (14:22:44): > Not just a pretty face.
Kevin Rue-Albrecht (14:22:46): > changed it tosetupand it worked
Kevin Rue-Albrecht (14:23:21): > point being that I didn’t neediSEEloaded to turnalleninto anSCE
Kevin Rue-Albrecht (14:27:14): > and thatdevtools::documentparses thehelper*files, thereby expecting objects present in those files to be defined later in places where they may not be
Federico Marini (14:44:59) (in thread): > I do know
Federico Marini (14:45:15) (in thread): > we defined the authors with the authors@R
Kevin Rue-Albrecht (14:45:21) (in thread): > don’t worry about this
Federico Marini (14:45:26) (in thread): > and that is not liked by R cmd check
Federico Marini (14:46:04) (in thread): > this is general for any other package
Federico Marini (14:46:20) (in thread): > sometimes I tried to do the checking outside RStudio but end up having to rely on it
Federico Marini (14:46:28) (in thread): > or at least on devtools::check
Aaron Lun (14:46:29) (in thread): > Should be fine after BUILD.
Federico Marini (14:46:37) (in thread): > most likely, yes
Federico Marini (14:47:09): > One thing that could be relevant for us
Federico Marini (14:47:27): > what conferences are you going to attend this year?
Federico Marini (14:47:57): > and related to this: are we fine in presenting iSEE to bigger audiences? (I think so)
Aaron Lun (14:53:17): > Happy with you guys presenting this, I probably won’t though.
Aaron Lun (14:53:37): > because I’m a conference party maniac
Kevin Rue-Albrecht (14:53:56) (in thread): > anyone coming to the Oxford Single-Cell Symposium this year? I think in May
Kevin Rue-Albrecht (14:54:20) (in thread): > last year Rahul Satija was keynote
Aaron Lun (14:55:53): > and will probably be too wasted to do a good job
Kevin Rue-Albrecht (14:56:26) (in thread): > Oxford Single Cell Symposium 2018 > will be held on Monday, 23 April 2018 at the Saïd Business School, Oxford > Speakers so far confirmed are: > - Wolf Reik, Babraham Institute > - Ido Amit, Weizmann Institute of Science
Kevin Rue-Albrecht (14:57:49) (in thread): > https://docs.google.com/forms/d/e/1FAIpQLSfKCPQHtTGGWI0593Ou27KjiJ6TXAnF0_67M6c36-jFAiA0-Q/viewform
Aaron Lun (15:09:42) (in thread): > Ugh… will to live –> 0.
Aaron Lun (15:11:24) (in thread): > I can’t remember a time when I was not reviewing this paper.
Aaron Lun (15:11:45) (in thread): > I bet everyone is having dinner, right?
Aaron Lun (15:11:47) (in thread): > Screw you guys.
Federico Marini (15:12:09): > :D:D
Federico Marini (15:12:34) (in thread): > i probably will not be able to attend
Federico Marini (15:12:50): > I was aiming at ERUM (may)
Federico Marini (15:13:11): > plus some more german-based conferences
Federico Marini (15:13:27): > where I wanted to propose a tutorial session on rnaseq
Charlotte Soneson (15:14:49) (in thread): > me neither I think
Aaron Lun (15:17:46): > knock yourself out.
Kevin Rue-Albrecht (15:20:12): > ERUM: “Barbara Borges Ribeiro is a software engineer at RStudio working primarily in the Shiny package.”
Aaron Lun (15:20:21): > Sweet
Aaron Lun (15:20:34): > dunno what we can ask for though.
Aaron Lun (15:20:48): > Oh, the ability to set brushes in advance.
Federico Marini (15:20:49): > named dropdown!!!!
Federico Marini (15:21:41): > https://github.com/rstudio/shinydashboard/issues/208 - Attachment (GitHub): Make dropdownMenu a Shiny input if an id is provided · Issue #208 · rstudio/shinydashboard > In an ideal world, here’s everything I’d like to implement: # ————————————————————- # # ————————– MOCK UP ————————– # # —…
Aaron Lun (15:23:17): > Oh yeah…
Kevin Rue-Albrecht (15:25:05): > Gábor Csárdi has been writing R tools for more than 10 years, and is the co-author of theigraphR package
Federico Marini (15:25:58): > BETTER DEFAULTS for the plotting func?:smile:
Kevin Rue-Albrecht (15:26:28): > anything about the d3.js thing that allows users to grab and shake the graph? ^^
Federico Marini (15:26:33): > i think he put the development of igraph pretty much on ice
Kevin Rue-Albrecht (15:26:57): > https://bl.ocks.org/mbostock/3750558 - Attachment (bl.ocks.org): Sticky Force Layout > Mike Bostock’s Block 3750558
Federico Marini (15:26:57): > as a perfect metaphor to data torturing
Kevin Rue-Albrecht (15:27:43): > sweet, when you double click, they return ‘to place’
Federico Marini (15:29:03): > there are some html widgets for that
Federico Marini (15:29:26): > I wanted to not add another dependency..
Kevin Rue-Albrecht (15:30:04): > good call. let’s keep it simple unless there’s some demand or we have extra time on our hands
Aaron Lun (15:42:16): > Review down.
Aaron Lun (15:42:19): > Going home.
Kevin Rue-Albrecht (16:41:58): > just thinking we could probably scratch some runtime memory by storing only X and Y coordinates inpObjects$coordinateshttps://github.com/csoneson/iSEE/blob/master/R/iSEE.R#L729 pObjects$coordinates[[plot_name]] <- p.out$xyI thinkp.out$xycontains all the plot data (including color, etc) > We shouldn’t have an issue stripping it down to XY, would we? - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
2018-02-04
Aaron Lun (06:46:09): > That is true, but it’s probably not that much.
Kevin Rue-Albrecht (06:46:43): > No worries. I’ve done it already on my local branch, but let me know and I can revert it back
Kevin Rue-Albrecht (06:47:25): > btw, coverage is going up nicely, i’m just about to cross the 30% mark
Aaron Lun (06:47:57): > Looking at your tests; are you checking for sane$xy$Xand$xy$Y?
Kevin Rue-Albrecht (06:48:03): > you can see the current status here:https://codecov.io/gh/csoneson/iSEE/tree/58369f2ee08e61b894048bc71eccd7b4ae9e7c62/R - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Aaron Lun (06:48:05): > e.g., that they’re numeric or factors.
Kevin Rue-Albrecht (06:48:08): > not yet
Kevin Rue-Albrecht (06:48:20): > it’s easy enough to add extra tests
Aaron Lun (06:48:29): > Nice thing about returning the Colourby and the brushbY is that you can easily check them as well.
Kevin Rue-Albrecht (06:48:44): > and they’re lightning fast (the tests), considering that there’s not much evaluated
Kevin Rue-Albrecht (06:49:23): > so, all the more reasons to add as many checks as you like in there
Kevin Rue-Albrecht (06:50:00): > i’m aware we’ll have to do some refactoring in thehelperandsetupscripts for clarity
Kevin Rue-Albrecht (06:51:20): > i’m trying to roughly split things logically, but I already had to fiddle with thesetupscript names to make suresetup_scegets run beforesetup_plotting(alphabetical issue)
Aaron Lun (06:51:43): > Y’know, perhaps there’s no need to be too fancy with this.
Kevin Rue-Albrecht (06:51:53): > ay ay capt’n
Aaron Lun (06:51:58): > Even if we copy-pasted, it would be <10 lines.
Aaron Lun (06:52:05): > at the start of every script… probably fine.
Aaron Lun (06:52:16): > I do more copy-pasting for my manuscripts.
Kevin Rue-Albrecht (06:52:56): > there we go, just crossed the 30%https://codecov.io/gh/csoneson/iSEE/tree/kevin_tests/R - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Kevin Rue-Albrecht (06:59:00): > what’s your plan t’day, wanna add some of your own tests? I’m still juggling with that markdown website that i’m updating/recompiling for a project
Aaron Lun (07:04:36): > Got to deal with scater today. Again.
Kevin Rue-Albrecht (07:04:47): > lol
Aaron Lun (07:04:48): > Actually adding some proper tests for scater for a change.
Kevin Rue-Albrecht (07:05:32): > I’m actually just facing a frustrating thing with scater right now: the tSNE plot moved around despite using the same seed
Kevin Rue-Albrecht (07:05:56): > i’m just checking whether input genes may be ordered differently or something silly
Kevin Rue-Albrecht (07:06:31): > you know how wetlab collaborators are when they points move around from one pipeline run to the other
Aaron Lun (07:07:11): > … sometimes Rtsne updates, and there’s nothing we can do.
Kevin Rue-Albrecht (07:07:29): > yup could well be haven’t checked that side of the story yet
Kevin Rue-Albrecht (07:07:45): > i’m on scater 1.7.4 btw
Aaron Lun (07:07:55): > that, at least, shouldn’t have changed.
Kevin Rue-Albrecht (07:08:50): > I would have been surprised indeed. Seems like the input is identical anyway, row and column order
Aaron Lun (10:17:06): > Hazzah! Scater plot tests are partially complete.
Aaron Lun (10:21:16): > Looking at your plot tests - checking for matching commands might be a path to suffering.
Aaron Lun (10:21:44): > Especially if we decide to e.g., switch from[, "BLAH"]to[["BLAH"]], or other minor things that would change the matched string but not change the outcome.
Kevin Rue-Albrecht (10:22:00): > i’m still a bit inconsistent in my tests, tbh
Kevin Rue-Albrecht (10:22:41): > but I’m mainly checking only for a partial match, e.g. making sure that the covariate selected for ColorBy is indeed present incmd$color
Aaron Lun (10:22:52): > It seems safer to check forexpect_identicalinp.out$xy.
Kevin Rue-Albrecht (10:23:07): > there are so many things that can be tested ^^
Aaron Lun (10:23:08): > which would give the same thing, but allow us to not worry about the specific commands
Aaron Lun (10:23:27): > Especially if we have to reorder some of the commands.
Kevin Rue-Albrecht (10:23:34): > expect_identical? hmm I think i see what you mean
Aaron Lun (10:23:40): > or split them up in ways other thandata,brush,lim, etc.
Kevin Rue-Albrecht (10:23:50): > yeah, I’m not a big fan of myexpect_namedeither
Kevin Rue-Albrecht (10:26:06): > at least I’m putting in place the tests to cover the package, we can refine the actual expectations in rounds of refactors
Kevin Rue-Albrecht (10:26:33): > but these are all valid point, obviously
Aaron Lun (10:27:12): > Sure. I just thought I’d spare you from writing all those backslashes.
Kevin Rue-Albrecht (10:28:13): > omg.. those backslashes killed me:testthatthrew up a bunch of errors when I initially didn’t write them
Aaron Lun (10:28:36): > you probably need to setfixed=TRUEto avoid regexes.
Kevin Rue-Albrecht (10:29:27): > expect_match(... fixed = FALSE, ..., ...):sob:
Aaron Lun (10:31:13): > Anyway, the reason I suggest end-point testing ofxyis that the handling of nested dataframes will probably alter the commands rather dramatically.
Kevin Rue-Albrecht (10:33:06): > True. Still, even though commands may change format in the future, probably wouldn’t hurt to check that they always match/contain a certain pattern/information/structure.
Kevin Rue-Albrecht (10:33:48): > even if that means changing the unit tests whenever we dramatically change the behaviour of the package
Aaron Lun (10:34:00): > Well, it’s your call.
Kevin Rue-Albrecht (10:34:44): > i’ll leave it for now, as long as it works and doesn’t cause too much pain
Kevin Rue-Albrecht (10:35:56): > we can always ‘deprecate’ the tests by commenting them out temporarily, and deleting them altogether if we can’t restore them in a reasonable amount of time
Aaron Lun (10:36:10): > Okay. ¯*(ツ)*/¯
Kevin Rue-Albrecht (10:36:12): > (say a week, to draw a line)
Kevin Rue-Albrecht (10:37:48): > … do you have an example of nested dataframe in mind? i’m still trying to picture one
Aaron Lun (10:38:15): > see the latestcalculateQCMetricswithcompact=TRUE
Aaron Lun (10:38:30): > This stops the function from vomiting out everything intocolDataandrowData.
Aaron Lun (10:38:45): > My recent updates toscater allow the plotting functions to handle nested metadata.
Kevin Rue-Albrecht (10:39:07): > riiight, I saw the man page ofcalculateQCMetricsearlier today
2018-02-05
Kevin Rue-Albrecht (03:35:48): > Hi all. I left the unit tests last with a mystery unsolved. The last two tests (commented out) raise weird errors about functions and objects not found, that don’t really make sense
Kevin Rue-Albrecht (03:36:58): > Meanwhile, I think some refactoring of the setup and helper scripts is needed/may even help the above
Kevin Rue-Albrecht (03:37:49): > It’s week days again so I’ll be slower. If anyone wants to look at the kevin_tests branch, I probably won’t touch it today
Kevin Rue-Albrecht (03:39:27): > The problematic tests might also be due to a bunch of packages I updated yesterday, although I suspect it’s rather me doing something stupid
Kevin Rue-Albrecht (05:13:13): > FYI:https://www.rstudio.com/resources/cheatsheets/ - ggplot:https://github.com/rstudio/cheatsheets/raw/master/data-visualization-2.1.pdf - Shiny:https://github.com/rstudio/cheatsheets/raw/master/shiny.pdf - package dev (including unit tests):https://github.com/rstudio/cheatsheets/raw/master/package-development.pdf - R markdown(1):https://github.com/rstudio/cheatsheets/raw/master/rmarkdown-2.0.pdf - R markdown(2):https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf - Attachment (RStudio): Cheatsheets > RStudio Cheat Sheets The cheat sheets below make it easy to learn about and use some of our favorite packages. From time to time, we will add new cheat sheets to the gallery. If you’d like us to drop you an email when we do, let us
Aaron Lun (07:43:39): > Still fighting through scater.
Aaron Lun (07:55:08): > God
Aaron Lun (07:55:12): > It hurts so much
Kevin Rue-Albrecht (07:57:29): > The ungrateful job of refactoring… it’ll pay off though. Courage
Kevin Rue-Albrecht (08:00:10): > Btw, if no one gets to it first, I’ll reduce the setup and helper scripts of unit tests to one of each, including only minimal bits and pieces that can then be called and assembled into test objects local to the individual test scripts.
Kevin Rue-Albrecht (08:01:02): > I think I’ve initialised too many test objects in those setup/helper scripts
Kevin Rue-Albrecht (08:01:26): > Because then I’m editing them (I think locally) within individual tests.
Kevin Rue-Albrecht (08:02:11): > But that is confusing even for me, and if I’m wrong, it may actually cause some weird interactions between tests
Aaron Lun (08:02:46): > I still think you’ve overengineered it.
Kevin Rue-Albrecht (08:02:54): > The only thing I’d like to avoid is to ‘data(allen)’ at the start of every test script
Aaron Lun (08:02:57): > A small amount of C&P is fine.
Kevin Rue-Albrecht (08:03:09): > It would cost a few seconds each time
Aaron Lun (08:03:11): > meh
Kevin Rue-Albrecht (08:03:29): > C&P?
Kevin Rue-Albrecht (08:03:57): > Create and preprocess?
Aaron Lun (08:12:24): > Copy and pasting
Aaron Lun (08:12:50): > I mean, you can constructdata(allen)insetup,
Aaron Lun (08:13:05): > and also do the TSNE and stuff there;
Aaron Lun (08:13:12): > and just make a copy in each test script.
Aaron Lun (08:13:23): > This should protect against any interactions when the sce gets modified.
Kevin Rue-Albrecht (08:16:17): > Yup. I’ll look at it this tonight then
Aaron Lun (08:20:39): > I mean, true chaos is something like the old SCESet, when Davis (for reasons unbeknownst to me) turned off the locked environment; this meant that you could copy the object, modify the copy, and the original would also be modified.
Charlotte Soneson (10:22:41) (in thread): > Is it as simple as changingexpect_that(totest_that(? Or is there a reason you are usingexpect_that(? That’s the function that’s looking for theconditionargument.
Charlotte Soneson (10:39:43) (in thread): > And should theextra_cmdintest_that(".coerce_to_numeric handles gene text input", {belab_out?
Charlotte Soneson (10:45:21) (in thread): > with those changes, the tests seem to run for me:slightly_smiling_face:
Kevin Rue-Albrecht (10:52:24) (in thread): > oh no ..goes to hang himself
Kevin Rue-Albrecht (10:53:07) (in thread): > damn, I always pause a second to decide which ofexpect_thatortest_thatI need to type
Kevin Rue-Albrecht (10:53:52) (in thread): > it’s like reading version 472 of a manuscript: I couldn’t spot the typos anymore
Kevin Rue-Albrecht (10:55:04) (in thread): > I trackedextra_cmdto the code of the function that I was testing, but it doesn’t matter anymore: it theexpect_that/test_thatconfusion that’s causing all these particular problems
Kevin Rue-Albrecht (10:55:13) (in thread): > thanks
Charlotte Soneson (10:57:35) (in thread): > :+1:Better when things are easy to solve than when it’s some super complicated thing:slightly_smiling_face:
Kevin Rue-Albrecht (10:58:27) (in thread): > Definitely
Kevin Rue-Albrecht (10:59:17) (in thread): > Will be even easier when I refactor/simplify the setup for the unit tests
Aaron Lun (12:22:31): > My scater refactoring is done for the week!
Aaron Lun (12:36:34): > Still need to tacklefindImportantPCsandplotExplanatoryVariables.
Aaron Lun (12:36:56): > Now, onto this thing.
Kevin Rue-Albrecht (12:37:00): > i have fond memories of the latter ^^
Kevin Rue-Albrecht (12:37:46): > speaking of Davis, what’s the status about his Docker PR ?
Aaron Lun (12:37:52): > Dunno.
Kevin Rue-Albrecht (12:37:57): > I’m no expert at all in that regard
Aaron Lun (12:38:11): > I say we ignore it until we’re told to merge it.
Kevin Rue-Albrecht (12:38:50): > current strategy it is then:innocent:
Aaron Lun (12:41:22): > What’s the deal with those two tests at the end of plotting?
Aaron Lun (12:41:33): > what error are you getting?
Kevin Rue-Albrecht (12:41:37): > Charlotte figure it in a thread
Kevin Rue-Albrecht (12:41:53): > it was just a stupid error out of tiredness
Kevin Rue-Albrecht (12:42:03): > I confusedexpect_that/test_that
Kevin Rue-Albrecht (12:43:41): > The former is actually a deprecated function that I’ve never used (except for this ‘brain-fart’). I’d have preferred that they remove it altogether. Autocompletion is a poisoned gift.
Aaron Lun (14:15:55): > Added tests for brushing links.
Aaron Lun (14:16:18): > This involved a few corner-case bug fixes.
Aaron Lun (14:16:23): > Will deal with the tables tomorrow.
Aaron Lun (14:23:50): > ggplot team, that dual colour bar problem is back again.
Aaron Lun (14:25:37): > brushing for horizontal violin plots is also fucked up
Aaron Lun (14:29:04): > Oh wait, was on an ancient branch
Aaron Lun (14:29:15): > kevin_fixesrather thankevin_tests
Aaron Lun (14:29:18): > Oh thank god
Aaron Lun (14:29:24): > thought someone had reverted all of the bugfixes.
Aaron Lun (14:30:37): > had a little heart palpitation there.
Kevin Rue-Albrecht (14:47:52): > Three words: git fetch —prune:joy:
Aaron Lun (14:48:03): > Yeah, I did that.
Aaron Lun (14:48:21): > Didn’t quite clean it out.
Kevin Rue-Albrecht (14:49:23): > Cool for the brush tests. I was going to ask you if you could take care of those
Federico Marini (16:08:06) (in thread): > i think it is done. at least, a docker image is built and available at quay
Kevin Rue-Albrecht (16:09:47) (in thread): > I’ve never used docker images yet, but that sounds like good news. Does it mean we should merge the PR?
Federico Marini (16:10:26) (in thread): > the only thing is whether there are going to be probs with the bioc submission for having this “exotic” files
Federico Marini (16:10:36) (in thread): > probably not
Federico Marini (16:10:48) (in thread): > but AFAIK it is not a common practice
Kevin Rue-Albrecht (16:11:10) (in thread): > well, let’s just runcheckandBiocCheckand if they don’t complain, I think we’re good
Federico Marini (16:11:24) (in thread): > i don’t know how complicated it would be to otherwise maintain the extra branch
Kevin Rue-Albrecht (16:11:34) (in thread): > worse case, the Bioc reviewer will tell us to remove it, and i’m good with that too
Kevin Rue-Albrecht (16:13:19) (in thread): > ah right. well I think an extra branch would not pose any problem considering that the docker-related files are separate from the rest of the package
Kevin Rue-Albrecht (16:14:12) (in thread): > it’s just a matter of rebasing the branch every now and then, and maybe updating the list of dependencies in theinstall.Rfile:https://github.com/csoneson/iSEE/pull/90/files#diff-758da3847ee09133b4dd3197e5a60716 - Attachment (GitHub): Dockerfile by davismcc · Pull Request #90 · csoneson/iSEE > Adding Dockerfile and install.R script to build the package into a functioning docker image.
Federico Marini (16:14:49) (in thread): > that was my idea spoken out loud in detail, yep
Federico Marini (16:14:51) (in thread): > :slightly_smiling_face:
Federico Marini (16:15:27) (in thread): > thanks for the verbosity - I still am trying to recover from the nursing week where the kids all got the virus
Kevin Rue-Albrecht (16:15:35) (in thread): > once again, if it turns out to be more work than worth the effort, we can always drop it later
Federico Marini (16:15:53) (in thread): > agreed
Kevin Rue-Albrecht (16:15:58) (in thread): > verbosity? you mean on the channel?
Federico Marini (16:16:18) (in thread): > nono.. just here for detailing clearly the idea
Federico Marini (16:16:44) (in thread): > I was on--verbose=not_even_runtime
Federico Marini (16:16:46) (in thread): > :stuck_out_tongue:
Kevin Rue-Albrecht (16:17:19) (in thread): > ahh
Kevin Rue-Albrecht (16:17:43) (in thread): > i though like the other day, when the channel was sadly quiet:sweat_smile:
Kevin Rue-Albrecht (16:18:17) (in thread): > i’ve got an odd feeling that we more than partly contributed to hitting the 10k messages so rapidly hehehe
Kevin Rue-Albrecht (16:23:48): > it just hit me on my way home: i don’t think thehelperandsetupscript are meantat all for setting up test objects or even their components, but rather to define (sub-)functions specifically for the testing, e.g. a function that would compare too objects in a specific way not equivalent to any of theexpect_equal,expect_identical, … functions
Kevin Rue-Albrecht (18:03:28): > Alright. Everything cleaned up except fortest_plotting.R
Kevin Rue-Albrecht (18:03:46): > I’m keeping that one for tomorrow. Don’t wanna spoil all the fun tonight
Kevin Rue-Albrecht (18:06:25): > I think we’re not doing too bad on progress
2018-02-06
Kevin Rue-Albrecht (04:41:27) (in thread): > <!channel>Just thinking about last time we mentioned scheduling a Skype update. I found that Aaron wrote: > “I think we need a feature freeze and testing of everything we have now. But if you are going to add new features, I would suggest thinking about heatmaps.” > In any case, I would offer the idea of having one after I’m done refactoringtest_plotting.R, so that we can discuss the current test layout, and one we are all on the same page, distribute the remaining ~65% of code coverage amongst ourselves. Opinions?
Kevin Rue-Albrecht (04:42:28): > Apparently we can’t tag the channel in a thread. Makes sense:sweat_smile: - Attachment: Attachment > <!channel> > Just thinking about last time we mentioned scheduling a Skype update. I found that Aaron wrote: > “I think we need a feature freeze and testing of everything we have now. But if you are going to add new features, I would suggest thinking about heatmaps.” > In any case, I would offer the idea of having one after I’m done refactoring test_plotting.R, so that we can discuss the current test layout, and one we are all on the same page, distribute the remaining ~65% of code coverage amongst ourselves. Opinions?
Kevin Rue-Albrecht (04:53:46): > btw: I’ll be done refactoringtest_plotting.Rtonight, for sure, I’ve done half the job already this morning, but need to get on with other things for the rest of the day
Charlotte Soneson (04:54:50): > Another Skype sounds good. I started thinking about the heatmaps, sounded like some of you also had some ideas or potential issues so perhaps would be good to discuss that as well.
Kevin Rue-Albrecht (04:57:29): > I’m keen to discuss heat maps too. I’ve kinda closed myself in the habit of using theComplexHeatmappackage for manuscript figures, but I’m very curious aboutgeom_rasterthat sounds exactly what we’re looking for, to stick with theggplot2framework
Kevin Rue-Albrecht (04:57:59): > curious to hear about your individual experience/favorite ones
Aaron Lun (06:08:15) (in thread): > It would still be good to have SCE object creation in setup, otherwise it would really take too long. However, everything else should go inside the individual files, to keep them reasonably self-contained.
Aaron Lun (06:11:09) (in thread): > … which I see you’ve done.
Kevin Rue-Albrecht (06:11:37) (in thread): > I’m learning.. slowly but (kinda) surely
Federico Marini (07:24:02): > historical fan of pheatmap
Federico Marini (07:24:44): > since ComplexHeatmap was not out yet once I started doing heatmaps with extras such as annotation columns, “block” separation, etc
Federico Marini (07:25:16): > and yes, I could be in for another skype round
Kevin Rue-Albrecht (09:33:07): > nice job on the table links Aaron
Aaron Lun (09:58:04): > stuck in a refactoring fight with the tables, re. sanity of memory.
Aaron Lun (09:58:12): > Probably the same question applies with the brushes as well.
Aaron Lun (09:58:17): > Need to ponder this…
Kevin Rue-Albrecht (09:58:59): > Ok. I was hoping to leave you unit testing the brush parameters inplotting.Ranyway.
Aaron Lun (09:59:31): > Well, probably not too hard.
Aaron Lun (09:59:40): > Just check the commands get churned out
Kevin Rue-Albrecht (09:59:42): > similar to zoom, i had the feeling
Aaron Lun (09:59:43): > and the brushby is correct.
Kevin Rue-Albrecht (10:00:41): > In any case, first commit tonight will be the refactoring of existing tests. Then I’ll have a look at writing new tests.
Aaron Lun (10:00:57): > Okay
Aaron Lun (10:40:11): > UAUGHAUGA
Federico Marini (10:41:28): > almost an RNA sequence, phew
Kevin Rue-Albrecht (11:15:10): > Was that a termination codon? Over.
Aaron Lun (11:19:27): > Just tried a cycle of refactoring, and ended up right back at the start.
Kevin Rue-Albrecht (11:19:47): > ahh a circular plasmid then@Federico Marini
Federico Marini (11:21:58): > Aspartic acid, Alanine, Methionine, Asparagine!
Kevin Rue-Albrecht (11:23:18): > leucine-alanine-tryptophane-leucine
Federico Marini (11:44:51): > hey<!channel>
Aaron Lun (11:44:56): > Yo
Kevin Rue-Albrecht (11:45:00): > hey coco
Federico Marini (11:45:06): > after some nerve-damaging pkg collisions
Federico Marini (11:45:08): > http://shiny.imbei.uni-mainz.de:3838/iSEE/
Federico Marini (11:45:17): > it should be uppandrrrrunning
Aaron Lun (11:45:21): > fuck yeah
Kevin Rue-Albrecht (11:45:38): > :tada::tada::tada::tada::tada:
Federico Marini (11:46:07): > http only as of now
Kevin Rue-Albrecht (11:46:07): > port 3838 ?!?!?!?! what have you dooooone!:stuck_out_tongue:
Kevin Rue-Albrecht (11:46:11): > where is 42?
Federico Marini (11:46:16): > very brutalo brutalo
Federico Marini (11:46:30): > i had some other apps running:smile:
Federico Marini (11:46:33): > anyway
Federico Marini (11:46:41): > it is a snapshot of now
Kevin Rue-Albrecht (11:46:57): > it’s perfect. just what we need for demos at least
Federico Marini (11:47:05): > good thing, it is publicly facing DA INTERNETTAH
Federico Marini (11:47:10): > exactly
Federico Marini (11:47:27): > feel free to blame me if it works on your pc locally but not on this server:stuck_out_tongue:
Federico Marini (11:48:25): > I still did not manage to follow up on@Aaron Lun’s links for the 10x sets
Federico Marini (11:48:31): > otherwise we could demo also these
Federico Marini (11:49:11): > I will give another presentation on Fr, and it is going to be in front of neurobiologists
Federico Marini (11:49:32): > therefore I wanted to have them peek into iSEE + 10x subset
Federico Marini (11:58:50): > btw, feel free to give away the link - just keep in mind it is not so the monster machine one would hope for
Aaron Lun (11:59:35): > Okay, was going to do so.
Kevin Rue-Albrecht (11:59:49): > it’s already lagging:stuck_out_tongue:
Kevin Rue-Albrecht (12:00:49): > nah once booted up it’s fine
Federico Marini (12:01:00): > the boot phase is long-y
Federico Marini (12:01:08): > but should not be the main issue
Federico Marini (12:01:20): > i ensure some timeout when loading so the server does not crash
Kevin Rue-Albrecht (12:49:31): > Which one of us takes charge of the iOS iSEE app?
Kevin Rue-Albrecht (12:51:54): > Hm.. I just can’t figure out if brushing is possible on the phone.
Kevin Rue-Albrecht (13:30:16): > arf.. just got home and noticed that the screenshot didn’t get through
Kevin Rue-Albrecht (13:31:11): > @Kevin Rue-Albrechtshared a file:Lol - File (PNG): Lol
Federico Marini (13:36:46): > you need the new feature, the 4 finger iphone tapping:smile:
Kevin Rue-Albrecht (19:09:03): > alright, I’ll settle at 97.45% coverage tonight, forplotting.R
Kevin Rue-Albrecht (19:11:46): > not sure how to handle.find_linked_geneAaron, if you can take care of this one, and maybe another couple of lines in there, that I’m not sure to trigger.
Kevin Rue-Albrecht (19:12:42): > we should be ready for a merge tomorrow, i’d say, so that we can start distributing the remainder of the code to cover between us
2018-02-07
Kevin Rue-Albrecht (03:26:59): > i gotta say, experimenting with the testing code manually can get pretty fun, when you get to writing zooms and brushes programmatically
Kevin Rue-Albrecht (03:27:55): > i’ve got a clearer idea of how to write ‘modes’, i.e., to initialise the app with certain brushing connections already in place
Aaron Lun (07:18:04): > Yep, that’s why they’re there.
Aaron Lun (10:51:50): > Done.
Aaron Lun (10:52:00): > Merge when ready.
Kevin Rue-Albrecht (10:56:39): > ok cool !
Aaron Lun (13:15:22): > urgh got the beachmat reviews back. Well, that’s next week gone.
Federico Marini (13:16:27): > where did you submit aaron if i can ask?
Aaron Lun (13:18:48): > PloS comp bio
Federico Marini (13:21:33): > ok so in full article format i guess
Aaron Lun (13:24:15): > Yeah
Aaron Lun (13:25:03): > So… I won’t be doing anything on iSEE next week.
Aaron Lun (13:25:15): > The reviews were mostly reasonable, but work sucks no matter what it is.
Federico Marini (13:25:27): > I think we are already at a good point
Federico Marini (13:25:57): > IMHO we are submission-mature in terms of features (if we don’t want to slip in the heatmap functionality)
Federico Marini (13:26:17): > so it is a matter of small, fine edits
Federico Marini (13:28:03): > anyway. good luck with the beachmat plan then
Federico Marini (13:28:06): > heading home:wink:
Kevin Rue-Albrecht (13:49:12): > I like the 14 star-gazers we (already) have on GitHub, btw
Kevin Rue-Albrecht (13:54:35): > Alright. Merging the unit test branch now. At least it give us all a template/base from which to branch again and add different sets of tests.
Kevin Rue-Albrecht (13:55:02): > Plus it will make themasterbranch look better in terms of code coverage:innocent:
2018-02-08
Aaron Lun (05:10:22): > I’ll try to get some of the remaining tests done today.
Kevin Rue-Albrecht (07:04:30): > Awesome, thanks
Kevin Rue-Albrecht (07:04:42): > Anyone up for a Skype tomorrow afternoon?
Kevin Rue-Albrecht (07:05:04): > Pat ourselves on the back before heading to the pub? ^^
Aaron Lun (07:32:53): > Yes, I can do that.
Charlotte Soneson (07:53:27): > After ~3pm tomorrow (Swiss time) is fine with me
Federico Marini (08:31:58): > I’ll be attending a workshop the whole day
Federico Marini (08:34:15): > but say after 15.30 (swiss-german) it should be done
Federico Marini (08:34:20): > so I would be in
Kevin Rue-Albrecht (08:53:13): > After 3pm UK time should be fine for me.
Charlotte Soneson (08:53:57): > So shall we say 3pm UK/4pm continental time as usual then?
Kevin Rue-Albrecht (08:56:53): > Yep, let’s aim for that. We’ll adjust if anything comes up.
Aaron Lun (08:58:05): > sounds good to me.
Federico Marini (09:07:29): > ’kie:wink:
Charlotte Soneson (11:25:53): > Sorry, is it possible to postpone the call half an hour or so tomorrow? Turns out I should meet with some PhD applicants at 4…
Kevin Rue-Albrecht (11:29:09): > Should work for me. Otherwise, I don’t want to pressure anyone, we can postpone it for next week. Not sure where we want to draw boundaries, but I’m not opposed to weekends either if it helps avoid collision with work time.
Aaron Lun (12:23:25): > Don’t know what boundaries are.
Kevin Rue-Albrecht (12:24:00): > remove your hand of my shoulder first:sweat_smile:
Charlotte Soneson (12:38:14): > Most of this weekend is fine with me too if that works better for you.
Aaron Lun (12:38:28): > I only sleep with students.
Aaron Lun (12:38:40): > Okay, that sounded weird.
Aaron Lun (12:38:52): > As in, we had to share a room.
Kevin Rue-Albrecht (12:39:16): > don’t worry Aaron, in 10k messages, it will all be buried in Slack logs:innocent:
Federico Marini (13:13:08): > I am little more bound on the weekend
Federico Marini (13:13:30): > but once the wild ones are in bed, I can be operative:slightly_smiling_face:
Federico Marini (13:13:52): > or in any case, tomorrow 16.30 would also be ok
Charlotte Soneson (14:02:45): > Thanks, so it seems everyone is fine with 15:30 UK/16:30 CH-DE tomorrow, so let’s aim for that then.
Aaron Lun (14:14:52): > Waiting for something to run, so will refactor someiSEEcode in preparation for adding heatmaps.
Aaron Lun (14:20:46): > This is going to be a bit brutal…
Aaron Lun (15:57:28): > Imminent commit has a number of changes.
Aaron Lun (15:58:18): > 1. Themodehas been reconfigured toredDimPlot, rather than justredDim(and so on for all the others). This avoids the need to decide whether to stickTableorPlot(orHeatmap) onto the end of the mode name.
Aaron Lun (15:59:06): > 2.geneExprPlot(for Gene expression plots) has been changed tofeatExprPlotfor feature expression plots. This should be more general as you can stick anything in aSummarizedExperimentobject.
Aaron Lun (15:59:45): > 3.geneStatTablehas been changed torowStatTable, again for generality forSummarizedExperimentinput.
Aaron Lun (16:00:29): > 4. Shiny input fields were previously (but not always) named likeredDimColorBy1. Now they are consistently named asredDimPlot1_ColorBy, which should reduce confusion when setting/getting fields.
Aaron Lun (16:02:40): > I should probably also change the colouring options to “Row table” and “Feature name”… but I’m hungry.
Aaron Lun (16:03:32): > Let me know if there are any bugs.
Kevin Rue-Albrecht (16:14:07): > All good points. (2) crossed my mind when we talked about FACS but I kept forgetting to bring it up. Nice work!
Aaron Lun (16:21:10): > There’s probably a number of bugs that have slipped through; have a play around and check that things work.
Aaron Lun (16:21:17): > It’s on theaaron_testsbranch.
Kevin Rue-Albrecht (16:50:35): > will do. just gotta write a few supp table legends for a manuscript first
Kevin Rue-Albrecht (17:27:18): > doesn’t look broken in any way that I can see
2018-02-09
Aaron Lun (04:23:07): > Impending push will also change the names of the colouring options, to “Row table” and “Feature name”.
Aaron Lun (04:23:52): > The greatest potential for bugs will be when I’ve forgotten to put_in the Shiny input name.
Federico Marini (05:31:21): > quick update from the workshop
Federico Marini (05:31:31): > they really want people to focus on the talks
Federico Marini (05:31:39): > i.e., they do not have wifi
Federico Marini (05:32:05): > but i expect to be able to flee and make it on time for our scheduled skype meetup
Federico Marini (05:38:24): > (also expect maybe a couple of stars & watchers on the repo which they photographed during the talk:slightly_smiling_face:)
Federico Marini (05:39:15): > I did not photobomb their pics:wink:
Aaron Lun (05:39:28): > lol
Federico Marini (05:41:29): > I admit I was tempted
Federico Marini (05:41:45): > lots of people in suits here
Federico Marini (05:41:53): > need to put back some balance
Kevin Rue-Albrecht (06:13:36): > what balance?
Aaron Lun (06:14:41): > in the force
Kevin Rue-Albrecht (10:17:47): > I just realised it’s in 10 min right? 15:30 uk time
Kevin Rue-Albrecht (10:18:51): > It’s all good too, but I was about to miss it by an hour hehehe
Aaron Lun (10:21:00): > yep
Charlotte Soneson (10:29:29): > I’m running a bit late with the interviews, please start without me. I’m joining soon, sorry:grimacing:
Aaron Lun (10:29:38): > skype is being a bitch for me
Aaron Lun (10:29:47): > why does it need my email
Aaron Lun (10:29:49): > ???
Aaron Lun (10:30:48): > and my DOB?
Kevin Rue-Albrecht (10:31:03): > your Skype doens’t have boundaries either:stuck_out_tongue:
Aaron Lun (10:31:18): > well anyway, I’m in.
Aaron Lun (14:24:42): > Holy shit this shit is insane.
Aaron Lun (14:24:49): > Uploading once it clears check.
Aaron Lun (14:24:52): > it will BLOW your mind.
Kevin Rue-Albrecht (14:26:24): > i look forward to it:slightly_smiling_face:
Aaron Lun (15:00:48): > Waiting for CI. But this is best enjoyed with: > > library(iSEE) > example(iSEE, ask=FALSE) # just Ctrl'C out > > rowData(sce)$ave_count <- rowMeans(counts(sce)) > rowData(sce)$n_cells <- rowSums(counts(sce) > 0) > rowData(sce)$rand_letters <- sample(LETTERS[1:5], nrow(sce), replace=TRUE) > iSEE(sce) >
Aaron Lun (15:05:00): > @Federico Marinineed to update the code tracker for the new colours (also take this opportunity to centralize the colour specification; for example, there are some colours indynamicUI.Rto specify the shinydashboard box colours).
Federico Marini (15:34:00): > ok will do:wink:
Federico Marini (15:44:51): > I can’t see the rowData plot - is it supposed to be there already or did I mishear that in the skype conv`?
Charlotte Soneson (15:46:23): > It’s visible to me
Charlotte Soneson (15:47:42): > Looks nice:slightly_smiling_face:
Federico Marini (15:49:42): > danmn
Federico Marini (15:49:48): > i pulled
Federico Marini (15:49:51): > but did not install
Federico Marini (15:49:55): > JEEZ
Charlotte Soneson (15:50:21): > It’s late Friday evening, that’s understandable
Federico Marini (15:50:51): > thanks for the comprehension:slightly_smiling_face:
Federico Marini (15:56:50): > got it too
Federico Marini (15:56:53): > neat work:slightly_smiling_face:
Kevin Rue-Albrecht (16:28:02): > neat work += 1
Federico Marini (16:44:44): > somewhat there are two genetables for each one which really exists - in the panels graph?
Federico Marini (16:46:46): > I’ll check
Federico Marini (16:53:15): > ok, got it. after aaron’S changes, they all get inserted already in the graph, so I dont needhttps://github.com/csoneson/iSEE/blob/a5a82b06623fedf7192668e82d07ee812d76d60f/R/codetrack.R#L129 - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Aaron Lun (17:09:42): > If someone can give smarter x/y-axis labels to the reduced dimension plot, we can close one of the issues.
Aaron Lun (17:11:41): > And thinking further about it - I suspect that a record of the “incoming” brush would be better served as a UI element hyperlinked to the location of the transmitting plot, which should make it easy to navigate around the interface. I am also open to having similar links to all the child plots; this information can be extracted relatively easily frombrush_links.
Kevin Rue-Albrecht (18:32:00) (in thread): > @Aaron Luni’m about to commit the update, but I can’t find an issue that mentions it
Kevin Rue-Albrecht (18:32:21) (in thread): > the only naming issue is about the plots/tables themselves
Aaron Lun (18:56:03) (in thread): > Oh yeah, that was the one I was thinking of.
Aaron Lun (18:56:22) (in thread): > I think otherwise, the names are unnecessary as everything is embedded in the plot itself.
Aaron Lun (18:56:43) (in thread): > (Note that the red dim axis names need a bit of protection from absent names.)
Aaron Lun (18:58:03) (in thread): > (Or duplicate names)
Kevin Rue-Albrecht (19:00:37) (in thread): > argggh ok, i won’t merge the PR just yet then
Kevin Rue-Albrecht (19:00:49) (in thread): > damn users who don’t name their red dim
Aaron Lun (19:01:43) (in thread): > You could extract the naming scheme for the red dim choices fromdynamicUI.R; make it a bit more elegant (e.g., to avoid modifying the names when they are unique), and apply it to.make_redDimPlot.
Kevin Rue-Albrecht (19:01:51) (in thread): > they already reduced their red dim names to acronyms… to be honestSCEshould probably enforce names on that slot
Aaron Lun (19:02:09) (in thread): > It did, at the start.
Kevin Rue-Albrecht (19:02:10) (in thread): > doesn’t really make sense having unnamed red dims
Kevin Rue-Albrecht (19:02:15) (in thread): > ah
Aaron Lun (19:02:21) (in thread): > And then somewhere along the line… it stopped.
Aaron Lun (19:02:57) (in thread): > Someone also asked for unnamed spike-ins… I quickly put a stop to that madness.
Kevin Rue-Albrecht (19:03:01) (in thread): > ok
Kevin Rue-Albrecht (19:03:37) (in thread): > that’s a job for another day, when i’m motivated enough to think about all the thing that can go wrong
Aaron Lun (19:03:47) (in thread): > Well, arguablly the assysshould all be named as well, but that was a battle that was lost some time ago.
Kevin Rue-Albrecht (19:04:08) (in thread): > actually, that makes sense too
Kevin Rue-Albrecht (19:04:18) (in thread): > any historical reason why the fight was lost?
Aaron Lun (19:04:34) (in thread): > ¯*(ツ)*/¯
Aaron Lun (19:04:47) (in thread): > Probably something technical and compsci-ey.
Kevin Rue-Albrecht (19:05:05) (in thread): > meh.
Aaron Lun (19:08:41) (in thread): > Anyway, back to the main topic; the panel names are not going to change. I will add a note to the issue.
Kevin Rue-Albrecht (19:09:33) (in thread): > Ok. It will be a nice feature, but I wasn’t particularly bothered by it yet
Kevin Rue-Albrecht (19:10:32) (in thread): > i’m just glad that people are picking on small aesthetic stuff like that: means that the main functionalities of the app are on target
Aaron Lun (19:15:50) (in thread): > See my note. We might be able to do a good-enough job by adding a title to each plot.
Aaron Lun (19:16:14) (in thread): > For example, the reduced dimension plots would be titled “PCA” or whatever, and then the axes could just go back to being “Dimension 1” etc.
Aaron Lun (19:17:38) (in thread): > This would make it a lot cleaner if we had to disambiguate unnamed entries with more numbers. Otherwise you’d get(1) PCA Dimension 2, which is pretty weird.
Kevin Rue-Albrecht (19:18:11) (in thread): > mmmh
Aaron Lun (19:19:43) (in thread): > Titles for the other plot types are pretty simple; just do A vs B for whatever is being plotted (or just A, if there is nothing on B).
Kevin Rue-Albrecht (19:19:43) (in thread): > simple as pi to move the red dim type to title, i’ll think about the disambiguation with a clearer mind tomorrow/soon
Aaron Lun (19:20:24) (in thread): > Okay, I’ll leave it to you then.
2018-02-10
Charlotte Soneson (05:45:08): > Just to say that I’m working on the heatmaps now, if everything works out I will push an initial scaffold later today.
Aaron Lun (07:09:42): > Seet
Aaron Lun (07:09:44): > sweet
Aaron Lun (07:10:09): > Fleshing out my response to reviewer 1…sigh
Aaron Lun (07:11:20): > Note that, for the heatmap prototyping, because it won’t be interacting with any other plots yet, it may be easier/faster to prototype in a completely separate shiny server/UI framework.
Charlotte Soneson (10:55:42): > Ok, first crude heatmap prototype is now in theheatbranch. Many issues still remain though: > - Column ordering is not yet possible to change > - Shall we allow coloring by more than one column? > - Color also by gene expression? > - Currently the size of the coloring bar changes with the number of genes (since each gene row is 1 unit wide, but the actual space corresponding to 1 unit changes with the number of genes, and I defined the color bar to be 0.6 units) > - Currently only gene input via selectize is possible > - Colors are ugly (I’m not using the provided colormaps yet for the colData coloring) > - No way to reorder the genes except deleting a gene and retyping/selecting it in the position where one wants it > - Color by “None” doesn’t actually remove the coloring (we need a.process_colorby_choice_for_heatmaps()function > - Documentation needs updating > > I’ll keep on working on it, but it may not be today.
Aaron Lun (13:01:47): > GUARGH
Aaron Lun (13:01:52): > SICK OF REVIEWER 1
Aaron Lun (13:02:00): > Gonna do some iSEE coding for a change
Aaron Lun (13:17:32): > Fortunately, one of my colleagues gave me some cake to get me through the night.
Aaron Lun (14:19:51) (in thread): > This is done, except for the hyperlinks.
Kevin Rue-Albrecht (14:36:55): > Just realised i forgot to turn on slack today. Nice to hear about the heat map prototype
Kevin Rue-Albrecht (14:41:10): > I still have a few figures to put together, so I probably won’t be able to look at iSEE tonight, i’ll put a bit of effort in tomorrow, to finish up my branch at least (information in plot title, subtitle, and maybe throwing in the n of N samples selected, too while I’m at it)
Kevin Rue-Albrecht (14:42:34): > shaaaaame:Merging #133 into master will decrease coverage by 0.97%.
Aaron Lun (14:44:52): > Yeah, well, there’s plenty more where that came from.
Aaron Lun (15:01:33): > The other thing you have to consider with heatmap gene clustering is that you’ll need to center them.
Aaron Lun (15:01:46): > Otherwise all highly expressed genes will cluster together, regardless of their relative expression profiles.
Aaron Lun (15:05:09): > Holy fuck:https://gist.github.com/vnijs/2e6eec27eaea12beec00
Aaron Lun (15:05:31): > Drag and drop in selectize; this would make it really easy to arrange stuff on a heatmap.
Aaron Lun (15:06:04): > We can automatically suggest an order based on cluster; but the user is also free to fiddle with it.
Charlotte Soneson (15:10:32): > cool!
Aaron Lun (15:56:04): > Moreover, we can link to row statistics table, by simply updating the selectize upon any row selection. (This would be strictly additive, clicking on another row will not delete existing selections).
Aaron Lun (15:57:08): > We can add two buttons to the heatmap: “Cluster genes”, which will cluster the genes, and “Update heatmap”. The latter is important as a button as the heatmap is much larger than the other plots in terms of data points (Ncells * Nselected genes) so it would probably be unwise to have it fully reactive.
Kevin Rue-Albrecht (15:57:30): > yep for button
Aaron Lun (15:58:39): > However, I’m still not particularly convinced of the utility of brushing on the heatmap. I’m not even sure how we could engineer zoom for it, if we’re not going to usecoord_cartesianin aggplot framework.
Aaron Lun (15:59:39): > On a separate note; I think ggplot itself is pretty fast, the row data plots have about 20000 points and render very quickly.
Kevin Rue-Albrecht (16:01:07): > the other think that i was recently thinking about is whether sending a ‘color’ brush to a heat map would makes sense/be useful or not
Kevin Rue-Albrecht (16:01:37): > i picture it as a bunch a red pixel scattered around the heat map
Aaron Lun (16:02:06): > It would be possible to have an extra colour bar denoting brushed/unbrushed.
Aaron Lun (16:02:32): > But red pixels in the heatmap itself wouldn’t make much sense, this would imply some brushing of specific cell/gene combinations.
Aaron Lun (16:03:19): > The choice would be analogous to “Brush to restrict” vs “Brush to color”
Aaron Lun (16:03:56): > This is probably limited to the cells, as the genes must be pre-selected so there’s not all that much brushing that can be done.
Aaron Lun (16:07:24): > On another note, we can useinput$tableID_rows_allto import the rows that are currently being shown on a linked row table. So you could search in the row table, and then import the search (not the selection) into the gene list for the heatmap.
2018-02-11
Aaron Lun (08:53:55): > Hello? Is anyone there?
Kevin Rue-Albrecht (08:54:00): > yup
Aaron Lun (08:54:06): > Oh good.
Aaron Lun (08:54:16): > Just wanted to know.
Aaron Lun (08:54:22): > Sort of quiet in the office.
Kevin Rue-Albrecht (08:54:50): > god, if you count all the times I cited your F1000 workflow in the paper as distinct citations in the manuscript that I’m editing right now, you’d be rich
Aaron Lun (08:54:56): > lol
Kevin Rue-Albrecht (08:55:44): > so you work from the office on weekends too, I feel from the messages at least the last few weekends?
Kevin Rue-Albrecht (08:56:38): > I’d go as well, to enjoy the large screen of the desktop. But that’d mean to get out of my pyjamas:stuck_out_tongue_winking_eye:
Aaron Lun (08:56:59): > yep
Aaron Lun (08:57:48): > My home computer runs on an AMD semptron. Made in 2004.
Aaron Lun (08:57:55): > So… not conducive to work.
Kevin Rue-Albrecht (08:58:17): > ahhh… not all things age well:grin:
Kevin Rue-Albrecht (08:58:30): > computers most of all
Aaron Lun (09:00:05): > I can’t even watch 720p videos without it crashing.
Kevin Rue-Albrecht (09:01:42): > I’m just trying to picture my favorite YouTube playlists with 8-bit music instead ^^
Kevin Rue-Albrecht (09:17:14): > (I can’t be particularly talkative for the next hour or so, sifting through a manuscript, both proof-reading and replacing for the first time all the “PMID: xxxxx” by the appropriate EndNote reference). > That ambivalent moment where you both enjoy seeing the project coming to a close, and hate it for all the tedious editing bits to do
Kevin Rue-Albrecht (09:17:56): > kind of addressing a reviewer’s comment:wink:
Aaron Lun (09:21:53): > ok
Aaron Lun (09:22:12): > I am trying to figure out whether it’s worth recompiling R with--enable-memory-profilingto deal with a reviewer’s comment.
Kevin Rue-Albrecht (09:27:39): > i was about to say “lol”, but i see the dilemna/pain
Aaron Lun (09:32:01): > Guh I’ll just do it.
Aaron Lun (09:32:06): > And work oniSEEin the meantime.
Kevin Rue-Albrecht (09:45:20): > finally:sleepy:Damn you Pubmed, (e.g.https://www.ncbi.nlm.nih.gov/pubmed/15723809 )
Kevin Rue-Albrecht (09:45:28): > Error occurred: The following PMID is not available: 15723809
Kevin Rue-Albrecht (09:46:09): > had to download a bunch of RIS/EndNote/Bibtex files straight from the individual journal websites
Aaron Lun (10:11:02): > Well, the memory profiling told me absolutely nothing. Why? BecauseRprofmemdoesn’t capture memory allocations that aren’t performed by R’s C API!
Kevin Rue-Albrecht (10:12:03): > can’t remember last time i played with that feature:confused:
Aaron Lun (10:35:37): > Okay, enough of that.
Aaron Lun (10:35:43): > Going to do something enjoyable instead.
Kevin Rue-Albrecht (10:36:44): > as it happens, i also just snapped, and went to finish up my reddim names branch
Kevin Rue-Albrecht (10:38:25): > so do we agree on: > -title="(1) TSNE"if the reddim is named > -title="(1)"if the reddim is not named?
Aaron Lun (10:38:39): > Yeah, that’ll be fine.
Kevin Rue-Albrecht (10:38:41): > or should the latter be a bit more verbose?
Aaron Lun (10:39:00): > Hm… well, we should go with whatever is in the select options.
Kevin Rue-Albrecht (10:39:03): > I don’t want to over-complicate things
Aaron Lun (10:39:06): > As long as those are sync’d.
Aaron Lun (10:39:46): > I would suggest refactoring out the function that generates the names in the SelectInput indynamicUI.R, so you can re-use the same code when generating the title.
Kevin Rue-Albrecht (10:40:00): > i was just looking at it
Kevin Rue-Albrecht (10:40:22): > makes sense, to avoid copy-pasting or writing it differently
Kevin Rue-Albrecht (10:40:46): > goes inmisc.R?
Kevin Rue-Albrecht (10:41:28): > ordefaults.R?
Kevin Rue-Albrecht (10:41:44): > there’s a bit of renaming going on in the latter already
Kevin Rue-Albrecht (10:42:06): > ah no I meantconstants.R
Aaron Lun (10:43:28): > Maybe just leave it indynamicUI.R, not sure that it really belongs in constants.
Aaron Lun (10:43:48): > And misc.R is intended solely to take things out of iSEE.R
Aaron Lun (10:44:00): > But whatever feels natural.
Kevin Rue-Albrecht (10:44:01): > ok, makes sense too
Aaron Lun (10:56:39): > Oh selectizeInput is NOT happy with the thousands of entries it’s being asked to handle.
Kevin Rue-Albrecht (10:57:21): > trying to decide; what do we want as a title for colData scatter plots?"covariate_y vs. covariate_x"?
Kevin Rue-Albrecht (10:58:01): > (just found out a nice feature of Slack: if you hit the up arrow right after sending a message, it opens the ‘edit message’ mode)
Aaron Lun (10:58:13): > Yeah, that should be fine.
Kevin Rue-Albrecht (10:58:43): > ok, i’ll put it like that for now as a sort of placeholder, we can refine later if we want
Kevin Rue-Albrecht (10:58:56): > selectize went on strike?
Aaron Lun (10:59:15): > Only when we ask it to drag and drop. Then it becomes really unhappy.
Aaron Lun (10:59:36): > Actually, it seems unhappy either way.
Aaron Lun (10:59:46): > Quite a long delay to get the app running.
Aaron Lun (11:00:32): > Is anyone else noticing this?
Kevin Rue-Albrecht (11:01:06): > i do wait a few seconds, on thenamed_reddimbranch
Kevin Rue-Albrecht (11:01:50): > i can’t tell when exactly the delay increased that noticeably
Aaron Lun (11:02:39): > On the heat one we wait a lot longer for the UI to even render.
Aaron Lun (11:02:43): > Let alone show the plots.
Kevin Rue-Albrecht (11:03:17): > hm…:confused:
Kevin Rue-Albrecht (11:04:20): > it’d be really nice if we could have a way like unit tests, to automatically measure the time for the app to initialise, for every commit
Kevin Rue-Albrecht (11:04:38): > no clue if that’s possible, but it would help track down that kind of issues
Aaron Lun (11:05:08): > https://stackoverflow.com/questions/35664657/r-shiny-optimize-page-loading-time-with-updateselectizeinput - Attachment (stackoverflow.com): R Shiny - Optimize page loading time with updateSelectizeInput > Our shiny page has multiple selectizeInput controls, and some of them have long lists in their drop-down boxes. Because of this, the initial loading time is very long as it needs to pre-fill drop-d…
Kevin Rue-Albrecht (11:05:40): > at the moment, testing it manually if 1) subjective, 2) depends on the machine and the current CPU load
Kevin Rue-Albrecht (11:06:18): > ideally, we should get Travis to do that, objectively
Aaron Lun (11:11:47): > Man, now it’s lightning fast.
Aaron Lun (11:11:54): > Using server-side storage of the row names.
Aaron Lun (11:12:07): > Oh, and drag and drop.
Aaron Lun (11:12:08): > Sweet as
Kevin Rue-Albrecht (11:12:10): > ahhh yeah there is a TRUE/FALSE option isn’t there
Aaron Lun (11:12:52): > @Charlotte Sonesonare you working on this right now? I’m just going to move some things around.
Aaron Lun (11:13:12): > Like for example, y-axis doesn’t make much sense for the heatmap, there’s always going to be features.
Charlotte Soneson (11:13:53): > Sure, I’m not working on it right now. My thinking with XAxis/YAxis was that one might want to transpose the heatmap
Aaron Lun (11:14:12): > Do people do that?
Charlotte Soneson (11:16:19): > Don’t know. Maybe one could, depending on the ratio of samples/variables
Aaron Lun (11:16:43): > Hm.
Aaron Lun (11:20:14): > Do we need colouring options for the heatmap?
Aaron Lun (11:20:26): > I mean, all the colours should be pr-defined via ECM.
Aaron Lun (11:20:43): > And based on what column data you choose to order on.
Kevin Rue-Albrecht (11:21:02): > think so, I was already asked to use different color maps in projects, so I suspect it’ll be the same for iSEE
Kevin Rue-Albrecht (11:22:16): > e.g. > green-black-red for microarray fold-change > black-purple-yellow for logcounts, typically > blue-yellow-red for row z-scores of gene expression typically
Aaron Lun (11:22:40): > yes, but that should be defined in the ECM.
Kevin Rue-Albrecht (11:22:50): > oh yeah definitely
Aaron Lun (11:23:05): > so what’s the need for explicit colouring options in the heatmap?
Kevin Rue-Albrecht (11:23:13): > (btw ignore the fold-change thing, doesn’t apply to us)
Aaron Lun (11:23:51): > well, we’ll need to compute log-fold changes as well, actually
Aaron Lun (11:24:02): > you can see if you chuck in actb in the heatmap that it just doesn’t look very good
Kevin Rue-Albrecht (11:24:10): > what do you mean? rowData colours for genes, colData colours for samples, and assay colours for the tiles, no?
Kevin Rue-Albrecht (11:24:17): > or is there more to take care of
Aaron Lun (11:24:27): > No, as in an explicit panel for heatmap colours.
Aaron Lun (11:24:35): > Because all the colours for all the variables are already defined.
Aaron Lun (11:24:46): > There is no need for extra colour specification from the user.
Kevin Rue-Albrecht (11:26:14): > Not sure I follow the ‘explicit panel’ part. Your last two points was what I was talking about
Aaron Lun (11:27:10): > So you know how all the other panels have a coloring parameters section?
Aaron Lun (11:27:15): > I’m saying that a heatmap doesn’t need that.
Aaron Lun (11:27:20): > Because there are no choices to make.
Kevin Rue-Albrecht (11:27:21): > ahhhh got you now
Kevin Rue-Albrecht (11:28:06): > Mmmmh, I think you’re right. Point of a “heat” map is that it’s always coloured.
Kevin Rue-Albrecht (11:28:47): > Oh wait hang on. User would still need an explicit colour panel to decided whichassaythe colour data represents, no?
Kevin Rue-Albrecht (11:29:03): > or is that defined elsewhere, for each particular heat map?
Kevin Rue-Albrecht (11:30:07): > Let’s imagine they want to see the heat map with ‘logcount’ colouring and ‘tpm’ colouring side by side. They’ll need a colour panel to select those two sources of colors separately for each heat map
Charlotte Soneson (11:31:20): > I guess the user order the cells by one covariate and color the “annotation bar” according to another
Aaron Lun (11:35:21): > tHAT SHOULD BE INT HE PLOTTING PARAMETERS
Aaron Lun (11:35:24): > whoops
Aaron Lun (11:35:43): > because it’s a fundamental part of what’s being shown, rather than an aesthetic overlaid on top.
Kevin Rue-Albrecht (11:35:51): > ahhhhhhhhh
Kevin Rue-Albrecht (11:36:13): > With you now. Yep. No colour panel needed.
Charlotte Soneson (11:37:26): > Ah yes, I guess the coloring can go there too, since the number of annotation bars depends on the choice.
Charlotte Soneson (11:37:43): > So it is just not overlaid on something that is there anyway
Aaron Lun (11:42:25): > In that case, I’ll take the liberty of excising the heatmap colour options.
Aaron Lun (11:45:04): > I’ll also remove the y-axis choice; I can’t really imagine a strong demand for the transposed version.
Charlotte Soneson (11:45:14): > Ok, sure
Kevin Rue-Albrecht (11:47:54): > i’ve reached something apparently stable for plot title names, although it gets a bit ugly for gene-gene featExprs plotsgene (assay) vs gene (assay)
Aaron Lun (11:48:47): > Don’t worry about the assay in the title, I think.
Aaron Lun (11:49:01): > Poeple can see the axes for more detail, the title just gives all info in the glance.
Kevin Rue-Albrecht (11:49:07): > i don’t think we want to change it, but i noticed that we don’t let users plotsgene (assay1) vs gene (assay2), instead both gene expression are always from the same assay
Aaron Lun (11:49:16): > Yes, I had thought about that.
Aaron Lun (11:49:31): > On balance, it’s probably the right thing to do.
Kevin Rue-Albrecht (11:49:45): > maybe something low priority for the backburner, if there is some demand
Aaron Lun (11:50:18): > Well, as in, it is the right thing to NOT allow them to change it.
Aaron Lun (11:50:41): > Currently, the setup ensures you’re alway splotting the same thing against each other.
Aaron Lun (11:50:56): > If you provided options to have different assays, users woul dhave to manually sync them together.
Kevin Rue-Albrecht (11:51:26): > i likesplotting, we should use that shorthand forscatterplotting
Kevin Rue-Albrecht (11:52:16): > Anyway, I completely agree. It’s the superfluous feature that would get used once every blue moon
Aaron Lun (12:13:48): > Well, it’s done on my end, but there’s an unnecessary re-rendering of the heatmap upon re-rendering the UI, and I’m not sure where it’s coming from… will figure it out later.
Aaron Lun (12:14:49): > Well, to be precise, I know where it’s coming from; I don’t know how to avoid it.
Kevin Rue-Albrecht (12:27:48): > Alright, i pushed my changes too. Still on a side branch.
Aaron Lun (12:38:11): > Waiting for check to pass locally, then will push toheat.
Aaron Lun (12:54:31): > Feel free to close the issue once you’ve merged it.
Aaron Lun (12:56:07): > @Charlotte SonesonNote the new ColData storage field; this can be used to define how to order the cells.
Aaron Lun (12:56:19): > Incidentally, probably good to add a validate against empty ColData or FeatNames.
Kevin Rue-Albrecht (12:56:22): > sure, shall i merge now? it’s ready to go, but it’ll force everyone to update their own branch before they can merge
Aaron Lun (12:56:32): > Well, we’re gonna have to do that sooner or later.
Aaron Lun (12:57:26): > Okay, onto implementing this damn matrix multiplication.
2018-02-12
Aaron Lun (04:54:43): > ALRIGHT LET’S DO SOME OF THIS
Aaron Lun (04:57:54): > Nice titles@Kevin Rue-Albrecht
Aaron Lun (04:58:33): > Feature expression plot isn’t happy with another gene feature tho
Kevin Rue-Albrecht (04:58:41): > Thanks. Argh!
Aaron Lun (04:58:44): > just gives me 000Rik NA
Kevin Rue-Albrecht (04:59:02): > weird, I thought I tested the version that I pushed
Aaron Lun (05:00:12): > Getting NA’s in both y-axis and x-axis choice for featNames
Aaron Lun (05:00:25): > Which, if you think about it, is funny. Could be neuraminidase.
Kevin Rue-Albrecht (05:00:26): > ahhhhhh
Kevin Rue-Albrecht (05:00:32): > I didn’t testthat scenario
Kevin Rue-Albrecht (05:00:46): > only tested when linked from table
Kevin Rue-Albrecht (05:00:48): > damn
Kevin Rue-Albrecht (05:01:20): > I know exactly what it needs
Kevin Rue-Albrecht (05:01:36): > that’s the culprit:https://github.com/csoneson/iSEE/blob/master/R/plotting.R#L146 - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (05:01:52): > I was processing linked tables and text input the same way, which I can’t
Aaron Lun (05:02:09): > In any case, I’m going to switch the feature names to selectize.
Kevin Rue-Albrecht (05:02:26): > I already got caught unaware when I realised (again) that linked tables send an integer rather than the gene id itself
Kevin Rue-Albrecht (05:02:42): > so i fixed one and broke the other without realising
Kevin Rue-Albrecht (05:03:11): > ok.. do you wanna fix the name as part of that selectize update then?
Aaron Lun (05:03:21): > No, it won’t touch that.
Kevin Rue-Albrecht (05:03:23): > ok
Kevin Rue-Albrecht (05:04:31): > well, in fact, if you switch the feature names to selectize, you’ll actually be implicitly fixing the issue
Kevin Rue-Albrecht (05:04:59): > because both linked tables and feature names will return an integer, which is what my code currently expects
Aaron Lun (05:05:55): > Hm. Not sure. Will it? selectize would still return a sttring, won’t it?
Kevin Rue-Albrecht (05:07:02): > depends what you give to selectize, if you give it a named input, you can show users strings yet return the associated integer to the input$
Kevin Rue-Albrecht (05:07:50): > or worst case, selectize might return the integer as a string, which can then simply be coerced back to integer
Kevin Rue-Albrecht (05:08:11): > but I think it will return the actual integer it you tell it so
Aaron Lun (05:13:24): > I think it returns a string, I tried this before with the PCA dimensions.
Aaron Lun (05:13:40): > or specifically, the named string of seq_along for the reddim slots.
Aaron Lun (05:13:59): > Well, you can sit tight and wait, or you can do it; your call.
Kevin Rue-Albrecht (05:16:10): > yep, i’ll look into it with your code update
Aaron Lun (05:47:58): > … harder than I thought.
Aaron Lun (06:53:34): > Does anyone know the priority ofrender*functions relative toobserve?
Kevin Rue-Albrecht (06:54:15): > ufff.. not right now, no
Kevin Rue-Albrecht (06:55:07): > however, i do remember something about setting priorities:https://stackoverflow.com/questions/24010346/priority-value-in-reactive-like-in-observe-r-shiny - Attachment (stackoverflow.com): Priority value in reactive() like in observe() (R Shiny) > How do you set the priority in which reactive() calculations are performed in Shiny? For example, there is a priority option in observe(), quoting the help file priority An integer or numeric …
Aaron Lun (06:55:29): > I need to stage the priorities so thatrenderUIexecutes first, followed by theupdateSelectizeInput, and then onlyobservefor the various fields.
Kevin Rue-Albrecht (06:55:43): > It was a couple of years go that I last looked at that, but I seem to remember that it was discouraged anyway by the RStudio people
Aaron Lun (06:56:17): > Otherwise if it tries to update the selectize before the UI re-renders, it will fail; and if it observes before the selectize updates, it will wipe any existing memory (as the re-rendered but not-updated input will be NULL).
Kevin Rue-Albrecht (06:57:19): > You might want to throw in someshiny::req(), if you can set up a condition that tracks whether something should happen only once something else is in place
Kevin Rue-Albrecht (06:58:22): > shiny::req()is what I’d call a requirementinvisible to the user, in contrast toshiny::validate
Aaron Lun (06:58:40): > Hm…
Aaron Lun (07:01:59): > I’d prefer to use the priority system if that’s possible, it’s a more natural way of dealing with it.
Aaron Lun (07:02:15): > You can check it out onheatnow.
Kevin Rue-Albrecht (07:02:58): > No worries. I’m not close enough to the code right now to know what’s best, so it’s really your call.
Kevin Rue-Albrecht (07:03:23): > I’ll have to head off to a seminar. Might be able to check out the code from there
Kevin Rue-Albrecht (07:27:57): > nice:slightly_smiling_face:
Kevin Rue-Albrecht (07:28:42): > I just noticed thatrowDataPlothas issues withargument "title" is missing, with no default. I’ll take care of that this evening if thats alright.
Kevin Rue-Albrecht (07:29:29): > I’d also move the heat map legend tobottombtw
Aaron Lun (08:49:39): > FUCK I AM A CSS GOD
Kevin Rue-Albrecht (08:49:51): > lol
Aaron Lun (09:03:19): > Behold the new era!
Aaron Lun (09:05:00): > How fancy are the colours… ridiculously.
Aaron Lun (09:08:49): > <!channel>Please treatheatas the effective master for the time being; many things have been touched and improved.
Aaron Lun (09:09:33): > Or at least, make sure that if you fix things inmaster, that you merge them (carefully) intoheat.
Aaron Lun (12:43:47): > MARCO
Kevin Rue-Albrecht (13:28:52): > not sure whether to answer ‘polo’ or ‘SCARA2’
Aaron Lun (13:29:26): > lol
Aaron Lun (13:29:39): > Does everyone appreciate the new colours?
Aaron Lun (13:29:48): > Had to go pretty deep to get that to work.
Kevin Rue-Albrecht (13:35:10): > I like it. Just wondered if it’s possible to ‘soften’ a bit the purple, kinda ggplot-style of my memory is correct
Aaron Lun (13:35:36): > Knock yourself out, it’s inpanel_colours.R, though it has to be dark enough for the white text to be visible.
Kevin Rue-Albrecht (13:36:03): > Maybe it’s the Retina display, but I found it a tiny bit intense
Kevin Rue-Albrecht (13:36:17): > But overall, I like the combo of colours
Kevin Rue-Albrecht (13:37:37): > Just heading home now. I’ll fix the rowDataPlot title tonight, if it’s still there
Aaron Lun (13:37:53): > All colours are now centralized; so you only have to change it once, and everything else automatically reflects it.
Aaron Lun (13:38:03): > e.g. brush colors, colors on the graph, etc.
Kevin Rue-Albrecht (13:38:06): > That’s absolutely perfect!
Aaron Lun (13:38:23): > So feel free to play around with the colours to get something nice.
Aaron Lun (13:38:44): > Though (i) they should be dark enough to see the white text, but (ii) not too dark to get a nice brush.
Kevin Rue-Albrecht (13:39:04): > My previous comment got me to wonder whether we couldn’t just pick a ggplot2 palette, actually
Kevin Rue-Albrecht (13:39:22): > And benefit from their work setting up discrete colour maps
Aaron Lun (13:39:31): > Probably their palette is a bit too light
Kevin Rue-Albrecht (13:39:32): > But I like how it looks now
Aaron Lun (13:39:47): > You’d have to darken it a bit to get it to work.
Kevin Rue-Albrecht (13:40:04): > Yep I get you
Aaron Lun (14:15:32): > Okay, I’m going home.
Federico Marini (14:37:26): > catching up on the whole messages… > - i recallvalidate(need)was somehow better - at least in my cases - for providing a useful alt msg when some objects where not present, or some condiitons not satisfied
Federico Marini (14:38:40): > plus, - yes, somehow the updateselectizeinput done via server is superfast
Federico Marini (14:39:01): > - but also, no, I have no clue on the priority setting:disappointed:
Kevin Rue-Albrecht (14:56:00): > I agree thatvalidate(need)is a lot more informative.reqsimply prevents further processing without message until the condition is met.
Kevin Rue-Albrecht (14:56:56): > However, I have found this useful especially at startup to prevent re-rendering of plots asinputgets populated
Kevin Rue-Albrecht (15:01:17): > for instance inTVTB::tSVEI remember having a fairly linear chain of dependencies between a series ofreactiveand settingreqto prevent downstream ones to be computed until upstream ones were notNULL:https://github.com/kevinrue/TVTB/blob/master/inst/shinyApp/server.R#L288 - Attachment (GitHub): kevinrue/TVTB > TVTB - The VCF Tool Box
Kevin Rue-Albrecht (15:04:28): > But again, I’m not saying it’s “best practice” or that I recommend it in our case. That was my first ‘large’ Shiny app, and there’s a bunch ofad hoc bits of code in there.
Kevin Rue-Albrecht (15:20:41): > i’m pushing back an update that giverowDataPlota title
Federico Marini (15:41:15): > about the million of points in ggplot2
Federico Marini (15:41:24): > here’s an interesting contribution
Federico Marini (15:41:26): > https://fronkonstin.com/2017/11/07/drawing-10-million-points-with-ggplot-clifford-attractors/#tidyverse - Attachment (Fronkonstin): Drawing 10 Million Points With ggplot: Clifford Attractors > For me, mathematics cultivates a perpetual state of wonder about the nature of mind, the limits of thoughts, and our place in this vast cosmos (Clifford A. Pickover – The Math Book: From Pyth…
Aaron Lun (17:46:40): > That’s more a case of generating the data points sequentially; it’s still one big dataframe that goes into ggplot, though.
Aaron Lun (17:47:01): > I wonder how fast it is.
Aaron Lun (17:47:43): > Re the reactives:observeEventhasignoreInit=TRUE, which avoids them triggering upon app startup.
Aaron Lun (17:50:55): > Or specifically, avoids triggering upon creation of the observer.
Aaron Lun (17:52:16): > You can check this yourself by insertingprintstatements inrenderPlotand seeing how many times they get hit. I vaguely remember checking this, and it only seemed to get hit once during startup - I might be wrong, though.
Aaron Lun (17:52:40): > In any case, we can’t really putreqinrenderPlot, because it doesn’t take values directly frominputanymore; they pass through the memory.
Aaron Lun (18:24:18): > The priorities are probably okay, upon reflection. But let me know if it breaks; this will manifest as deletion of existing entries in any of the featNames fields (color/x-axis/y-axis) when the UI is re-rendered, e.g., due to addition of new panels/deletion of old panels.
Aaron Lun (18:37:54): > It’s probably okay. It’s probably okay.
Aaron Lun (18:38:00): > I think.
Aaron Lun (18:38:14): > GAHH I’ll check it again tomorrow.
2018-02-13
Kevin Rue-Albrecht (03:36:44): > well, I’ll report weird behaviours as I encounter any, because I don’t think I could figure out how to debug this aspect of the app myself eheheh:sweat_smile:
Aaron Lun (05:31:33): > If there are any slowdowns in ggplot at large numbers of points, it may be due to transparencies or point size, which are inherently more difficult to render, I believe.
Federico Marini (09:43:25): > throwing in a thought: do we want heatmaps also to be linked in the graph once the brushes are on?
Federico Marini (09:43:42): > (and the links could come from two sides)
Aaron Lun (09:44:15): > Yes, that would be the idea.
Federico Marini (11:55:37): > … and another sparse thought - mostly relevant for the classic SummarizedExperiment objects
Federico Marini (11:56:13): > do we want to have some option for having the points labeled? In lowN-sample-land this could come in handy, also because the labels are given
Aaron Lun (11:56:51): > … not really.
Aaron Lun (11:56:56): > I mean, I guess we could
Aaron Lun (11:57:01): > but there’s not much to explore in such cases.
Aaron Lun (11:57:25): > I mean, do you really need an interactive framework to deal with 6 samples?
Kevin Rue-Albrecht (11:57:57): > i’d rather focus on showing label information on hover
Aaron Lun (11:58:10): > that turns out to be much harder than first thought.
Aaron Lun (11:58:15): > We’d have to switch to plotly
Kevin Rue-Albrecht (11:58:16): > but not labels on the plot itself
Kevin Rue-Albrecht (11:58:27): > arf
Federico Marini (11:58:44): > Germans would say Jein as a mix of ja and nein. I was only thinking of the wetlab people grabbing the SE and doing lots of plots on their own
Charlotte Soneson (11:58:58): > They can color the samples by ID:slightly_smiling_face:
Federico Marini (11:59:03): > true that
Federico Marini (11:59:29): > ok, right. they could throw in a factor for the IDs
Federico Marini (12:00:36): > ok, closing my own issue:smile:
Aaron Lun (12:00:56): > I mean, hover wouild be the ideal way to do it; you get the sample names as you hover over the plot.
Federico Marini (12:01:13): > rightey. but it’s not natively there as in plotly
Aaron Lun (12:01:13): > Of course, at ~millions of points,t his might be stupid.
Federico Marini (12:01:59): > BTW, talking about other cool stuff going on. This can be a good “competitor”, although we have a different focus
Federico Marini (12:02:01): > https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1382-0 - Attachment (Genome Biology): SCANPY : large-scale single-cell gene expression data analysis > Scanpy is a scalable toolkit for analyzing single-cell gene expression data. It includes methods for preprocessing, visualization, clustering, pseudotime and trajectory inference, differential expression testing, and simulation of gene regulatory networks. Its Python-based implementation efficiently deals with data sets of more than one million cells ( https://github.com/theislab/Scanpy ). Along with Scanpy, we present AnnData, a generic class for handling annotated data matrices ( https://github.com/theislab/anndata ).
Federico Marini (12:02:41): > or at least, something with slightly similar focus, different language, yet applicable to the whole body of big datasets
Aaron Lun (12:04:25): > ah, scanpy.
Aaron Lun (12:04:32): > I was wondering where that was.
Federico Marini (12:05:25): > was on biorxiv already, came out last week or so
Federico Marini (12:05:38): > I am biased on this, but I tend to like R more:smile:
Aaron Lun (12:05:51): > I like python as a language.
Aaron Lun (12:05:57): > But as an analysis framework, it sucks
Federico Marini (12:06:03): > scanRcould have been a nice name too
Aaron Lun (12:06:08): > I mean, i get incredibly frustrated trying to install anything.
Federico Marini (12:06:13): > but I am proud we came up with iSEE
Aaron Lun (12:06:14): > And I’m pretty good at my job.
Federico Marini (12:06:18): > :smile:
Federico Marini (12:06:38): > still not convinced john took you for the looks, nah?
Aaron Lun (12:06:38): > So imagine how your average user will suffer
Aaron Lun (12:06:54): > With the typical pip error messages.
Federico Marini (12:07:04): > I also have had lots of pip-pain
Federico Marini (12:07:07): > ohjaaa
Aaron Lun (12:09:04): > Well, never mind them.
Aaron Lun (12:09:17): > Let’s just flood the field with good R software
Aaron Lun (12:11:20): > Or in other words
Aaron Lun (12:11:33): > you might visit python every now and then, but R is where the heart is.
Federico Marini (12:13:48): > I can’t say that better:slightly_smiling_face:
Federico Marini (12:20:54): > “slightly” unrelated: Charlotte is already aware of this, but since the madness started last week, I could not notdo this
Federico Marini (12:20:55): > https://github.com/federicomarini/aaaaR - Attachment (GitHub): federicomarini/aaaaR > aaaaR - Let the aminoacids do the talking!
Federico Marini (12:21:37): > the lady at home stared me with big, big, big eyes
Aaron Lun (12:22:22): > lol
Federico Marini (12:22:44): > fun fact: all of our names cannot be completely turned into AA
Federico Marini (12:22:52): > we all have Os
Aaron Lun (12:24:38): > AlaAlaArgOAsp LeuSecAsp?
Aaron Lun (12:25:03): > U = selenocysteine, apparently.
Kevin Rue-Albrecht (12:25:35): > I don’t have O:grin:
Federico Marini (12:25:55): > The option to go to 3letters should also come at a debt timepoint
Federico Marini (12:25:59): > ops:smile:
Federico Marini (12:26:37): > there was a letter missing also for you, last time I checked - EDIT: ok, I should stop contributing to reaching the 10k msg limit
Aaron Lun (13:46:28): > Okay, I’m sick of dealing with this MS. Will work on iSEE for a few hours.
Aaron Lun (14:11:24): > @Kevin Rue-AlbrechtRow data plots need title.
Kevin Rue-Albrecht (14:35:06): > Oh I thought I did that
Aaron Lun (14:35:21): > Oh.
Kevin Rue-Albrecht (14:35:23): > I’m home in 5 min. Let me check
Aaron Lun (14:35:26): > Maybe i haven’t pulled.
Aaron Lun (14:35:52): > And indeed.
Kevin Rue-Albrecht (14:36:08): > Ok good for me
Kevin Rue-Albrecht (14:37:24): > I’m a bit fuzzy today: spent the entire day querying the Ensembl MySQL database to debug an annotation pipeline
Aaron Lun (14:37:30): > lol
Kevin Rue-Albrecht (14:37:54): > Works for mouse Ensembl 81 but not 91 (2 year difference)
Kevin Rue-Albrecht (14:38:14): > :tired_face:
Aaron Lun (14:39:30): > oh dear
Aaron Lun (15:25:56): > Dammit. It’s really annoying that selectize uses an empty string to signal a no-selection.
Kevin Rue-Albrecht (15:26:43): > i’d like to smile, but I can feel the frustration ’til here ^^
Aaron Lun (15:28:48): > I think we need to eliminate all uses of""as a valid input.
Aaron Lun (15:29:28): > A purge, basically.
Aaron Lun (15:31:06): > which would allow us to usereqwithout fear.
Kevin Rue-Albrecht (15:38:28): > I’ll risk a “that sounds sensible”, although I’ve kind of lost track of how much of a spring cleaning that would be in the code
Aaron Lun (15:43:10): > Big enough.
Aaron Lun (15:44:33): > The brushing is particularly problematic, as I was using the empty string to denote “do not receive a brush”
Kevin Rue-Albrecht (15:46:34): > canNULLbe a valid replacement? otherwise, a good ol’NA_character_should do the trick
Aaron Lun (15:47:11): > No, it won’t allow NULL.
Aaron Lun (15:47:48): > We could probably allow NA_character, though by that point, we’d need to be using a named vector, and if I’m doing that I might as well just use integers to signal valid entries.
Aaron Lun (16:00:50): > Holy shit these plots are getting evaluated ridiculous numbers of times at start up
Aaron Lun (16:00:59): > No wonder the damn thing takes so long.
Aaron Lun (16:01:29): > There’s 5 evaluations for the reduced dimension plots alone.
Kevin Rue-Albrecht (16:01:57): > i suppose it gets re-evaluated every time another plot is added (?)
Kevin Rue-Albrecht (16:02:33): > (i think we have 5 plots and 1 table on startup at the moment)
Aaron Lun (16:03:06): > Okay, fixed.
Kevin Rue-Albrecht (16:03:13): > wow, that was fast
Aaron Lun (16:03:15): > No, it was evaluated for every damn observer
Aaron Lun (16:03:39): > As an observer was created, it would trigger a plot re-rendering; which meant that we re-rendered for every observer.
Kevin Rue-Albrecht (16:04:00): > ahhh yep. brings me back toTVTB::tSVEthat one
Aaron Lun (16:04:01): > I’ll continue working on this tomorrow, I’m going home now.
Aaron Lun (16:10:37): > Pushed to heat, YMMV.
2018-02-14
Kevin Rue-Albrecht (03:28:52): > Playing with the app last night I noticed the heatmap is broken upon gene selection, but I guess that’s just work in progress?
Kevin Rue-Albrecht (03:30:00): > I’m going to look into the issue I opened myself: indicating count of brushed points on the plots
Kevin Rue-Albrecht (03:33:14): > Wednesday is a crazy day for me though : lab meeting and journal club that I can’t dodge
Aaron Lun (04:08:13): > Possibly.
Aaron Lun (04:08:18): > God you people wake up early.
Kevin Rue-Albrecht (04:10:08): > I used to be the opposite (work late/wake up late), had to change that a bit recently, still not super efficient in the morning, tbh:sleepy:
Kevin Rue-Albrecht (06:41:22): > Just reading the diffusion pseudotime (DPT) nature protocol for our journal club. Makes me wonder whether it’d be possible to overlay branches/trajectories, provided maybe a Bioc container for this type of information. > In its dumbest implementation, i’d picture it as ageom_pathin the reduceDim space with all cells ordered by pseudotime. > Obviously, 1) that wouldn’t deal with branches, where one cell would have to be connected to two daughter cells > 2)geom_pathwould drawN-1lines forNcells, so we’d rather look at some sort of smoothing (which I think was presented at the Boston Bioc last summer)
Kevin Rue-Albrecht (06:42:46): > https://www.nature.com/articles/nmeth.3971 - Attachment (Nature Methods): Diffusion pseudotime robustly reconstructs lineage branching > Diffusion pseudotime (DPT) enables robust and scalable inference of cellular trajectories, branching events, metastable states and underlying gene dynamics from snapshot single-cell gene expression data.
Aaron Lun (06:42:47): > Any reason why we need to draw lines, and we can’t use colouring to do the same job?
Aaron Lun (06:43:05): > E.g., colour from distance from the pseudotime trajectory
Kevin Rue-Albrecht (06:44:05): > hm.. true
Kevin Rue-Albrecht (06:44:40): > I’m thinking about highlighting branches
Kevin Rue-Albrecht (06:45:32): > branches may not be particularly obvious in reduced dim space, and colour by pseudotime only would not help discriminate them
Aaron Lun (06:45:32): > Yeah, you could do that too.
Aaron Lun (06:45:51): > Just colour by distance from the branch to which it is assigned.
Kevin Rue-Albrecht (06:46:41): > hm.. yep. I’ll keep thinking about it, but not rushed about implementing anything extra for now
Aaron Lun (06:46:49): > At the end of the day, there is some data assigned to each cell; we should be able to do something with it.
Kevin Rue-Albrecht (06:52:39): > Sure. Hopefully, users will find how to use the current features of iSEE to visualise that kind of data as colData. Who knows, we can always wait and see if someone comes up with a thoughtful suggestion for an extra feature of the kind.
Aaron Lun (06:59:52): > I mean, you can just colour by branch ID.
Kevin Rue-Albrecht (07:00:28): > yes, i picture them switching betweenbranch_idandpseudotime
Aaron Lun (07:01:09): > Or you can have two plots.
Kevin Rue-Albrecht (07:02:18): > duh
Kevin Rue-Albrecht (07:02:21): > sorry
Aaron Lun (09:27:22): > Alright, that’s enough revisions.
Aaron Lun (09:27:29): > Time to do something fun for a change.
Kevin Rue-Albrecht (09:39:50): > What’s the battle plan for the heat map at the moment? I can look at it this week end at the earliest.
Aaron Lun (09:40:23): > I will continue linking existing infrastructure to it.
Kevin Rue-Albrecht (09:43:22): > Ok. Something that crossed my mind when I looked at the heat map scaffold code is that we might ‘want’ to document a bit our plotting funcctions (create_plot, griddotplot, violinplot, …) about what they expect in and whichcmds[...]they generate. > This way, it would be make it easier to write new functions like the heat map, knowing what ins and outs it should have itself, to match the behaviour of existing functions
Kevin Rue-Albrecht (09:44:26): > From my limited experience with roxygen, I saw that one can write the doc for functions, while omitting the@exporttag, which allows to document functions that are internal to the package
Kevin Rue-Albrecht (09:45:37): > Anyway. It wasn’t really a priority as long we were refactoring the code frequently, but we may have reached a stage where the code is stable enough to be worth the effort
Federico Marini (09:51:44): > these functions will not be exported, but nothing forbids us to document them - probably it is not even needed to do it roxygen-ish style
Federico Marini (09:51:50): > but it would just be a nice addon
Federico Marini (09:52:10): > and or, we are aided in changing/maintaining the code afterwards
Aaron Lun (09:52:28): > I don’t see the need for Roxygen-specific stuff here, though.
Federico Marini (09:52:37): > IMHO it does not have to be ultra verbose, it can just be pieces of info
Federico Marini (09:53:03): > in the codetracking part, there is always a two/3-liner short desc of each function
Kevin Rue-Albrecht (09:53:58): > I don’t want to force an roxygen-style either. We just need to think whether there’s any chance that individual functions may end up exported in the future. In that case, having roxygen-style doc could turn the export on/off by only the@exporttag
Kevin Rue-Albrecht (09:54:28): > (as I understood it)
Kevin Rue-Albrecht (09:55:21): > It’d be interesting to see if users could call the (higher-level) code-generating functions from the CLI rather than having to open the app
Federico Marini (09:55:35): > well I am pro-documenting the unexported funcs too- especially after the rounds of refactoring, it makes life much easier for the three of us 4 that did not write the func itself:slightly_smiling_face:
Kevin Rue-Albrecht (09:56:14): > “What’s the typical template code to draw violin plots given a SCE object and a few arguments”
Kevin Rue-Albrecht (09:56:49): > they may not care about generating the absolutely precise command, just get the scaffold to fiddle with
Kevin Rue-Albrecht (09:57:13): > anyway, I had@Federico Marini’s last point in mind, too
Aaron Lun (09:58:37): > Knock yourselves out, then.
Kevin Rue-Albrecht (09:58:58): > we’ve all contributed bits and pieces everywhere, to the point that i’m not always immediately clear about the current behaviour of code that has changed since I last saw/touched it
Kevin Rue-Albrecht (10:00:12): > I’m expecting us to write bits of doc that will highlight things that we don’t fully understand ourselves individually, where someone else can jump in and clarify that one bit, so that we’re all on the same page
Kevin Rue-Albrecht (10:03:13): > (I’m especially keen to document what gets passed in the...of.create_plot, down to the individual specialised plotting functions, as some of those arguments get handed over 2-3 levels of functions)
Federico Marini (10:04:38): > fine for me - and yes, leaving a trace for someone else (whoever it may be) to pick up and continue developing the func, is what I think is optimal. > We should probably just do it peu à peu while we do the final edits
Kevin Rue-Albrecht (10:04:59): > nice French touch there:wink:
Aaron Lun (10:05:07): > gene selection is fixed in heatmap.
Federico Marini (10:14:54): > well germans use it too - dunno what the official eng translation would be for that
Kevin Rue-Albrecht (10:15:12): > ahhh
Federico Marini (10:15:47): > i thought bitewise was overkilling
Aaron Lun (10:29:21): > OMG it’s beautiful
Aaron Lun (10:29:27): > will push in a second
Kevin Rue-Albrecht (10:29:52): > :smiley:
Federico Marini (10:30:16): > (we knew you@Aaron Lunwould have taken iSEE dev as a battery charger in the fight against ms revision:slightly_smiling_face:)
Aaron Lun (10:30:32): > Oh, that’s almost done.
Aaron Lun (10:30:44): > Just need to run a few things overnight.
Aaron Lun (10:31:11): > And change all my “simple” references to “ordinary”.
Aaron Lun (10:31:16): > For “simple matrix”.
Aaron Lun (10:31:22): > Because apparently that was too confusing for people.
Aaron Lun (10:32:20): > Anyway… I think we should probably center the rows on the heatmap, otherwise we’d just get blobs of highly expressed genes and lowly expressed genes, and nothing about the differences between cells.
Federico Marini (10:33:57): > without row centering I always get ugly plots:stuck_out_tongue:
Federico Marini (10:34:17): > one other thought would be whether to make it Z’d
Aaron Lun (10:34:54): > scaling has never made much sense to me; a more relevant approach would be winsorizing.
Aaron Lun (10:35:18): > As in capping the extremes to preserve dynamic range.
Federico Marini (10:35:42): > well sometimes I think heatmaps in general are nothing but a graphical cheating instrument to support whatever claims one can have:stuck_out_tongue:
Aaron Lun (10:35:48): > Anyway, I’m going off to deal with something, but enjoy the importing functionality.
Federico Marini (10:36:07): > :pray:thank you!
Federico Marini (10:36:30): > unrelated to iSEE, relevant to sc-world: did anyone of you try this?https://github.com/lmcinnes/umap - Attachment (GitHub): lmcinnes/umap > umap - Uniform Manifold Approximation and Projection
Aaron Lun (10:38:35): > Nope, but I’ve seen it.
Aaron Lun (12:02:28): > Almost all empty strings have been cleared. The exception is the row stat table link choices (e.g., when colouring by a linked table), for which an empty string is a valid input when there are no row stat tables.
Kevin Rue-Albrecht (12:27:39): > Well done.
Kevin Rue-Albrecht (12:28:20): > Just coming out of journal club. Give me some time to recover and I’ll pull and have a look this evening
Kevin Rue-Albrecht (13:36:06): > brilliant ‘import features’ button:ok_hand:
Aaron Lun (13:40:12): > Works for both tables and plots.
Kevin Rue-Albrecht (13:40:32): > I tried it on the plot only
Kevin Rue-Albrecht (13:42:50): > eh.. if i ‘receive from table’, and ‘import features’, it still pick the first 100 genes
Aaron Lun (13:43:03): > Well, you need to put in a search string.
Kevin Rue-Albrecht (13:43:37): > ok, i just caught the latest pull, give me a second to build and test again
Federico Marini (13:44:37): > are you also having the issue with the vignette?
Kevin Rue-Albrecht (13:44:53): > lately i’ve built without vignette
Aaron Lun (13:45:38): > The tests have been fucked for a while.
Aaron Lun (13:45:52): > Probably true of the vignette, now that some options cannot be empty strings.
Kevin Rue-Albrecht (13:45:58): > ok.. not sure what’s going on: i typed a search string that reduced the rowTable to 6 genes, but there’s a superfluous gene in the heat map (7), which doesn’t match the string
Federico Marini (13:46:10): > should be something in the setup_memory where it tries the max on aDataFrameif that can help trackin it down
Kevin Rue-Albrecht (13:46:51): > I think it’s the first geneId in the object (scary how i got used to its id)
Aaron Lun (13:47:03): > Yes. Importing will only add to existing entries.
Kevin Rue-Albrecht (13:47:14): > ahhh
Aaron Lun (13:47:16): > You could imagine that people just want to add things.
Kevin Rue-Albrecht (13:47:19): > makes sense
Federico Marini (13:48:02): > very nice work on the heatmap feats
Kevin Rue-Albrecht (13:48:11): > btw, a mechanical reflex made me realise that Ctrl-A works to select and wipe out selectize
Kevin Rue-Albrecht (13:48:35): > no need for a “Clear all” button, if users ask
Aaron Lun (13:50:12): > Sweet, I was just thinking of that.
Federico Marini (13:50:53): > cool trick:slightly_smiling_face:
Federico Marini (13:51:17): > should it import only the one genes matching the string?
Federico Marini (13:51:34): > because for me it still gets also some of the …rik bunch
Aaron Lun (13:52:16): > It’ll add to whatever’s there.
Federico Marini (13:52:32): > ok
Aaron Lun (13:52:36): > Though the cntrl A doesn’t really work for me, not consistently…
Aaron Lun (13:52:46): > I got it to work once, trying to figure out how to do it again.
Federico Marini (13:52:50): > uh
Federico Marini (13:53:01): > creating a new heatmap and deleting it crashes the app
Aaron Lun (13:53:06): > Hm.
Aaron Lun (13:53:19): > Well, deleting the heatmap crashes it.
Federico Marini (13:53:23): > > Warning: Error in as.igraph.vs: Invalid vertex names > No stack trace available >
Federico Marini (13:53:28): > yep
Aaron Lun (13:53:45): > Probably need to add the heatmap to the set of igraph dependencies.
Aaron Lun (13:53:57): > Wait… not sure.
Aaron Lun (13:54:56): > okay, I know what the problem is.
Aaron Lun (13:55:25): > Will patch, but there’s a more general question of whether the heatmap should be included based on its links to the linked gene table plot.
Aaron Lun (13:55:49): > I would say not, because it’s not automatic and you still have to fiddle with the set of genes.
Federico Marini (13:56:03): > General thought: let us pat on our shoulders. I think we made a small contribution in putting a line bw. pre-iSEE and post-iSEE
Federico Marini (13:56:07): > :slightly_smiling_face:
Federico Marini (13:56:28): > fine with not having it automatically in
Federico Marini (13:56:42): > probably only the samples
Federico Marini (13:56:56): > if we brush into it
Aaron Lun (13:57:41): > The problem is that I set a precedent for this sort of thing with the rowDataPlot -> rowStatTable links.
Aaron Lun (13:58:06): > Though that, at least, is automatic.
Aaron Lun (13:59:18): > I mean, the heatmap will eventually receive a brush from a sample-based plot.
Aaron Lun (13:59:58): > So it will eventually fall into the graph.
Aaron Lun (14:00:31): > From a practical perpsective, there is no need for a gene-based link, because it’s not reactive; so I’m inclined to say no.
Aaron Lun (14:19:50): > What’s the deal with plotly? Do you need a plotly account to run it?
Aaron Lun (14:20:01): > Surely not, that would be crazy.
Aaron Lun (14:29:52): > Might be as simple as substituting inrenderPlotlyforrenderPlot, if someone is willing to test it out.
Aaron Lun (14:38:34): > Then we’d get access to the lasso selection.
Kevin Rue-Albrecht (14:54:21) (in thread): > it doesn’t work consistently when you click the field before doing the ‘Ctrl A’ thing?
Aaron Lun (14:55:12) (in thread): > Looks like I have to Ctrl-A twice, for some reason.
Kevin Rue-Albrecht (15:24:52) (in thread): > ok, well i can’t say why, i just tested again, and it works on the first try for me
Kevin Rue-Albrecht (15:38:03): > @Kevin Rue-Albrechtuploaded a file:Preview of subtitle info - File (JPEG): Preview of subtitle info
Kevin Rue-Albrecht (15:39:43): > however, i have the code in the wrong place at the moment, hence the placeholder values
Aaron Lun (16:00:12): > Nice.
Aaron Lun (16:00:26): > I will experiment with plotly tomorrow.
Kevin Rue-Albrecht (16:03:08): > i’m stuck wondering what’s best: passing the plot id or the ‘self brush’ to.create_plot. > I need to evaluate thebrushedPointsbit to know how many points are available, so that I may add thesubtitlebit to the.build_labscall
Kevin Rue-Albrecht (16:03:10): > argh
Kevin Rue-Albrecht (16:04:44): > eh no sorry, my brain is lagging behind
Kevin Rue-Albrecht (16:05:26): > i need to applybrushedPointson the coordinates of the plot itself, therefore after thecmds[["data"]]was evaluated
Kevin Rue-Albrecht (16:06:05): > but.create_plotdoesn’t have the id of the plot being drawn, only the id of the sender plots
Kevin Rue-Albrecht (16:06:45): > anyway, I think I’ll pass down the id of the plot iself tocreate_plotfor now, if it works we can always clean up
Aaron Lun (16:11:57): > That should be in the row names of param_choices.
Kevin Rue-Albrecht (16:12:06): > ahhhhhh
Kevin Rue-Albrecht (16:12:36): > true, now you remind me of how you process self-brushes, testing againsttransmitter
Kevin Rue-Albrecht (16:15:11): > awesome thanks
Kevin Rue-Albrecht (16:35:29): > … after far more stupid typos that I’m willing to share, I think I’m getting to the ~10-20 lines of codes that were actually needed to handle that feature forall plots (except heat map of course)
Aaron Lun (16:35:57): > Note that if we switch to plotly, there will be no brushes.
Kevin Rue-Albrecht (16:36:10): > :sob:
Aaron Lun (16:36:17): > There will only be a general selection.
Aaron Lun (16:36:27): > Because you can’t define a brush with a lasso
Kevin Rue-Albrecht (16:36:32): > yup
Kevin Rue-Albrecht (16:36:36): > let’s see how that goes
Aaron Lun (16:37:01): > This is largely fine - but it will be a pain when trying to reproduce the code, as we’ll have to report all the TRUE/FALSE indices on the page.
Aaron Lun (16:37:20): > Probably we’d need RleLogical or something like that.
Kevin Rue-Albrecht (16:37:36): > what T/F are we talking about here?
Aaron Lun (16:37:43): > Whether it was selected or not.
Kevin Rue-Albrecht (16:37:46): > ouch
Kevin Rue-Albrecht (16:39:32): > Don’t ask me why 317/379=0.8%:shushing_face:
Kevin Rue-Albrecht (16:39:56): > @Kevin Rue-Albrechtuploaded a file:Done - File (JPEG): Done
Kevin Rue-Albrecht (16:41:28): > and the answer is ~6 lines to implement the change (excluding passing down one extra argument down to.build_labs
Kevin Rue-Albrecht (16:45:31): > pushed
Kevin Rue-Albrecht (16:45:42): > enjoy:grin:
Kevin Rue-Albrecht (16:46:44): > (and sorry about the blurry photo, i was too lazy to screenshot and attach)
Kevin Rue-Albrecht (16:47:13): > damn that’s going to be nice for CyTOF data
Aaron Lun (17:03:03): > The plotly lassos are going to be awkward for the violin plots.
Aaron Lun (17:04:40): > We probably need to include all points within the range defined by the selected points.
Kevin Rue-Albrecht (17:49:55): > Alright. Going to sleep now. Curious to hear about how you do with plotly, I don’t have experience with it yet
Federico Marini (17:58:37) (in thread): > also for the no:wink:
Federico Marini (17:59:26) (in thread): > appreciate your putting 42 wherever 42 could fit
Federico Marini (17:59:44) (in thread): > nice usage of the subtitle
2018-02-15
Kevin Rue-Albrecht (03:29:38): > Before I forget, one thing that I’m not 100% happy my implementation last night is that it’s evaluating brushedPoints a second time.
Kevin Rue-Albrecht (03:58:51) (in thread): > hehehe.. thought of your wish to see that value more
Kevin Rue-Albrecht (03:59:12) (in thread): > but it looks even better now that real values are displayed
Kevin Rue-Albrecht (04:02:20): > btw@Aaron Lun, I suppose that you’ll test plotly on a separate branch, to avoid convoluting the heat map work with it? I mean branching offheat, notmasteritself
Aaron Lun (04:07:09): > Yes.
Aaron Lun (09:12:59): > I’d like to seeheatmerged sooner rather than later, it is getting very long.
Kevin Rue-Albrecht (09:13:41): > I was thinking that too. Wasn’t sure who was planning to do what on that branch before it gets merged
Kevin Rue-Albrecht (09:16:06): > Do we want to open a few issues on GH to clarify what needs to be done? > Part of it is integrating it comparably to the other types of plots. > Other part is to clarify how differently it behaves to the other ones (i’m thinking about the brushing aspects, which to my understanding are not in the cards right now, right?)
Aaron Lun (09:17:53): > Yeah sure
Kevin Rue-Albrecht (09:19:22): > my problem these days is that when I sit down in front on iSEE, I barely get the time to identify what needs to be done that I need to get back to something else. Issues would help me kickstart start
Aaron Lun (09:20:12): > Well, for starters, the plot titles get in the way of the plotly control bar.
Charlotte Soneson (09:20:51): > I was working on the heatmaps right now, and I’m getting stuck on the annotations. Basically, the problem is that there should be possibly many annotation bars, and they should all have their own color scale, which doesn’t work very well with ggplot2.
Kevin Rue-Albrecht (09:21:30) (in thread): > do you mean they overlay?
Aaron Lun (09:21:33): > Does it only want one annotation bar?
Aaron Lun (09:21:44) (in thread): > yeah. But there’s bigger problems to fix ATM.
Charlotte Soneson (09:21:48): > Only one color scale
Aaron Lun (09:21:55): > Huh. That’s pretty limiting.
Kevin Rue-Albrecht (09:22:51): > Well, the plot area will also eventually be limiting for the number of annotation bars that can be shown on screen, to be fair
Charlotte Soneson (09:23:30): > Maybe the legends will be limiting first…especially with lots of levels per annotation
Aaron Lun (09:24:46): > Perhaps we should restrict it to one coldata to order by?
Charlotte Soneson (09:25:18): > That would make everything easier:slightly_smiling_face:Ordering is fine btw, it is just the coloring
Charlotte Soneson (09:25:46): > Potentially you could have one continuous and one discrete
Charlotte Soneson (09:26:11): > Or it might be possible if you manually specify the exact color of each annotation level and usescale_color_identity.
Aaron Lun (09:26:28): > That shouldn’t be a problem, that’s what the ECM is for.
Charlotte Soneson (09:27:04): > Yeah…I can try and see if it works, I just saw a post about it
Aaron Lun (09:32:03): > I’m confident I know how to get plotly to work.
Aaron Lun (09:39:22): > It’s still a bit screwy with the ggplot aesthetics, but i can make the underlying shiny work with no issues.
Kevin Rue-Albrecht (09:42:32): > Cool. Happy to see you guys go ahead. Not sure where I can add to that without risking conflicts right now, so I think I’ll let you push some code first and see then how I can help
Kevin Rue-Albrecht (09:42:58): > I might get on that ‘documentation of internal functions’ maybe, in the meantime
Kevin Rue-Albrecht (09:43:34): > except if notable changes are expected, with the plotly udpate?
Aaron Lun (10:07:44): > The problem with plotly is that there is no longer a brushing box. We only get the selection, nothing about the user-drawn box or lasso.
Aaron Lun (10:09:05): > This is usually fine except upon re-rendering, at which point everything resets.
Kevin Rue-Albrecht (10:09:24): > argh
Kevin Rue-Albrecht (10:09:41): > yeah, so you can’t ‘draw’ the selected area anymore, with plotly, right?
Aaron Lun (10:09:45): > nope.
Kevin Rue-Albrecht (10:09:52): > or you’d have to guess it from the selection ^^
Kevin Rue-Albrecht (10:10:11): > like ageom_pathcomputed from the outermost points
Aaron Lun (10:10:40): > I’m not so worried about the path; I’d just like a way of highlighting the selected points.
Aaron Lun (10:11:12): > More generally, there’s no way to get persistence for selections in the plotly navigation bar. Oh well.
Aaron Lun (10:14:16): > That’s probably less of a concern, TBH.
Kevin Rue-Albrecht (10:24:27) (in thread): > before I forget, there may be ways to move the title or the plotly bar around, usingggplot::theme
Kevin Rue-Albrecht (10:24:57) (in thread): > but I’ll look at this later, once the bigger problems are resolved
Aaron Lun (10:29:41): > Hm. The plotly mode bar itself resets upon replotting, not just re-rendering.
Aaron Lun (10:29:54): > Which makes chained brushing really inconvenient.
Kevin Rue-Albrecht (10:31:33): > ugh
Federico Marini (10:58:27): > my couple of cents on the “big picture”: if the brush-linking does not work, it does not feel anymore like the “original concept” - which I digged pretty much, and I think many others did too:slightly_smiling_face:
Aaron Lun (10:58:35): > Basically, for plotly to work, we’d have to write another function (iSEE_ly?) that has fixed panel sizes and positions that are specified upon initialization of the app.
Aaron Lun (10:59:53): > Brushing by restriction would mostly be okay, but not for violin plots, where replotting is unavoidable.
Aaron Lun (11:00:20): > I mean, the major advantages of plotly over standard shiny is the ability to pan and lasso.
Aaron Lun (11:02:03): > So I wonder whether it is possible to get that functionality somehow else.
Aaron Lun (11:02:45): > Hell, we could just rip the lasso selection from plotly.js.
Aaron Lun (11:02:57): > And bind it to shiny ourselves.
Aaron Lun (11:06:17): > Nick it fromscatterD3 , just like we took the collapsible box fromshinyBS .
Kevin Rue-Albrecht (11:09:31): > in all those cases, the brush would then return the data points selected rather than the definition of the brush, right?
Aaron Lun (11:09:41): > unfortunately, yes.
Federico Marini (11:10:00): > I have to say, the only thing I really would like to have is the lasso selection - the hover info is nice and so, but the lasso is damn cool
Aaron Lun (11:10:23): > This is possible with D3.
Aaron Lun (11:10:35): > https://github.com/skokenes/D3-Lasso-Plugin - Attachment (GitHub): skokenes/D3-Lasso-Plugin > D3-Lasso-Plugin - A lasso selection tool for D3
Aaron Lun (11:10:46): > The key is just binding it to shiny, which is done inscatterD3 .
Aaron Lun (11:12:55): > We could even home-brew a lasso, using repeated clicks.
Aaron Lun (11:13:11): > That would be the best way to store all of the relevant information.
Kevin Rue-Albrecht (11:13:27): > well, i’m not too sentimental with having the brush definition: it’s probably even better if we compute it based on the selected point (it would add a bit of smoothing to the clunky manual selection), plus that time could come from the time saved counting the number of point in the brush (for my subtitle thing)
Aaron Lun (11:15:02): > Actually, I like the repeated clicks idea a lot. We can even update it very quickly by simply adding layers to the existing ggplot object, rather than doing a whole new evaluation.
Aaron Lun (11:15:32): > At each click, the plot will add a way-point; clicking back to the start will close the loop.
Aaron Lun (11:15:39): > Neat, huh.
Kevin Rue-Albrecht (11:15:46): > if possible, yeah!
Aaron Lun (11:15:54): > I’ll check it out this WE.
Aaron Lun (11:16:04): > Easier than refactoring it all for plotly.
Federico Marini (11:16:05): > this sounds cool
Kevin Rue-Albrecht (11:16:05): > how would the app know when the user is done clicking?
Aaron Lun (11:16:13): > When you close the loop.
Kevin Rue-Albrecht (11:16:19): > oh damn you said it
Aaron Lun (11:16:33): > Going to have a walk and think about this while my HDF5 timings run forbeachmat .
Aaron Lun (11:16:40): > Slows down all my IO.
Aaron Lun (11:17:57): > In any case, I’ve manually implemented a lasso before (in Javascript, no less!) so it shouldn’t be too hard.
Kevin Rue-Albrecht (11:18:32): > ah well, if you talk from experience, then i’ve got nothing to add:sweat_smile:
Aaron Lun (11:19:09): > The key is to make sure it’s as responsive as possible.
Aaron Lun (11:19:51): > Okay, going for my walk now.
Aaron Lun (12:10:56): > In the meantime, perhaps someone can figure out where the code spends most of its time when rendering the plot. I’d like to know whether it’s during the creation of theggplot object, or in the final step where the ggplot object is actually shown.
Aaron Lun (12:27:01): > If it’s the final step, then… well, nothing we can do. But if it’s all the steps leading up to and including the creation of the ggplot object, then there’s a chance to avoid all this work when we’re just adding a brushing box or lasso points.
Kevin Rue-Albrecht (12:27:41): > absolutely. worth a look
Aaron Lun (12:28:18): > In the meantime, I have to figure out howbeachmat is being faster than theoretically possible.
Kevin Rue-Albrecht (12:48:46): > > function(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10){ > return(42) > } >
Aaron Lun (14:18:18): > Finally finished my head-bashing session withscran ’s unit tests.
Aaron Lun (14:18:22): > Ontobeachmat again.
Aaron Lun (14:43:46): > Well, I just gave up on that. It wasn’t clear what was going on.
Kevin Rue-Albrecht (15:00:24) (in thread): > i’m just thinking back when i wrote and manually ran the unit tests: the code-writing functions seemed pretty fast, so i’m wondering whether it’s the rendering that’s slowing things down. I’m trying to look at it now
Kevin Rue-Albrecht (15:30:06): > what’sforce(rObjects[[plot_name]])for iniSEE.Ragain?
Kevin Rue-Albrecht (15:32:31) (in thread): > Anyway, it definitely doesn’t look like it’s in the plotting code. Although that was kind of obvious from the start considering that the slowdown is at boot time, and that plots refresh pretty fast once the app is up and running
Charlotte Soneson (16:17:50): > So, I have added ordering and coloring by annotations to the heatmaps (still haven’t managed to put a label on the annotations though), but I can’t test it since I just realized that I updatedDelayedArrayand nowscaterdoesn’t load…(it assumes thatArrayRegularGridis exported byDelayedArray, which it does not seem to be anymore). I am almost falling asleep so I will not solve it tonight, but I can push it to theheatbranch and rely on Travis for testing if someone else wants to have a look tomorrow…
Aaron Lun (16:35:38): > Uh.scater shouldn’t depend on ArrayRegularGrid.
Aaron Lun (16:36:16) (in thread): > Trigger replotting in response to updates to the reactive value with the same name as the plot.
Kevin Rue-Albrecht (16:36:58): > separately, i’m partially through time-stamping the initialisation of the app server-side, and all the way to the panel observers takes a mere ~2s so far
Aaron Lun (16:37:35) (in thread): > The question is whether or not we would benefit from have a stored ggplot object; then, when we get a new brush, we retrieve the stored object and just add a layer withgeom_rect. Would this be appreciably faster than re-rendering the entire plot?
Kevin Rue-Albrecht (16:39:17): > (getting to the brush observers now)
Kevin Rue-Albrecht (16:40:01): > nope, lighting fast too
Kevin Rue-Albrecht (16:45:15) (in thread): > well, it doesn’t seem to me like rendering plots is the issue, time-wise
Kevin Rue-Albrecht (16:48:14) (in thread): > ok thanks
Kevin Rue-Albrecht (16:48:59) (in thread): > i just got again to the dot-related plotting section, gets processed in less than a second too
Kevin Rue-Albrecht (16:50:00) (in thread): > i printed a message through each iteration of the loop: they all get printed within a second, while the app lag still follows all those messages
Kevin Rue-Albrecht (16:51:59) (in thread): > now i’m at the end: row table and heat map sections
Kevin Rue-Albrecht (16:55:04) (in thread): > ok.. must come from the UI side: i’ve timestamped all the server-side of the app and all computations are complete ~6s before the plots all show simultaneously
Aaron Lun (17:01:44) (in thread): > Maybe we need to test on a heavier example. TCGA?
Kevin Rue-Albrecht (17:07:52) (in thread): > I want indeed to test my freshly added progress bar on the TCGA next. > However, for the purpose of understanding why there is a 5s gap between the completion of.panel_generation(that feeds intooutput$allPanels) and the plots showing, there is no need for a data set larger than theallenone
Kevin Rue-Albrecht (17:08:54) (in thread): > right now, without knowing the details of how Shiny stacks the UI operations, I think it’s just about Shiny-side initialisation of the various plotting panels
Kevin Rue-Albrecht (17:09:48) (in thread): > if you like, I’ll push theprogressbarbranch that I’ve created, and includes all the timestamped console messages
Kevin Rue-Albrecht (17:10:33) (in thread): > so that you can see for yourself, because it’s a bit hard to picture it in Slack
Kevin Rue-Albrecht (17:14:20) (in thread): > there you goprogressbarpushed
Kevin Rue-Albrecht (17:18:51) (in thread): > I just tested on the TCGA and it’s ridiculously fast (the progress bar)
Aaron Lun (17:19:09) (in thread): > Yes, progress bars are basically useless.
Aaron Lun (17:19:29) (in thread): > Just to be clear; once the plots are set up, what’s the lag upon updating a plot?
Kevin Rue-Albrecht (17:20:05) (in thread): > thinking about it, I actually quite like seeing the progress bar flash through: it’s kind of patting ourselves on the back
Kevin Rue-Albrecht (17:20:32) (in thread): > might discourage a few users of complaining of any tiny lag
Kevin Rue-Albrecht (17:21:10) (in thread): > well, i wouldn’t call a ‘lag’ the spit second that it takes toupdate the plots
Aaron Lun (17:21:17) (in thread): > Putting that aside. What’s the relative time required to make a ggplot object compared to printing it?
Kevin Rue-Albrecht (17:21:24) (in thread): > the only lag that I can see is to initialise them
Aaron Lun (17:21:38) (in thread): > Hm.
Kevin Rue-Albrecht (17:21:38) (in thread): > the ggplot objects are lighting fast to make
Kevin Rue-Albrecht (17:21:55) (in thread): > it’s the displaying or writing to file that takes most time
Kevin Rue-Albrecht (17:22:07) (in thread): > because a ggplot object is just a bunch of instructions
Kevin Rue-Albrecht (17:22:23) (in thread): > those instructions only seem to get evaluated at render/write time
Aaron Lun (17:22:34) (in thread): > Well, sometimes the dataframe mangling can take some time if htere’s a lot of points.
Aaron Lun (17:23:00) (in thread): > And one could iamgine reevaluation of the brushes would also be a pain.
Kevin Rue-Albrecht (17:23:11) (in thread): > do you mean the manglinginside ggplot (I can’t imagine any right now), or ours?
Aaron Lun (17:23:25) (in thread): > Ours.
Aaron Lun (17:23:35) (in thread): > And calculation of the jitter, etc.
Aaron Lun (17:24:07) (in thread): > plotly achieves its speed by translating the plot to JSON and then updating only the relevant layer.
Kevin Rue-Albrecht (17:24:12) (in thread): > hm.. you make me wonder whenour manglingactually happens
Kevin Rue-Albrecht (17:25:27) (in thread): > when does ourevalactually happens, you think?
Aaron Lun (17:25:52) (in thread): > WEll, yes, I would assume.
Kevin Rue-Albrecht (17:27:43) (in thread): > because i timed all the server-side calls, and even thepanel_generationthat’s the last thing in the UI. All of those things produce their console message ~5s before the plots initialise for the first time
Kevin Rue-Albrecht (17:28:48) (in thread): > i don’t see any later place to put a timestamp between my latest timestamp and the moment plots initialise
Kevin Rue-Albrecht (17:35:47) (in thread): > regarding your point aboutplotly: does it boot faster than our current app? or is it just a matter ofupdating plots after they’re initialised?
Aaron Lun (17:36:08) (in thread): > updating plots. I’m not worried about booting.
Kevin Rue-Albrecht (17:37:12) (in thread): > Ahh ok then. Well i’m not so worried about updating to be honest. I find the current app fast enough as it is, even for the TCGA
Aaron Lun (17:37:19) (in thread): > If it’s actually printing the plots that soaks up time after the.make_redDimPlot(etc.) call, then there’s nothing we can do. But if it’s taking a few fractions of a second for the function to return, that suggests we can cut down time and improve responsiveness by caching the ggplot object.
Aaron Lun (17:41:53) (in thread): > In particular, cache the object and only update the brushing box - this would be pretty simple. The same principle could be used fo the lasso waypoints.
Aaron Lun (17:43:03) (in thread): > Or more generally, we could separate out the functions to constructplot.datafrom the generation of theggplot object, which would save us a few milliseconds.
Kevin Rue-Albrecht (17:43:11) (in thread): > yeah updating only the brush layer makes sense
Aaron Lun (17:44:20) (in thread): > Though I guess printing out the plot takes so much longer, setup time is mostly negligible.
Kevin Rue-Albrecht (17:44:25) (in thread): > the ggplot object, minus the brush, can be cached, and then the brush layer is just added as agg_object+brush_layerat the end
Aaron Lun (17:44:40) (in thread): > Yes, that would be the idea.
Aaron Lun (17:45:11) (in thread): > Could you just give me an idea of how long it takes to return from the function, vs. how long it takes to show the plot?
Kevin Rue-Albrecht (17:45:11) (in thread): > well, we can try that at some point, but I guess it’s still relatively low priority given the current performance
Kevin Rue-Albrecht (17:48:21) (in thread): > hmm.. i’d say the progress bar flashes through in 0.5s, and the plot takes maybe up to a second to refresh
Kevin Rue-Albrecht (17:48:45) (in thread): > (allen data)
Aaron Lun (17:55:04) (in thread): > Try timing outside of the app using a similar approach totest_plotting.R.
Aaron Lun (17:55:25) (in thread): > Even half a second is worth saving when users are expecting an immediate response.
Aaron Lun (17:56:21) (in thread): > Basically,system.timethe actual.make_redDimPlotcall, and alsosystem.timethe save to file viapng(which I presume is what shiny is doing under the hood).
Kevin Rue-Albrecht (18:00:47) (in thread): > yeah it’s probably doing that to a temp file somewhere
Kevin Rue-Albrecht (18:01:10) (in thread): > i’ll have a look, probably rather this weekend better than tonight
Aaron Lun (18:06:02) (in thread): > My guess is that probably DelayedMatrixStats is depending on it, which causes its reverse dependencies to fail as well.
Kevin Rue-Albrecht (18:33:40) (in thread): > i’m probably just too tired, but i get a error when I try to use the setup code in my unit tests: > > > all_memory <- iSEE:::.setup_memory( > + sce, redDimArgs, colDataArgs, featExprArgs, rowStatArgs, rowDataArgs, heatMapArgs, > + redDimMax = 1, colDataMax = 1, featExprMax = 1, rowStatMax = 1, rowDataMax = 1, heatMapMax = 1) > Error in max(all_maxes[[x]], all_args[[x]]) : > invalid 'type' (S4) of argument >
Kevin Rue-Albrecht (18:34:02) (in thread): > seems to be at the lineall_maxes[[x]] <- max(all_maxes[[x]], all_args[[x]])
Kevin Rue-Albrecht (18:34:36) (in thread): > going to sleep now
2018-02-16
Charlotte Soneson (04:03:32): > Ok, I just pushed the ordering/coloring update to the heatmaps. I don’t know how we want to do with the annotation bars: since I defined the colors manually in order to be able to have multiple annotations, there are no colorbars generated automatically. It may take a lot of space to have them all, but it is difficult to interpret without them.
Kevin Rue-Albrecht (04:04:07): > well, that’s up to the user, how much information they want to display
Kevin Rue-Albrecht (04:04:34): > how does it look when you strech the heatmap panel to say the whole screen width?
Kevin Rue-Albrecht (04:04:57): > does it give enough extra space to add more colorbars I mean?
Charlotte Soneson (04:06:51): > Yes. I don’t know what’s happening to the color bars when I do that though, it looks ugly…perhapsgeom_segmentwasn’t a good way to go:tired_face:
Charlotte Soneson (04:08:00): > Or, well, I can see what’s happening, but it doesn’t look good
Charlotte Soneson (04:10:09): > That was a stupid choice
Kevin Rue-Albrecht (04:10:24): > ok. I don’t have much experience withgeom_segmentso I’ll have to look at it to see what you mean
Charlotte Soneson (04:10:50): > It is actually drawing a line for each sample
Charlotte Soneson (04:10:58): > A diagonal one, while it should make a box
Kevin Rue-Albrecht (04:11:06): > from the sound of it, I understand…. ah, exactly what you just wrote
Charlotte Soneson (04:11:07): > It looked nice until you zoom in
Charlotte Soneson (04:11:20): > I just was’t thinking far enough
Kevin Rue-Albrecht (04:12:29): > hm.. I know that ggplot only allows one colour scale per plot, but we can’t be the first ones who wish to have colour annotation on the side of heat map
Kevin Rue-Albrecht (04:13:34): > again, I’m biased towardComplexHeatmaponly by lack of experience withpheatmap, butComplexHeatmapmakes it really easy to have adata.frameof colour annotations. I suspectpheatmapcan do that too?
Charlotte Soneson (04:14:31): > Yes
Charlotte Soneson (04:16:34): > geom_rectinstead ofgeom_segmentworks, but then we run into the same problem since it needs afill, and thefillscale is already used for the heatmap itself…
Kevin Rue-Albrecht (04:17:57): > I wonder whetherfacets(2x2) could be hacked to have a panel with row annotations to the right, and a panel with column annotations at the top. But again, that sounds far from optimal
Charlotte Soneson (04:21:23): > We would still have the problem with only one color/fill scale.
Kevin Rue-Albrecht (04:21:39): > yeah.. because it’s allone plot
Charlotte Soneson (04:21:53): > Otherwise we would have to combine different plots, which would cause issues with alignment
Kevin Rue-Albrecht (04:23:07): > mhhh, i’ve been googling the last 10min and I can’t find any example of ggplot with side annotations
Kevin Rue-Albrecht (04:23:32): > only grid-based stuff
Charlotte Soneson (04:24:38): > Actuallycowplotcan be used to align plotting regions, perhaps it is an option to generate separate plots for each annotation bar
Kevin Rue-Albrecht (04:26:28): > That could be an option. Still haven’t taken the time to learncowplot. Happy to see someone test out the idea on a new branch
Kevin Rue-Albrecht (04:28:11): > from the looks of it (https://cran.r-project.org/web/packages/cowplot/vignettes/introduction.html ), this might work out with a bit of fine-tuning: > > plot2by2 <- plot_grid(plot.mpg, NULL, NULL, plot.diamonds, > labels=c("A", "B", "C", "D"), ncol = 2) >
Charlotte Soneson (04:29:23): > This is what I got with a quick test
Charlotte Soneson (04:29:30): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-16 at 10.28.52.png - File (PNG): Screen Shot 2018-02-16 at 10.28.52.png
Kevin Rue-Albrecht (04:29:50): > damn i’m pretty happy with that
Kevin Rue-Albrecht (04:30:16): > i’m sure we can squish the top annotation with some relative panel width magic
Charlotte Soneson (04:30:33): > Sure, you just setrel_heights
Kevin Rue-Albrecht (04:31:40): > ahhhhh… all those functions are making@Aaron Lun’s own magic superfluous these days:wink:
Kevin Rue-Albrecht (04:33:13): > while you’re fiddling with options, I’d suggest havingtheme(legend.position="top")for the column annotation panel,theme(legend.position="bottom")for the data, andtheme(legend.position="right")for the row annotation panel
Charlotte Soneson (04:33:36): > Ok. I’ll just push the version withgeom_rect()instead ofgeom_segment()toheatand then I’ll branch off that and try out cowplot.
Kevin Rue-Albrecht (04:33:58): > it’s a small price to pay not having all the legends in a single place, but it may actually help readability
Aaron Lun (04:52:00): > Huh?
Kevin Rue-Albrecht (04:52:20): > which part?
Aaron Lun (04:52:31) (in thread): > whoops, that should probably say nrow(all_args[[x]]).
Kevin Rue-Albrecht (04:52:32): > (i was just teasing about magic and wizard)
Aaron Lun (04:52:46) (in thread): > Not sure how that managed to get past. It ran fine for me.
Aaron Lun (04:53:01): > What is superfluous?
Kevin Rue-Albrecht (04:54:15): > nah, nothing superfluous, I just meant that this function may avoid the need to call upon home-brewed magic
Aaron Lun (04:54:36): > for the lasso?
Kevin Rue-Albrecht (04:54:55): > eh.. no, to align the heat maps
Kevin Rue-Albrecht (04:56:00): > basically, i just meant that for other situations, like the lasso, we’ll definitely need your magic, while in this case the nice and friendly functionrel_heightsdoesn’t require it
Aaron Lun (04:56:10): > oh
Kevin Rue-Albrecht (04:56:17): > ‘superfluous’ was clumsy of me
Aaron Lun (05:22:49): > Just saw this PIVOT paper on my google scholar feed:https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1994-0 - Attachment (BMC Bioinformatics): PIVOT: platform for interactive analysis and visualization of transcriptomics data > Many R packages have been developed for transcriptome analysis but their use often requires familiarity with R and integrating results of different packages requires scripts to wrangle the datatypes. Furthermore, exploratory data analyses often generate multiple derived datasets such as data subsets or data transformations, which can be difficult to track. Here we present PIVOT, an R-based platform that wraps open source transcriptome analysis packages with a uniform user interface and graphical data management that allows non-programmers to interactively explore transcriptomics data. PIVOT supports more than 40 popular open source packages for transcriptome analysis and provides an extensive set of tools for statistical data manipulations. A graph-based visual interface is used to represent the links between derived datasets, allowing easy tracking of data versions. PIVOT further supports automatic report generation, publication-quality plots, and program/data state saving, such that all analysis can be saved, shared and reproduced. PIVOT will allow researchers with broad background to easily access sophisticated transcriptome analysis tools and interactively explore transcriptome datasets.
Aaron Lun (05:22:57): > Seems like they’re trying to be a galaxy replacement.
Kevin Rue-Albrecht (05:25:07): > something to pencil down as one of our own references, I guess
Kevin Rue-Albrecht (05:27:15): > i just pasted it in there:https://docs.google.com/document/d/1MW3MZQzuxkUBm795o-Rmm1to7VS5lx6FRT6xvhdK8RM/edit?usp=sharing
Kevin Rue-Albrecht (05:27:17): > @Kevin Rue-Albrechtshared a file:iSEE_manuscript_draft - File (Google Docs): iSEE_manuscript_draft
Aaron Lun (05:37:47) (in thread): > Is this fixed? Or should I do it?
Aaron Lun (05:48:47): > The heatmap looks nice, but it would be nice to get some annotations. I’m actually happy if you just have the name of the covariate on the heatmap; we can generate another plot containing the legend for all of the colours.
Aaron Lun (05:49:02): > A legend plotwithin the “column data parameters” panel.
Charlotte Soneson (05:49:46): > I think the naming will work nicely withcowplot
Aaron Lun (05:50:02): > Okay.
Charlotte Soneson (05:50:15): > And also the filling of the annotation bars, currently it is only a colored frame around grey boxes
Aaron Lun (05:50:49): > Okay. Well, I’ll leave that to you, then.
Kevin Rue-Albrecht (05:52:46) (in thread): > It’s not fixed yet. I just spotted it before going to sleep. I was too tired and confused to think about fixing it right then
Aaron Lun (05:54:29) (in thread): > Well, it’s done.
Kevin Rue-Albrecht (05:54:41) (in thread): > nice. thanks!
Aaron Lun (05:56:44): > I’m going to refactor the internals so that it saves the ggplot objects from the scatter plots. This should make it a lot easier to add the brushes and waypoints for the lasso, as we can just add layers on top rather than going deep intoplotting.R.
Aaron Lun (06:14:46): > Not quite as easy as I thought, as you’d need to cache the commands as well…
Kevin Rue-Albrecht (06:15:32): > maybe i don’t understand, but aren’t the command cached, to show up in the tracker?
Kevin Rue-Albrecht (06:15:51): > or you mean caching the partial commands on top of the full ones?
Aaron Lun (06:16:28): > yes.
Kevin Rue-Albrecht (06:16:50): > note that one of the unspoken reasons why i initially started storing commands by section and names would be that they could be sliced and subsetted
Kevin Rue-Albrecht (06:17:41): > although that requires knowing deterministically what to expect in any one list of commands
Aaron Lun (06:17:54): > I mean, we could really break it up, right.
Aaron Lun (06:18:16): > We could have data generation observers; ggplot generation obsevers; and “extra” stuff observers.
Kevin Rue-Albrecht (06:18:38): > Not sure if we still have some ‘optional’ commands in there. I seem to remember some commands set toNULLtonot appear in the list of commands
Kevin Rue-Albrecht (06:19:28): > at the moment, we’ve been ‘stashing’ commands in there, but I guess a spring cleaning in there was only a matter of time
Charlotte Soneson (07:16:52): > Nicer looking heatmaps are now in thecowplotbranch
Aaron Lun (07:42:02): > cool
Aaron Lun (07:42:16): > From my end, I’ve spun out the brushing box.
Aaron Lun (07:42:27): > so changing the brush doesn’t cause a full re-evaluation of the plot.
Charlotte Soneson (07:42:37): > Nice! There is a weird thing happening with theselectizeInput(both for gene and annotation selection): the memory doesn’t seem to update when all genes (or annotations) are removed. Any idea why that would happen?
Aaron Lun (07:43:03): > A known issue. The observer will not respond to NULL inputs for selectize, which is what happens when you delete all inputs.
Charlotte Soneson (07:43:10): > Ah, ok
Aaron Lun (07:43:20): > This is a consequence of shiny’s happy-go-lucky way of indicating whether an input was specified or not.
Aaron Lun (07:43:38): > I can’t tell whether a NULL is because you deleted all genes, or whether it was because the input hasn’t been initialized yet, etc.
Aaron Lun (07:44:14): > If I don’t provide NULL protection, the plot will be rendered twice at start up.
Aaron Lun (07:44:49): > … which might not be so bad, as it would just flash with the “Invalid plot” message.
Aaron Lun (07:47:48): > @Kevin Rue-AlbrechtCan I also suggest displaying the % of points brushed as a UI element below each plot? This would avoid unnecessary re-evaluation.
Aaron Lun (07:49:54): > The brushing currentlyseems more responsive… though it’s hard to tell.
Aaron Lun (07:50:34): > Also, it doesn’t work for horizontal violin plots, as the brush box needs to be transposed and that information isn’t available at theiSEElevel.
Aaron Lun (07:51:37): > Is it possible to tell whethercoord_fliphas been run, by looking at theggplot object?
Aaron Lun (07:59:22): > … and the answer is yes, with copious applications of dark magic.
Aaron Lun (07:59:38): > > is(gg$coordinates, "CoordFlip") >
Kevin Rue-Albrecht (08:52:31) (in thread): > Absolutely, i just thought that the plots were getting re-rendered anyway because of the brush being re-drawn. > However, if the brush is spun out, then it makes sense to spin out that info to a UI output
Kevin Rue-Albrecht (08:53:45): > lol. nice
Aaron Lun (08:58:11): > geom_pathdoesn’t seem to do anything.
Aaron Lun (08:59:31): > How do I draw some damn lines?
Kevin Rue-Albrecht (09:01:04): > well, not that I have used it much but as I understand it,geom_pathwants a data frame ordered from the first to the last point to connect, with x and y columns
Kevin Rue-Albrecht (09:01:18): > how do you have the data?
Aaron Lun (09:01:35): > yes, that’s how it’s set up.
Kevin Rue-Albrecht (09:01:41): > :confused:
Kevin Rue-Albrecht (09:02:27): > ```
Aaron Lun (09:02:30): > Ah, I know why. It’s because I set alpha =0
Aaron Lun (09:02:35): > copying from brushing box.
Kevin Rue-Albrecht (09:02:49): > Ah. You’re onto something there I think ^^
Aaron Lun (09:04:41): > Preliminary lasso support on “spinout”
Kevin Rue-Albrecht (09:05:02): > woooow, can’t wait to see that
Aaron Lun (09:05:27): > Currently you can’t close the lasso yet; and it does strange things if you try to brush at the same time.
Aaron Lun (09:06:19): > Also, double-clicking no longer seems to work…
Aaron Lun (09:06:23): > to zoom, I mean.
Aaron Lun (09:07:50): > Ah… I know why.
Kevin Rue-Albrecht (09:21:43): > I like how the lasso appears, in any case
Aaron Lun (09:23:02): > I’d like start and end dots, to help people figure out where they’re brushing from.
Kevin Rue-Albrecht (09:23:43): > hm, add aSizeBythat’s larger for the first point?
Kevin Rue-Albrecht (09:24:41): > Later we can add a ColorBy(PairedDarkColor)/FillBy(PairedLightColor) to emphasise it even more
Aaron Lun (09:25:08): > Filling might not be straightforward for complex polygons.
Kevin Rue-Albrecht (09:25:21): > sorry i meant filling the points
Aaron Lun (09:25:24): > oh
Aaron Lun (09:25:36): > In any case, I need to figure out how to avoid the brush and the click fighting each other.
Kevin Rue-Albrecht (09:25:41): > i’m making up a dummy data.frame right now for illustration
Aaron Lun (09:26:01): > Or you can just modify §plotting.R§
Kevin Rue-Albrecht (09:26:21): > if (!is.null(brush)){inactivePolygone<- TRUE}?
Kevin Rue-Albrecht (09:26:49): > this way people would have to clear the brush before they can lasso
Kevin Rue-Albrecht (09:27:13): > and if they brush, it clears the lasso
Aaron Lun (09:27:44): > Yes, we could do that.
Kevin Rue-Albrecht (09:28:11): > I can absolutely play with plotting.R, just let me know whenever you’re done with the prototype and I’ll fiddle with the aesthetic part
Aaron Lun (09:28:19): > Yes, please do.
Aaron Lun (09:28:31): > I will wonder off and think about how to get this to work.
Aaron Lun (09:33:12): > Hm. What about clicking anywhere outside the plotting area will close the lasso (if open) and will destroy the lasso (if closed).
Aaron Lun (09:33:41): > Sort of like those old shooting games where you have to shoot outside the screen to reload.
Aaron Lun (09:33:42): > TIME CRISIS.
Kevin Rue-Albrecht (09:34:34): > hahaha nice one too
Kevin Rue-Albrecht (09:34:51): > btw, I meant I can look at it this evening, not right now
Kevin Rue-Albrecht (09:37:05): > I took a minute to look at the code, and I can see where you add points, but I’d like to spend focused time on that to do things cleanly
Aaron Lun (09:41:13): > ok
Aaron Lun (09:48:06): > Hm… seems like clicking outside the plot area is a bit buggy.
Kevin Rue-Albrecht (10:04:32): > Finally got the dummy example right:
Kevin Rue-Albrecht (10:04:36): > > library(ggplot2) > library(RColorBrewer) > > display.brewer.all() > pairedColors <- brewer.pal(2, "Paired")[1:2] > > sizeScale <- c(4,1); names(sizeScale) <- c(TRUE, FALSE) > fillScale <- pairedColors; names(fillScale) <- c(TRUE, FALSE) > > df <- data.frame( > x = c(1,2,3,1), > y = c(1,2,2,1), > first = c(TRUE, rep(FALSE, 2), TRUE) > ) > ggplot(df, aes(x,y)) + > geom_path(colour = pairedColors[2]) + > geom_point(aes( > size = first, > fill = first), > stroke = 1, shape = 21, colour = pairedColors[2] > ) + > scale_size_manual(values = sizeScale) + > scale_fill_manual(values = fillScale) >
Kevin Rue-Albrecht (10:05:11): > now i just have to spend the time this evening to plug it into your lasso code
Aaron Lun (10:09:42): > I’d like something like a closed circle for the starting point of the lasso path, and an open circle for the current location.
Kevin Rue-Albrecht (10:10:16): > thanks, exactly the feedback i hoped for from this example
Kevin Rue-Albrecht (10:10:42): > although what do you call open/close?
Kevin Rue-Albrecht (10:11:10): > you mean the shape, or the fill?
Aaron Lun (10:11:14): > as in shape.
Aaron Lun (10:11:34): > Just something to indicate where the lasso begins (because you need to click the starting point to close the lasso)
Aaron Lun (10:11:44): > and something to indicate where you’re currently at (just in case you get lost).
Kevin Rue-Albrecht (10:12:07): > yep i know, i’m just trying to picture which shape exactly is an “open” circle
Aaron Lun (10:13:02): > Okay; lasso behaviour is now more-or-less stable.
Kevin Rue-Albrecht (10:13:31): > > max_shape <- 25 > plot(1:max_shape,rep(1,max_shape),pch = 1:max_shape,cex =2) >
Kevin Rue-Albrecht (10:13:50): > > please make your selection: > $ >
Aaron Lun (10:14:14): > Note that once you have a closed lasso, any further click will destroy it. And then the next click will start a new lasso.
Kevin Rue-Albrecht (10:14:29): > agreed
Aaron Lun (10:21:35): > And something visual to indicate that it’s closed would be nice. So maybe a change in the line style or something (from dashed to solid).
Aaron Lun (10:22:08): > Happy to usegeom_polygonas well, with the specifiedbrush_fill_color.
Kevin Rue-Albrecht (10:36:32): > hmm, yep, excellent idea the dash-to-solid transition
Kevin Rue-Albrecht (10:36:42): > easy to do, outside theaes
Aaron Lun (10:37:21): > I actually think the polygon would be nicer, mimics what happens with the brush a bit more.
Aaron Lun (10:37:32): > As in, only show the filling when the lasso is closed.
Kevin Rue-Albrecht (10:37:50): > ohhh, I get you
Kevin Rue-Albrecht (10:38:24): > so no need for the dash-solid transition (we can revisit that as a later bonus)
Kevin Rue-Albrecht (10:38:49): > oh my.. that’s gonna look good:slightly_smiling_face:
Aaron Lun (13:00:38): > There’s at least one problem; self brushing to restrict with lasso will not work unless we find a way to intersect polygons.
Aaron Lun (13:00:50): > Presumably there’s R functions to do that, so will have to dig into it.
Aaron Lun (13:01:29): > Also, tables won’t find it easy to receive lassos.
Aaron Lun (13:01:51): > We’ll have to create a dummy field*_Selected_*and fill it with TRUEs or FALSEs depending on whether it was selected or not.
Aaron Lun (13:02:01): > Though that’s an arguably better approach than what we’re doing now, anyway.
Aaron Lun (13:14:43): > Brushing by lasso is now supported from reduced dimension plots. It’s fairly easy to do this for every other plot, if someone wants too do it; I’m pretty tired now.
Kevin Rue-Albrecht (13:27:41): > i’m going to pull and look at it now. Good work anyways
Aaron Lun (13:29:01): > double click also clears the lasso if one is present. Otherwise it will zoom out.
Aaron Lun (13:30:19): > ALso note that mgcv::in.out uses a boundary crossing algorithm, which will give unintuitive (wrong?) results for complex polygons where the sides intersect. Though arguably any such cases are stupid and shouldn’t be specified.
Kevin Rue-Albrecht (13:31:02): > hm.. i wonder if there’s an easy way to clear polygons whose sides intersect
Aaron Lun (13:31:34): > What do you mean?
Kevin Rue-Albrecht (13:32:23): > detect when a polygon, even unfinished, has sides that intersect, and just clear the whole polygon as soon at this happens
Aaron Lun (13:32:39): > Probably not worth it.
Kevin Rue-Albrecht (13:32:47): > it’s probably over the top yeah
Kevin Rue-Albrecht (13:32:59): > especially if users can clear with dbl click
Kevin Rue-Albrecht (13:33:20): > we can’t always clear people’s mess
Kevin Rue-Albrecht (13:34:38): > … how precise did you set the lasso closing click?
Kevin Rue-Albrecht (13:35:19): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 18.35.07.png - File (PNG): Screenshot 2018-02-16 18.35.07.png
Kevin Rue-Albrecht (13:35:36): > or better asked, is there a way to control how relaxed it is?
Aaron Lun (13:35:50): > Yes.
Kevin Rue-Albrecht (13:36:26): > for example i had to add the last little bit of click here to finish up the lasso (although the first one was a bit inaccurate of me)
Kevin Rue-Albrecht (13:37:14): > anyway, don’t worry about it for now, it’ll become easier to decide once i add the open/closed circles with a certain diameter
Aaron Lun (14:03:19): > Currently it’s 2% of the plot range.
Aaron Lun (14:10:02): > I think we should just ban people from using restrict with a self-brush. This is, AFAICS, useless.
Aaron Lun (14:10:28): > Banning it would avoid us having to deal with intersecting polygons; not impossible, but an unnecessary pain.
Kevin Rue-Albrecht (14:11:16): > i can’t see a use for it either
Aaron Lun (14:12:36): > Because if you want to take the intersection, you can always draw a tighter box or lasso directly.
Aaron Lun (14:18:19): > Also, code tracking is completely FUBAR now due to changes in how the commands are arranged.
Kevin Rue-Albrecht (14:19:35): > i can imagine:confused:well we broke a few walls recently to experiment with stuff
Kevin Rue-Albrecht (14:20:11): > i’m getting near a prototype implementation of thegeom_pointcombo with lasso
Aaron Lun (14:21:27): > @Charlotte SonesonCowplot looks nice. Do we need a separate plot for categorical legends?
Aaron Lun (14:21:59): > Or for any legends, for that matter.
Charlotte Soneson (14:22:23): > Perhaps. So the legends are in the plots, and they can be extracted withcowplot::get_legend, but it requires that theggplotobject is saved.
Aaron Lun (14:23:19): > That’s fine, we have a cache for ggplots anyway. But I would suggest doing this after the spinout branch gets merged into heat.@Kevin Rue-Albrecht, feel free to do so after you clean up the aesthetics.
Aaron Lun (14:23:45): > I’m going home now. Am pretty pooped.
Kevin Rue-Albrecht (14:24:05): > @Kevin Rue-Albrechtuploaded a file:why nothing is drawn? - File (PNG): why nothing is drawn?
Kevin Rue-Albrecht (14:24:11): > i wanna be slapped -_-
Aaron Lun (14:24:19): > lol
Kevin Rue-Albrecht (14:25:48): > almost there :Warning: Error in parse: <text>:11:0: unexpected end of input
Kevin Rue-Albrecht (14:26:39): > something about the command that I wrote, anyway i’ll post again when i have good news ^^
Kevin Rue-Albrecht (14:42:21): > hmm ok.. I can get the command to work outside of Shiny if I prefix it withggplot() +, but there’s a typical ggplot error message otherwise > > Error in geom_path(aes(x = x, y = y), data = data.frame(x = c(0.468228548105303, : > non-numeric argument to binary operator >
Kevin Rue-Albrecht (14:45:15): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 19.40.33.png - File (PNG): Screenshot 2018-02-16 19.40.33.png
Kevin Rue-Albrecht (15:12:56): > Ok it’s working. One last rebuild for cleanup and I’ll push
Kevin Rue-Albrecht (15:14:23): > ilove that lasso
Aaron Lun (15:15:48): > Are the points at the vertices necessary? These could make it difficult to figure out where the last point actually is, in complicated lassos.
Aaron Lun (15:16:23): > Also, you could consider using ageom_rectat the first point, to mark the area where you can click to close the lasso.
Kevin Rue-Albrecht (15:16:29): > i have downsized them
Kevin Rue-Albrecht (15:16:42): > (the intermediate ones)
Kevin Rue-Albrecht (15:17:21): > i see what you mean with the geom_rect, but why don’t you take what I just pushed for a test run
Aaron Lun (15:17:29): > Can’t, at home now.
Kevin Rue-Albrecht (15:17:39): > ok let me boot up and screensht
Kevin Rue-Albrecht (15:18:11): > the plot above isn’t representative of what the app looks like
Aaron Lun (15:18:55): > Also, the refactoring I did earlier means that you don’t need to deparse the lasso waypoints.
Kevin Rue-Albrecht (15:19:09): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 20.18.54.png - File (PNG): Screenshot 2018-02-16 20.18.54.png
Kevin Rue-Albrecht (15:19:28): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 20.19.15.png - File (PNG): Screenshot 2018-02-16 20.19.15.png
Kevin Rue-Albrecht (15:19:38): > ain’t it pretty?
Aaron Lun (15:19:54): > Yes.
Aaron Lun (15:20:02): > Switch togeom_polygonupon close?
Kevin Rue-Albrecht (15:20:18): > ’f course, next up
Kevin Rue-Albrecht (15:20:31): > just gotta get dinner with the lady, and i’m back on it
Aaron Lun (15:20:33) (in thread): > You only need to assume that aall_lassos[[plot_name]]matrix exists.
Aaron Lun (15:20:58): > Cool.
Aaron Lun (15:21:25) (in thread): > Check out how.create_plot()handles it, for example, when using it to select points for brushing.
Aaron Lun (17:32:48): > Wait. Hold on. Am I the only unmarried dude here?
Kevin Rue-Albrecht (17:33:16): > not married, moving that way though
Aaron Lun (17:34:30): > Well, I must be the only single here, then.
Aaron Lun (17:35:08): > outrageous
Aaron Lun (17:35:22): > And I didn’t even get any flowers from my secret admirers on wednesday
Kevin Rue-Albrecht (17:37:30): > well, the price of productivity: i’m only back on iSEE now, for a short bit. Dinner turned into Netflix time
Aaron Lun (17:38:40): > lol
Kevin Rue-Albrecht (17:39:45): > i’m removing thefillaesthetic now, as it had only a minor visual impact on brush dots, and to avoid possible conflicts with other more useful fills in the future
Kevin Rue-Albrecht (17:40:07): > i’m dotting the path while the brush is not finished
Kevin Rue-Albrecht (17:40:25): > and i’m using shape to mark the initial lasso point
Aaron Lun (17:40:59): > oh okay
Aaron Lun (17:41:17): > yeah, dotting the path while we’re going along is good
Kevin Rue-Albrecht (17:41:21): > i like your idea of square for the start and dots for the others
Aaron Lun (17:41:28): > makes it more enjoyable to put down waypoints
Kevin Rue-Albrecht (17:41:34): > exactly
Aaron Lun (17:41:39): > and will distract people from the non-instant responsiveness
Kevin Rue-Albrecht (17:42:17): > oh man that’s fun (even though i switch the shapes the wrong way around)
Kevin Rue-Albrecht (17:42:28): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 22.41.57.png - File (PNG): Screenshot 2018-02-16 22.41.57.png
Aaron Lun (17:43:19): > nice
Kevin Rue-Albrecht (17:43:28): > it’s distracting me alright
Aaron Lun (17:44:04): > You can usegeom_rectto match the size of the starting rectangle to the area in which clicking will close the lasso.
Kevin Rue-Albrecht (17:44:30): > true
Kevin Rue-Albrecht (17:44:31): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 22.44.13.png - File (PNG): Screenshot 2018-02-16 22.44.13.png
Aaron Lun (17:44:56): > can we get dense lines?
Aaron Lun (17:44:58): > denser
Kevin Rue-Albrecht (17:44:59): > but a ‘decently’ sized point seems a good enough approximation, no?
Aaron Lun (17:45:08): > Dunno.
Kevin Rue-Albrecht (17:45:14): > that’sdottedstyle, I candashit
Aaron Lun (17:45:17): > Okay
Aaron Lun (17:45:23): > Anyway, people might get misled
Aaron Lun (17:45:33): > Just like when you’re playing FPS
Kevin Rue-Albrecht (17:45:35): > it’s just that i’d like to avoid adding unecessary layers
Aaron Lun (17:45:39): > and the target box != the visual
Aaron Lun (17:45:52): > I’m shooting the damn thing but it’s not losing HP
Kevin Rue-Albrecht (17:49:13): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 22.49.01.png - File (PNG): Screenshot 2018-02-16 22.49.01.png
Kevin Rue-Albrecht (17:49:14): > more like this?
Kevin Rue-Albrecht (17:50:34): > full selection of linetypes available at:http://sape.inf.usi.ch/quick-reference/ggplot2/linetype
Aaron Lun (17:56:07): > Yeah, that’s better.
Aaron Lun (17:56:49): > Or “c”, perhaps.
Aaron Lun (17:57:01): > Your call
Kevin Rue-Albrecht (18:10:53): > what was the previous brush alpha?
Kevin Rue-Albrecht (18:11:12): > (to apply to the new polygon)
Kevin Rue-Albrecht (18:18:50): > hm..@Aaron Lun: i’ve written a new function.self_poygon_paththat returns the command to draw the closed polygon, but I can’t figure how to switch between this one and the lasso one on the condition ‘is_closed’ let’s say
Kevin Rue-Albrecht (18:20:32): > can i just push it and let you handle the switch, or is this something you can guide me through messages?
Aaron Lun (18:20:57): > you should be able to check it from the matrix of lasso waypoints
Aaron Lun (18:21:08): > > attr(mat, "closed") >
Aaron Lun (18:21:43): > You should have access to the waypoint matrix within the.self_laso_pathfunction, so I would keep everything in there.
Aaron Lun (18:22:02): > As for the alpha - the brush alpha isn’t specified by me, I think shiny does it under the hood.
Kevin Rue-Albrecht (18:22:02): > ahh i get you
Aaron Lun (18:22:12): > So just fiddle with it until you get something nice, I guess.
Kevin Rue-Albrecht (18:23:23): > cool thanks
Aaron Lun (18:24:53): > Make sure you protect against the NULL attribute, though; have a look at the .zoomClick observer for an example.
Kevin Rue-Albrecht (18:25:40): > like that?
Warning: Error in if: argument is of length zero
Kevin Rue-Albrecht (18:25:47): > :laughing:
Aaron Lun (18:25:57): > …yes.
Kevin Rue-Albrecht (18:26:38): > i was already building it when you sent the warning ^^
Kevin Rue-Albrecht (18:34:33): > is it better to only setattr(..) <- closed, or wouldn’t it make sense to also setattr(..) <- openwhen is not the case? or a logical, in fact?
Kevin Rue-Albrecht (18:35:05): > oh no wait
Kevin Rue-Albrecht (18:35:10): > i’m getting tired again
Kevin Rue-Albrecht (18:39:40): > ok, based on past experience, another two builds that include stupid typos, and I’ll have a working app
Kevin Rue-Albrecht (18:41:10): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 23.40.44.png - File (PNG): Screenshot 2018-02-16 23.40.44.png
Kevin Rue-Albrecht (18:41:17): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-16 23.40.52.png - File (PNG): Screenshot 2018-02-16 23.40.52.png
Kevin Rue-Albrecht (18:41:39): > it’s a bit weak alpha (0.1)
Kevin Rue-Albrecht (18:43:47): > alright ladies and gentlemen, I pushed
Kevin Rue-Albrecht (18:45:06): > We’ll have to refactor that soon, to be honest. Prototype code gets ugly fast:confused:
Aaron Lun (18:51:54): > I can barely see the alpha, TBH
Aaron Lun (18:52:12): > re attr open; the attributes get wiped upon rbind, and it was too much of a chore to keep on resetting it.
Aaron Lun (18:56:39): > In fact, I don’t think the brush rectangle has any alpha. So best to keep that to zero.
2018-02-17
Kevin Rue-Albrecht (04:34:02): > Btw. I didn’t post one last screenshot where I upped the alpha to 0.2 instead of 0.1. I can remove if we prefer
Kevin Rue-Albrecht (04:55:05): > ok i found where the brush box was alpha=0, applied the same to the polygon now
Kevin Rue-Albrecht (05:08:00): > hang on. no i don’t get it:.self_brush_boxdoes indeed producegeom_rectwithalpha=0, but visibly there is something with an alpha≠0 in the app, when brushing a rectangle
Kevin Rue-Albrecht (05:08:32): > I can’t see where that one is coming from: is this a Shiny thing beyond our control?
Kevin Rue-Albrecht (05:14:19): > yep. still don’t know where the bush_box alpha is coming from, but it seems to be a 0.2
Kevin Rue-Albrecht (05:39:16): > brush_fill_coloris my answer
Kevin Rue-Albrecht (05:41:01): > default shiny brushopacity = 0.25. damn i was close (?brushOpts)
Kevin Rue-Albrecht (06:03:14): > @Kevin Rue-Albrechtuploaded a file:ECM-2.pdf - File (PDF): ECM-2.pdf
Kevin Rue-Albrecht (06:03:25): > @Charlotte Sonesondoes this help our discussion from a few days ago?
Kevin Rue-Albrecht (06:04:24): > i’ve kinda mixed a bit functions, slots, and variable names, but it gives the gist i think?
Charlotte Soneson (06:08:55): > Yeah, I think so.
Kevin Rue-Albrecht (06:11:20): > although it’s still not completely accurate, indeed
Kevin Rue-Albrecht (06:24:01): > closer now
Kevin Rue-Albrecht (06:24:03): > @Kevin Rue-Albrechtuploaded a file:ECM-3.pdf - File (PDF): ECM-3.pdf
Kevin Rue-Albrecht (06:28:58): > i’ve got a few more figures to put together for another manuscript, but i’ll try to clean up this one further: more homogenous use of function_or_ slot names, “not defined?” as a condition to drill further into more general color maps, etc..
Aaron Lun (06:53:02): > Ugh. Why do you guys wake up so early on weekends.
Aaron Lun (06:53:26): > Anyway, I’ve realized that the opacity of the polygon probably shohuldn’t be set
Aaron Lun (06:53:33): > Because the brushing box doesn’t have any fill either.
Kevin Rue-Albrecht (06:53:40): > you can change it in one place for all brushes now
Kevin Rue-Albrecht (06:53:44): > constants.R
Kevin Rue-Albrecht (06:54:26): > not much of a choice waking up earlythis weekend: figures for manuscript, PPT for “genomics forum” next Thursday, and I still want to spare some time for iSEE
Aaron Lun (06:54:34): > lol
Charlotte Soneson (06:55:13): > What about the olympics?!?
Charlotte Soneson (06:55:21): > That’s why I was up early today:wink:
Aaron Lun (06:55:52): > I guess we don’t have much of a winter in australia, so I never bothered.
Kevin Rue-Albrecht (06:56:10): > well if there’s tri-hacka-thlon at the olympics, we could whip up a team I think
Charlotte Soneson (06:56:15): > No excuse, there are skiers from Tonga:slightly_smiling_face:
Charlotte Soneson (06:56:33): > Correction: skier
Aaron Lun (06:57:30): > Well, yeah, we send people, but it doesn’t really capture the imagination.
Aaron Lun (06:57:35): > Except for steve bradbury
Kevin Rue-Albrecht (08:40:39): > eh….https://neuroconductor.org ?!
Kevin Rue-Albrecht (08:41:25): > although that page is fun:https://neuroconductor.org/dependency-graph
Aaron Lun (09:11:42): > WTF
Aaron Lun (09:11:53): > They couldn’t have just used Bioconductor?
Aaron Lun (09:11:54): > Weird.
Kevin Rue-Albrecht (09:13:21): > for a name like that, i was half-expecting some familiar names, but it seems they just leeched the popularity of BioC:confused:
Charlotte Soneson (09:13:54): > https://twitter.com/gringene_bio/status/950452400279642113 - Attachment (twitter): Attachment > @JayPykw @mikelove @StrictlyStat @bcaffo https://pbs.twimg.com/media/DTCva74VAAAJnG9.jpg
Aaron Lun (10:08:35): > @Kevin Rue-Albrechtlegend pops up, can we kill it?
Kevin Rue-Albrecht (10:09:05): > when does it pop up?
Aaron Lun (10:09:12): > for the lasso
Aaron Lun (10:09:24): > let me just do some refactoring first, tho.
Kevin Rue-Albrecht (10:09:51): > + guides(fill="none",color="none")usually does the trick (fill or color are example here)
Kevin Rue-Albrecht (10:10:01): > give me the green light and i look at it
Aaron Lun (10:10:10): > Okay.
Aaron Lun (10:29:17): > Done, and the legend is also fixed.
Aaron Lun (10:29:25): > See if you can understand what’s happened.
Kevin Rue-Albrecht (10:29:29): > cool
Kevin Rue-Albrecht (10:29:41): > i’m scared:sweat_smile:
Aaron Lun (10:29:48): > Now there is no deparsing done within the commands themselves.
Aaron Lun (10:30:01): > But it requires us to deparse at the top of the code tracker… which I will do.
Kevin Rue-Albrecht (10:30:15): > Nice. I wasn’t sure how to address that one last night
Kevin Rue-Albrecht (10:40:18): > two more manuscript figures and i’m allowed to play with the other kids (on iSEE)
Aaron Lun (10:41:10): > God I thought you had kids as well
Kevin Rue-Albrecht (10:42:28): > nah.. one step at a time:sweat_smile:
Aaron Lun (10:55:57): > Code tracker is fixed.
Aaron Lun (10:56:27): > I think we need to add a point when the lasso starts, otherwise it’s a bit hard tot ell.
Aaron Lun (10:56:47): > I originally skipped it if it was <2 because I didn’t want to deal with problems when geom_path just gets a length 1 input.
Kevin Rue-Albrecht (10:57:08): > oh yeah, sorry it also crossed my mind
Kevin Rue-Albrecht (10:58:02): > i took the habit of rushing the first two waypoints to see sthg, but obviously we need something a bit more intuitive for users
Kevin Rue-Albrecht (10:59:12): > so basically: if nrow=1: a geom_point, if is_closed: a geom_polygon, else: a geom_path
Aaron Lun (10:59:45): > Yep
Aaron Lun (11:22:17): > Restricted brushing to self is now prohibited.
Kevin Rue-Albrecht (11:33:59): > you just make me think of all the things we’d have had to put in theNEWSif that was a released package or we even submitted to Bioc a couple of weeks ago ^^
Aaron Lun (11:47:30): > Brushing information now shows up in a separate UI element.
Aaron Lun (11:48:10): > Note that, for reasons I don’t understand, it still works correctly when the input$brush (but not the memory brush) is cleared by re-rendering. Not entirely sure why this is - I would have expected it to fail - but oh well.
Aaron Lun (11:49:25): > Or maybe not; perhaps only the visible brush gets cleared, and input$brush persists upon re-rendering. I hope this is the case, otherwise something funny is happening elsewhere.
Kevin Rue-Albrecht (11:59:16): > thanks for doing all that - i’m stuck digging up old code for the last figure over here - what do you mean by “it still works” ?
Kevin Rue-Albrecht (12:01:22): > “the brush gets cleared by re-rendering”: you mean it gets cleared visually, but not functionally? > ah hang on.. i’m getting the subtlety between input brush and memory brush
Kevin Rue-Albrecht (12:02:30): > darn i’m everytime time impressed how reactive the app is with this memory/caching system
Aaron Lun (12:30:20): > @Charlotte Sonesoninfrastructure for a legend plot is now available, seeiSEE.Randlegend_field.
Aaron Lun (12:30:49): > Once this is done, I suggest a merge back to master.
Aaron Lun (12:31:24): > Oh, infrastructure is currently living inspinout, so that needs to be merged back toheatfirst, and thenheatback tomaster.
Aaron Lun (12:31:58): > Can you guys brainstorm some options for (i) clustering, (ii) centering and (iii) colour scaling/capping?
Kevin Rue-Albrecht (12:33:07): > Alright, i’m done with my own stuff. god it took long. I’m catching up here.
Aaron Lun (12:36:31): > Clustering is probably not that important (yet), but centering and colour scaling/capping would probably be critical for any usability.
Kevin Rue-Albrecht (12:36:31): > what’s missing to mergespinouttomaster?
Aaron Lun (12:36:43): > Well,heattomasteris missing a few things.
Kevin Rue-Albrecht (12:36:56): > oh sorryspinouttoheat
Aaron Lun (12:37:01): > Nothing.
Aaron Lun (12:37:09): > Except the initial dot for the lasso
Kevin Rue-Albrecht (12:38:00): > @Kevin Rue-Albrechtuploaded a file:Pasted image at 2018-02-17, 5:37 PM - File (PNG): Pasted image at 2018-02-17, 5:37 PM
Aaron Lun (12:43:49): > Centered heatmaps only have one logical colour scale; [some dark color] -> [some light color at zero] -> [some other dark color]
Aaron Lun (12:44:10): > If they’re not centered, you can of course use the assayCOlorMap.
Aaron Lun (12:44:18): > But if they are centered, we’d probably need to limit their choices
Aaron Lun (12:44:34): > As the asssay color map probably won’t make much sense.
Kevin Rue-Albrecht (12:44:39): > light at the centre? I tend to use purple-black-yellow these days, for row-scaled heat maps
Aaron Lun (12:44:43): > Really?
Aaron Lun (12:44:52): > Well, okay.
Aaron Lun (12:44:55): > But black at zero, right?
Aaron Lun (12:45:01): > Otherwise it would be impossible to interpret.
Kevin Rue-Albrecht (12:45:35): > although I’ve come across some Vogel paper with pale colors at the center, let me dig that up
Kevin Rue-Albrecht (12:45:57): > yes, i meant black at zero (or your light color)
Kevin Rue-Albrecht (12:47:05): > @Kevin Rue-Albrechtuploaded a file:Vogel-style - File (PNG): Vogel-style
Aaron Lun (12:47:18): > That’s okay too.
Aaron Lun (12:47:44): > In any case, we need another panel box with heatmap-specific colour options.
Kevin Rue-Albrecht (12:48:37): > @Kevin Rue-Albrechtuploaded a file:Kevin recent - File (PNG): Kevin recent
Kevin Rue-Albrecht (12:48:49): > that’s something I’ve done recently
Aaron Lun (12:48:51): > The purple’s a bit hard on the eyes.
Aaron Lun (12:49:10): > Looks more realistic, though.
Kevin Rue-Albrecht (12:49:11): > yup, I think it’s"purple"in the pureR meaning
Aaron Lun (12:49:21): > As in, from a microarray chip.
Kevin Rue-Albrecht (12:49:33): > i’m sure sure we can soften the color tone a bit
Aaron Lun (12:49:47): > Well, in any case, we need a heatmap color box with checkboxes for centering and scaling
Aaron Lun (12:49:53): > and something to limit the maximum range
Kevin Rue-Albrecht (12:50:16): > yup. which reminds me: > 1) we can probably quantile-crop the range
Aaron Lun (12:50:27): > Or we can go one better
Aaron Lun (12:50:32): > and useupdateSliderInput
Kevin Rue-Albrecht (12:50:33): > like trim the lower/top 10%
Aaron Lun (12:50:36): > whenever anyone changes the heatmap
Kevin Rue-Albrecht (12:50:36): > oh yeah
Kevin Rue-Albrecht (12:51:41): > while you mention microarrays, there’s always the green-black-red style
Aaron Lun (12:52:00): > Not good for colour blind people.
Kevin Rue-Albrecht (12:52:20): > ahh that’s why it died out
Aaron Lun (12:52:46): > anyway, I need to review something, so will not be responsive for a bit.
Kevin Rue-Albrecht (12:53:09): > ok, i’m adding the single-dot lasso thing, and then i merge back toheat
Aaron Lun (12:53:38): > Okay. See how far you can get with the heatmap UI.
Kevin Rue-Albrecht (12:54:33): > still need to get familiar with it; i kinda followed theheatdevelopment from a distance this week
Aaron Lun (12:54:57): > Sure. Once it’s done happy to merge back to master
Aaron Lun (12:55:07): > and then the gruelling task of re-testing everything begins.
Kevin Rue-Albrecht (12:55:43): > yeah.. i’m not really looking forward, but it’ll be good to go through that exercise anyway
Aaron Lun (12:56:55): > Well, that’s why they pay me the medium bucks.
Kevin Rue-Albrecht (12:58:23): > not sure whether to interpret that as more than small bucks or less than large bucks ^^
Kevin Rue-Albrecht (13:13:55): > @Kevin Rue-Albrechtuploaded a file:t’was a long day.. - File (PNG): t’was a long day..
Aaron Lun (13:14:07): > lol
Kevin Rue-Albrecht (13:14:36): > not sure I’m glad that R foundshape=1.5
Kevin Rue-Albrecht (13:14:50): > sometimes i long for error/warning messages
Kevin Rue-Albrecht (13:16:03): > alright, time to drop all themessage/printdebugs, and I push
Kevin Rue-Albrecht (13:20:29): > where should lasso constants go? (startShape, startSize, etc.)
Kevin Rue-Albrecht (13:20:44): > I’ve hard-coded them in a few places, and I’d like to refactor that out
Kevin Rue-Albrecht (13:20:58): > constantsorplotting?
Aaron Lun (13:24:36): > plottingunless they affect many things across files in which caseconstants.
Kevin Rue-Albrecht (13:25:22): > ok, plotting it is then
Kevin Rue-Albrecht (13:31:30): > ok that’s pushed now
Kevin Rue-Albrecht (13:31:53): > i just noticed: double-click doesn’t seem to zoom out anymore, known bug?
Aaron Lun (13:33:31): > It should zoom out only if ther’es no lasso.
Aaron Lun (13:33:40): > First double click should clear the lasso.
Aaron Lun (13:33:45): > Next double click will zoom out.
Aaron Lun (13:33:47): > I think.
Kevin Rue-Albrecht (13:34:02): > let me try again, but i think i trieda lot of double clicks
Kevin Rue-Albrecht (13:34:45): > oh, seems like i didn’t do it properly: it just worked now
Kevin Rue-Albrecht (13:34:57): > yep - sorry
Aaron Lun (13:34:57): > Good
Kevin Rue-Albrecht (13:35:10): > i’ll merge now as soon as CI completes then
Kevin Rue-Albrecht (13:37:48): > it’s really weird: new session and it doesn’t work anymore, neither redDimPlot nor colDataPlot
Aaron Lun (13:38:01): > ?
Aaron Lun (13:38:04): > The CI?
Kevin Rue-Albrecht (13:38:15): > the dlb-click to zoom out
Kevin Rue-Albrecht (13:38:46): > my only way to reset the plots is to change the XY plotting parameters
Kevin Rue-Albrecht (13:39:01): > or the color parameters
Kevin Rue-Albrecht (13:39:23): > looks like there’s a replotting that doesn’t trigger in certain conditions then
Aaron Lun (13:39:47): > hold on
Aaron Lun (13:42:10): > Also, the initial lasso point is being created even when brushing - this shouldn’t be possible.
Kevin Rue-Albrecht (13:42:29): > yes, that too
Kevin Rue-Albrecht (13:43:20): > it gets cleared visually, but i guess the fact that it appears at all highlights a problem somewhere
Kevin Rue-Albrecht (13:44:03): > btw, i don’t how, but for a few moment, in the same session, I managed to get the zoom-out feature again… and then lost it again
Kevin Rue-Albrecht (13:45:51): > oh wait a sec, maybe I know where that comes from
Kevin Rue-Albrecht (13:47:07): > I may have relaxed a bit too much the condition to draw the lasso:innocent:
Aaron Lun (13:47:36): > How so?
Kevin Rue-Albrecht (13:48:04): > if (is.null(current) || nrow(current) < 2L) {->if (is.null(current) {
Aaron Lun (13:48:24): > Well, that shouldn’t affect the upstream observers.
Kevin Rue-Albrecht (13:48:40): > < 1Lwould have been a smarter condition
Kevin Rue-Albrecht (13:49:43): > ah.. yep you’re right.. the lasso point still gets drawn with the brush box
Aaron Lun (13:50:05): > Well, I vaguely understand the cause of the zoom.
Kevin Rue-Albrecht (13:53:37): > meanwhile, i found a way to prevent the lasso-start to show on top of the brush box
Aaron Lun (13:53:46): > Zoom is fixed.
Kevin Rue-Albrecht (13:53:48): > who draws first?
Kevin Rue-Albrecht (13:53:49): > ah
Kevin Rue-Albrecht (13:53:52): > :dizzy_face:
Aaron Lun (13:54:41): > I’m curious to how the waypoints even exist.
Aaron Lun (13:54:50): > You shouldn’t need to intercept it at your level.
Kevin Rue-Albrecht (13:55:36): > not the waypoints
Kevin Rue-Albrecht (13:56:02): > it seems that the click used to draw the box is captured as a lasso first point
Aaron Lun (13:56:16): > even so, once brushing starts, it should destroy any existing way points.
Aaron Lun (13:56:27): > And it doesn’t make sense for brushing to start before the click.
Aaron Lun (13:56:54): > This is an observer problem.
Kevin Rue-Albrecht (13:57:01): > well.. except if “click” is only registered as a “KeyUp” event
Aaron Lun (13:57:21): > Don’t understand.
Kevin Rue-Albrecht (13:57:22): > (rather than a “KeyDown”)
Aaron Lun (13:57:26): > Oh right
Aaron Lun (13:57:29): > Possibly.
Aaron Lun (13:57:56): > Hm…
Aaron Lun (13:58:02): > Will think about this as I take a toilet break.
Kevin Rue-Albrecht (13:58:17): > then, while brush and click start together, brush may take priority when the click-up event happens, for whatever reason
Kevin Rue-Albrecht (13:59:33): > in any case, I can catch the issue at the lasso-drawing step: “do not draw the lasso start point if.brushDatais notNULL”
Aaron Lun (14:02:03): > hold on.
Aaron Lun (14:02:17): > let me try something upstream.
Aaron Lun (14:02:33): > Because this is a more fundamental issue; lasso waypoints and brush data should not coexist in memory.
Aaron Lun (14:08:55): > Well, I think I fixed it, but the order is still unclear to me.
Aaron Lun (14:09:10): > Say you have an existing brush. When a click event triggers, does the brush get removed first, or does the click observer trigger first?
Aaron Lun (14:16:44): > Okay, that should do it.
Aaron Lun (14:17:23): > Now the memory should always be valid (lasso and brush data should never co-exist)
Aaron Lun (14:17:44): > The only possibility is in between updates where the app is temporarily invalid, but your change to plotting should ensure that a lasso is never plotting with a brush.
Kevin Rue-Albrecht (14:30:23): > Ok, neat. I don’t know what the answer to your theoretical click event question is though.
Aaron Lun (14:31:32): > Does anybody?
Aaron Lun (14:31:41): > Anyway, it was a shame that my time crisis idea didn’t work.
Aaron Lun (14:31:58): > The events linked to double-click are now a bit too complicated.
Aaron Lun (14:32:17): > The others are fairly easy; brush kills lasso and lasso kills brush.
Kevin Rue-Albrecht (14:32:21): > so i wait that you push your fix and then we merge back toheat?
Aaron Lun (14:32:26): > Should be pushed now.
Aaron Lun (14:32:52): > Yep. See if you’re up for some heat UI.
Aaron Lun (14:33:01): > Okay, I really need to gett back to reviewing this.
Kevin Rue-Albrecht (14:35:02): > Ok no worries. I see Travis working now. To be fair I think i’ll take a break for the evening, and get back to that tomorrow, after I get a bit of PPT done for my talk Thursday. Still need to figure out a viable topic
Aaron Lun (14:38:41): > Just talk about Heatmaps and UI elements for them.
Aaron Lun (14:38:50): > I talked about matrices in my lab talk.
Aaron Lun (14:39:00): > Thought about doing it for my institute as well.
Aaron Lun (14:39:04): > Dressed up as Neo or something.
Aaron Lun (14:39:11): > I could also do trinity, but probably a bit too tight.
Kevin Rue-Albrecht (14:40:15): > hahaha.. nah, I appreciate the offer, but Steve set that “Genomics Forum” up for people in the Kennedy institute to present work in progress on ’omics projects
Kevin Rue-Albrecht (14:41:13): > but you remind me of a guy (different type of seminars) who had an introduction in the pureMatrix style, with a title about something “Reloaded”
Kevin Rue-Albrecht (14:41:25): > i think image analysis
Aaron Lun (14:43:17): > lol
Aaron Lun (14:57:39): > Probably worth adding lasso support to all the dot-related plots, if you have the time to do that.
Aaron Lun (14:57:43): > Just setclick=
Aaron Lun (15:01:50): > Going home now, am pooped.
Kevin Rue-Albrecht (17:07:47): > ok.. took me a bit of time, but I finally found whereclick=needed to be added:sweat_smile:
Kevin Rue-Albrecht (17:35:55): > alright.. i’ve been playing with it for a bit, but will sleep on it: lasso is enabled on all plots of types if
Kevin Rue-Albrecht (17:36:23): > anything categorical breaks the brush, i’ll go dream the fix
Kevin Rue-Albrecht (17:37:05): > PS: i’ve got a strong feeling it’s something to do with coercing factors to integer
Kevin Rue-Albrecht (17:54:33): > alright, no one sweats it: i found the fix
Kevin Rue-Albrecht (17:57:19): > hm.. violin plots covered. For some reason my fix doesn’t cover griddotplot, even though I thought it would
Kevin Rue-Albrecht (18:24:05): > argh.. i can’t find the solution even though i’ve identified the issue: > basically,.process_brushby_choiceneeds to know whether each of X/Y are grouped or not in thesender plot, so that their numeric representation (used for plotting) can becbind-ed into a matrix fed tomgcv::in.outto evaluate the brush. > As a consequence, the new argumentsgroupX, groupYthat I’ve just added to.process_brushby_choiceare completely useless, as they refer to the groupability of X/Y in thereceiver plot
Kevin Rue-Albrecht (18:27:10): > with that, I’m definitely off to bed, sorry to leave a bit of a mess behind tonight
Aaron Lun (19:53:47): > It should work off the bat, just likebrushedPointsworks off the bat.
2018-02-18
Kevin Rue-Albrecht (04:14:13): > Well I guess something bad is happening when we make that matrix
Kevin Rue-Albrecht (05:16:58): > yeah i think i got it now
Kevin Rue-Albrecht (05:17:45): > > df <- data.frame( > x = sample(letters[1:5], 100, TRUE), > y = rnorm(100) > ) > df.matrix <- as.matrix(df) > # df$x is a factor > # df.matrix is a character matrix >
Kevin Rue-Albrecht (05:19:47): > > library(ggplot2) > library(mgcv) > > df <- data.frame( > x = sample(letters[1:5], 100, TRUE), > y = rnorm(100) > ) > > ggplot(df, aes(x,y)) + geom_violin() + geom_jitter(width = 0.1, height = 0) > > lassoPoints <- matrix( # suited me to lasso 4 points for QC > c( > 3.5, 0, > 4.5, 0, > 4.5, 1, > 3.5, 1 > ), ncol = 2, byrow = TRUE, dimnames = list(NULL, c("x", "y")) > ) > > ggplot(df, aes(x,y)) + geom_violin() + geom_jitter(width = 0.1, height = 0) + > geom_polygon(aes(x, y), as.data.frame(lassoPoints), alpha = 0.2) > > in.out(lassoPoints, as.matrix(df)) # problem > > df.matrix <- as.matrix(df) # demonstrate character matrix issue > > df.matrix <- as.matrix(data.frame( # make numeric matrix > x = as.numeric(df$x), > y = df$y > )) > > in.out(lassoPoints, df.matrix) # works > table(in.out(lassoPoints, df.matrix)) >
Kevin Rue-Albrecht (05:54:11): > i’ve pushed a quick fix that fails only for horizontal violins and griddotplots
Kevin Rue-Albrecht (05:54:33): > (for some reason, itdoes work for griddotplots of a single level
Aaron Lun (06:52:58): > Have a look at whatshiny:::asNumberdoes.
Aaron Lun (06:53:26): > You’ll need toas.numeric(df$y)as well.
Kevin Rue-Albrecht (07:06:32): > i’ve done the latter in my latest push
Kevin Rue-Albrecht (07:07:52): > (without more success)
Kevin Rue-Albrecht (07:08:31): > actually, i wonder if it’s something to do withcoord_flip, i’ve lost a bit track of when coords are transformed or not, tbh
Kevin Rue-Albrecht (07:09:07): > but i wonder if it’s possible that a flipped lasso is applied to unflipped data, or vice versa
Aaron Lun (07:09:29): > that’s possible.
Kevin Rue-Albrecht (07:10:05): > going out for a bit though
Kevin Rue-Albrecht (07:10:10): > catch up later
Aaron Lun (08:36:12): > Well, looks like I need to put a “flipped” indicator in the matrix as well;in.outneeds the flipped coordinates, but the point plotters need the original coordinates.
Aaron Lun (08:52:34): > Should be fixed.
Aaron Lun (09:23:39): > Also fixed issues with lasso clearing upon plot updates.
Kevin Rue-Albrecht (09:23:47): > I just realised: you’re working onspinoutagain
Kevin Rue-Albrecht (09:24:17): > I was looking atheatto checkout your changes:sweat_smile:
Aaron Lun (09:24:20): > oh forgot it was merged
Kevin Rue-Albrecht (09:25:08): > mah, it’s actually not a bad thing, and easy to merge again
Aaron Lun (09:25:12): > well don’t do anything let me merge it.
Kevin Rue-Albrecht (09:25:34): > i’m not doing much this afternoon - trying to get started on my PPT
Kevin Rue-Albrecht (09:38:14): > just from a quick test run, horizontal violins seem ok, but griddotplot with multiple levels still cause the error:Warning: Error in mgcv::in.out: NA/NaN/Inf in foreign function call (arg 4)
Kevin Rue-Albrecht (09:40:05): > I’m guessing it has to do with > > .C(C_in_out, bx = as.double(bnd[, 1]), by = as.double(bnd[, > 2]), break.code = as.double(lowLim), x = as.double(x[, > 1]), y = as.double(x[, 2]), inside = as.integer(x[, 2] * > 0), nb = as.integer(n), n = as.integer(nrow(x))) > > inmgcv::in.out
Kevin Rue-Albrecht (09:40:20): > arg 4 isx = as.double(x[,1])
Kevin Rue-Albrecht (09:40:50): > maybe I shouldas.doublerather thanas.numeric
Aaron Lun (09:40:52): > probably need as.double(as.factor(…))
Aaron Lun (09:41:22): > Though I would have thought that we coerced it properly to a factor in the original construction ofplot.data
Kevin Rue-Albrecht (09:41:40): > yeah. i’m a bit puzzled too, not that Ias.doublevery often
Aaron Lun (09:42:16): > works fine for me.
Aaron Lun (09:42:46): > but not for dual grid plots
Aaron Lun (09:42:57): > We must be missing a coercion on one of the axes.
Kevin Rue-Albrecht (09:43:00): > ah you worried me that I missed a commit
Kevin Rue-Albrecht (09:43:22): > well, maybe justmessage(cmd)to check that out
Kevin Rue-Albrecht (09:44:28): > I wish I could spot those issues from the code itself, but it’s getting increasingly hard/convoluted/self-aware, that I’m falling back onmessageandprintincreasingly often:sweat_smile:
Kevin Rue-Albrecht (09:45:06): > “DeariSEE , tell me what you see here”
Aaron Lun (09:47:26): > coercion is definitely happening….
Kevin Rue-Albrecht (09:48:26): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-02-18 14.48.10.png - File (PNG): Screenshot 2018-02-18 14.48.10.png
Kevin Rue-Albrecht (09:48:50): > I knew Google would like to weigh in on this one:wink:
Aaron Lun (09:51:26): > Well, now it’s working okay…
Kevin Rue-Albrecht (09:51:39): > eh.. what did you change then>
Kevin Rue-Albrecht (09:51:54): > (although I can check out the diff:sweat_smile:)
Aaron Lun (09:52:08): > Nothing from the currentheat.
Kevin Rue-Albrecht (09:52:36): > mhh. i’m up-to-date.. let me try again
Aaron Lun (09:54:24): > yep, looks functional to me, after re-installing and restarting R.
Aaron Lun (09:55:40): > though occasionally I get “argument is of length zero” in.process_brushby_choice
Kevin Rue-Albrecht (09:56:09): > I’ll try that now then. In the meantime, here’s my failing use case: > - reddimPlot receives from colDataPlot > - colDataPlot: x=Core.Type, y=driver_1_s > - lasso the smallest group of point (bottom right intersection) > -Warning: Error in mgcv::in.out: NA/NaN/Inf in foreign function call (arg 4)shows for redDimPlot
Aaron Lun (09:57:14): > Not an issue with the x=dissection_s.
Aaron Lun (09:57:22): > So it must be because there are actually NAs in CoretType.
Kevin Rue-Albrecht (09:57:35): > ohhhh… yeah i think you’re right
Kevin Rue-Albrecht (09:57:57): > damn.. soas.doubleprobably doesn’t deal well withNAs
Aaron Lun (09:58:24): > well, it would just report more NAs.
Kevin Rue-Albrecht (09:58:40): > ah. somgcv::in.outthen ?
Kevin Rue-Albrecht (09:59:40): > i expect we should just exclude points withNAvalues in X/Y from what is passed tomgcv::in.out
Aaron Lun (09:59:54): > yes
Kevin Rue-Albrecht (10:01:29): > ok. what’s the status about branches/commit at the moment? do you have commits awaiting push somewhere?
Aaron Lun (10:08:45): > That should have fixed it.
Kevin Rue-Albrecht (10:12:10): > nice, thanks
Aaron Lun (10:46:30): > Well, the heatmap UI is fully done. Waiting for integration with heatmap plotting. Nag to@Charlotte Soneson.
Kevin Rue-Albrecht (10:51:14): > Cool. I’ll have to think about the dual use ofscale_sizethat conflicts between the lasso path and the boxes of the griddotplot though:Scale for 'size' is already present. Adding another scale for 'size', which will replace the existing scale.
Kevin Rue-Albrecht (10:57:25): > although for some reason, the boxes still look unaffected by the ‘replaced scale’
Aaron Lun (10:57:35): > ¯*(ツ)*/¯
Aaron Lun (10:57:43): > Ggplot. Can’t live with it, can’t live without it.
Kevin Rue-Albrecht (10:59:07): > I’ll do a couple of hours of pptx, and look back at it if I can think of something better. I kinda like the different point size for the start/waypoints, but maybe i can find shapes that do the same effect without the scale
Aaron Lun (12:03:50): > Brushing inputs into row stat tables are fixed.
Kevin Rue-Albrecht (12:04:41): > glad to see there’s no stopping you:slightly_smiling_face:
Kevin Rue-Albrecht (12:06:03): > i’m dying in the middle of reviews, screenshotting left and right to build a measure of background to my presentation:sleepy:
Aaron Lun (12:06:15): > Reviews?
Aaron Lun (12:06:21): > Lol
Kevin Rue-Albrecht (12:06:47): > thymic epithelial cells, John will know if you ask him:wink:
Aaron Lun (12:06:59): > Yes, I was talking to mike morgan about this on Friday
Kevin Rue-Albrecht (12:07:52): > right, it’s a small world/office indeed
Kevin Rue-Albrecht (12:08:31): > oh yeah, by reviews, i meant published ones that I’m shamelessly stealing from
Kevin Rue-Albrecht (12:08:50): > not reviewing anything myself these days, thank god
Kevin Rue-Albrecht (12:09:31): > that would take my last time slot (4-5am) away:stuck_out_tongue:
Aaron Lun (12:09:36): > lol
Charlotte Soneson (12:11:33): > What was your idea with the Lower/Upper values for the heatmap colors? To cap the colors (but keep the original values in the matrix/colorbar) or to cap the values in the matrix?
Kevin Rue-Albrecht (12:12:21): > I think the end result would be the same, but I’d rather cap the color scale
Aaron Lun (12:12:26): > Agreed.
Charlotte Soneson (12:13:14): > The heatmap would look the same, but not the color bar
Kevin Rue-Albrecht (12:14:29): > hm yes, capping the matrix would pretty rather bad: the color bar would not have any chance to show the true range of data
Kevin Rue-Albrecht (12:15:16): > rather, I’d rather cap the color bar, as in: any thing beyond the cap is the same color
Aaron Lun (12:15:18): > The color bar capping is fine.
Aaron Lun (12:15:21): > Yes.
Aaron Lun (12:15:46): > Otherwise if you have one gene that is expressed 100-fold higher in one small population, everything else will llook white
Aaron Lun (12:15:50): > or black, depending on your colour scale
Kevin Rue-Albrecht (12:15:58): > can’t remember the exact code right, but i think setting breaks in the right places should do the trick
Kevin Rue-Albrecht (12:16:52): > repeating the extreme colors twice at each end, once at the cap, then at the max value
Kevin Rue-Albrecht (12:17:18): > and setting breaks evenly within the uncapped range
Kevin Rue-Albrecht (12:17:57): > (i’m trying to think whether there’s a built-in, smarter way, to set caps)
Aaron Lun (12:18:49): > Well, if you’re not clustering, the caps won’t make a difference.
Aaron Lun (12:19:00): > And if you are clustering, then it’s pretty simple; just cluster on the uncapped data first.
Aaron Lun (12:19:29): > The “suggest feature order” button doesn’t have any observers attached to it yet, tho.
Kevin Rue-Albrecht (12:20:08): > indeed, but irrespective of clustering, I was just trying to draw out an implementation plan for the capped color scale
Charlotte Soneson (13:19:33): > Is there a reason that capping is only available for unscaled data?
Aaron Lun (13:21:04): > Is there a need to cap for dynamic range when you’re scaling as well?
Aaron Lun (13:21:25): > Seems a bit complicated, as you would have to cap before scaling.
Charlotte Soneson (13:22:18): > But wasn’t the idea to just cap the color scale? And keep the original values? I can imagine that values can get pretty large after scaling (if the sd is low).
Aaron Lun (13:23:42): > Yes, but how are you interpreting the cap values after scaling?
Charlotte Soneson (13:24:00): > As z-score limits above which the color stays the same
Aaron Lun (13:24:10): > MMmmm…
Charlotte Soneson (13:24:12): > Maybe I’m still not getting the idea
Aaron Lun (13:24:25): > Well I guess you could do that.
Kevin Rue-Albrecht (13:25:12): > I think I’m with Charlotte on this one. Even when I rowscale, I sometimes cap out those annoying outliers
Charlotte Soneson (13:25:24): > I would think that the default (-5,5) would be a stranger default for capping uncentered,unscaled data (which will typically be positive)
Aaron Lun (13:25:58): > Yes, that’s why the default is to center and not scale.
Kevin Rue-Albrecht (13:26:07): > however, I don’t have a written recipe for that, it’s kind of a case by case call, which value I cap
Aaron Lun (13:27:19): > Plotting uncentered data is a bit silly to me, though.
Aaron Lun (13:27:58): > Anyway, just bust out the textInputs from the .conditionalPanelOnRadio, and you can hvae access to the limits when you’re scaling.
Aaron Lun (13:28:31): > For uncentered data - if anyone would want it - just use the assayColorMap.
Aaron Lun (13:28:46): > Otherwise, we’ll have to supply custom color maps where zero is white/black/some neutral color.
Aaron Lun (13:28:53): > with enforced symmetry, etc.
Charlotte Soneson (13:29:09): > So no capping available for uncentered data at all then.
Charlotte Soneson (13:29:43): > And available for centered data regardless of scaling. I think that would make sense
Aaron Lun (13:29:50): > you can still cap for uncentered data.
Aaron Lun (13:31:02): > Though arguably we’d need to run anupdateTextInputto give sensible defaults when centering is turned off.
Charlotte Soneson (13:31:07): > Yes
Aaron Lun (13:31:46): > Though we would probably have to do that anyway… Anyway, just implement your stuff first.
Aaron Lun (13:32:51): > As in, whenever the plot parameters change (i.e., change to assay or input genes), we should runupdateTextInput
Aaron Lun (13:34:36): > Or better - if it’s set to Inf/-Inf, don’t do any capping. This should probably be the default.
Aaron Lun (13:35:33): > However, if it does get specified, the boundaries of the colour scale need to stretch to the specified bounds, even if the colours are well within the limits.
Aaron Lun (14:16:44): > Belting out some REO speedwagon
Aaron Lun (14:16:49): > and annoying the others in the office.
Kevin Rue-Albrecht (14:18:41): > careful when theycan’t fight this feeling anymore ^^
Aaron Lun (14:19:41): > lol
Charlotte Soneson (14:52:51): > I just pushed an update that centers, scales, sets the color scale and shows the legend plots. Need to think a bit more about how to best include the capping.
Kevin Rue-Albrecht (14:54:14): > nice one, i’ll check that out now, to start thinking where I can help
Aaron Lun (16:23:20): > BTW, thinking of submitting to BioC at the end of the month. I think that’s a good deadline to motivate people; we just need to brush up the tests and documentation.
Aaron Lun (16:23:55): > Also, finally vacuumed my room! At last, I can sit in my room without getting bitten by something.
Kevin Rue-Albrecht (16:29:11): > hahaha
Kevin Rue-Albrecht (16:31:47): > Alright, end of the month.. should be possible with one more weekend within that time
Aaron Lun (16:32:05): > TBH, something is still biting me… but with a lot less intensity, so I’ll call it a win.
Kevin Rue-Albrecht (18:08:28): > fixing XAxis selection for featExprsPlot when linked to a rowStatTable
Kevin Rue-Albrecht (18:08:56): > i think it’s looking for the gene index e.g.1inrownames
Aaron Lun (18:46:51): > Yes, I had noticed this but hadn’t gotten around to dealing with it.
Kevin Rue-Albrecht (19:08:12): > solved
Kevin Rue-Albrecht (19:08:21): > anyone still around?
Kevin Rue-Albrecht (19:11:11): > I got a fun one on branchCyTOFnow
Kevin Rue-Albrecht (19:11:56): > it’s not an exported function, so you’ll have to play withiSEE:::CyTOF(sce, 4)for example
Kevin Rue-Albrecht (19:13:41): > also, I’ve checked and updated tests inconstants,defaults,ECM, andplotting
Kevin Rue-Albrecht (19:14:11): > I think it’stest_brush_linksthat needs most an update next
Kevin Rue-Albrecht (19:14:30): > Going to sleep now.
2018-02-19
Kevin Rue-Albrecht (04:09:19): > just crossed my mind: tonight i’ll go through parts of the vignette that I can fix up, if I remember well some code chunks got outdated recently
Kevin Rue-Albrecht (04:55:13) (in thread): > @Charlotte Soneson, I’m about to play again with the CyTOF example in the vignette > I’d be curious to hear your opinion whenever you have time
Charlotte Soneson (04:57:31) (in thread): > Ok. I’ll check theCyTOFfunction now
Kevin Rue-Albrecht (04:58:46) (in thread): > Thanks. > Thefeaturesargument is still a rough clunky implementation (single character vector, that lists features for axes in the order y1, x1, y2, x2, …)
Kevin Rue-Albrecht (04:59:46) (in thread): > but the main point a the moment is then_linkedargument that initialises the app with that many featExprsPlot already (linearly) chained together
Charlotte Soneson (05:01:30) (in thread): > Just checked outCyTOF,mode_CyTOF()complains about missingn_featExprPlot
Kevin Rue-Albrecht (05:01:42) (in thread): > arff
Kevin Rue-Albrecht (05:02:16) (in thread): > I must have changed it after I commited
Charlotte Soneson (05:02:17) (in thread): > You have a more recent version locally?
Kevin Rue-Albrecht (05:02:52) (in thread): > I can see the erroneous code in my local version too
Kevin Rue-Albrecht (05:03:00) (in thread): > i know what’s wrong though
Kevin Rue-Albrecht (05:03:19) (in thread): > I must have stupidly changed the code between my last play and the commit. duh
Kevin Rue-Albrecht (05:03:44) (in thread): > for now just replace all occurences ofn_featExprPlotbyn_linked
Kevin Rue-Albrecht (05:03:47) (in thread): > sorry
Charlotte Soneson (05:03:50) (in thread): > Ok!
Charlotte Soneson (05:05:24) (in thread): > Yep, now it runs
Kevin Rue-Albrecht (05:09:28) (in thread): > Alright then. Possibly lower priority than, but feel free to send feedback or even refine the behaviour to more natural expectations of FACS/CyTOF users
Charlotte Soneson (05:10:18) (in thread): > Looks cool! I wonder whethermode_gatingwould be a more natural name (since lots of people are moving away from gating in CyTOF towards automatic cluster inference/tSNE/whatever plots).
Kevin Rue-Albrecht (05:10:31) (in thread): > For simplicity I voluntarily excluded all other plot and table types for a first backbone
Charlotte Soneson (05:10:43) (in thread): > I think that makes sense
Kevin Rue-Albrecht (05:10:48) (in thread): > Yep, that name crosses my mind too. I think you’re right
Charlotte Soneson (05:10:49) (in thread): > They can always be added later
Kevin Rue-Albrecht (05:11:44) (in thread): > Also I think one of the reasons for the main mode boot time is that we load one of each type. I think lighter, more specialised modes will boot up faster
Charlotte Soneson (05:11:45) (in thread): > And then, yes, maybe the features should be provided in a way that they can be immediately matched in pairs, and so that it is clear what will go on x- and y-axes
Charlotte Soneson (05:12:01) (in thread): > Yes, perhaps. This is indeed fast
Charlotte Soneson (05:12:21) (in thread): > Maybe there is a particular type that takes longer
Kevin Rue-Albrecht (05:12:46) (in thread): > I thought of a list of paired features. But it sounded more painful to type
Charlotte Soneson (05:13:00) (in thread): > That is true
Kevin Rue-Albrecht (05:13:00) (in thread): > Maybe a data frame of two columns
Charlotte Soneson (05:14:00) (in thread): > Yeah, with column namesXandYor so, so that they can be specified in any order. That would at least be interpretable. And the vector could be easily converted into such a data frame:slightly_smiling_face:
Kevin Rue-Albrecht (05:14:35) (in thread): > I’ve got a feeling that populatingselectizewidgets could be the slowdown
Kevin Rue-Albrecht (05:15:00) (in thread): > just thinking about ‘heavier’ widgets, with more items in them
Charlotte Soneson (05:15:16) (in thread): > Yes, that’s probably true.
Charlotte Soneson (05:23:01) (in thread): > I don’t notice a big difference in launching time for apps containing only a single panel, depending on that panel.
Kevin Rue-Albrecht (05:23:45) (in thread): > ah my theory collapses then:confused:
Charlotte Soneson (05:24:12) (in thread): > But I don’t know if there is a way of measuring this objectively, now I was just launching and waiting…
Charlotte Soneson (05:24:48) (in thread): > I timed the creation of the app (withapp<-iSEE(...)), and that was independent of the type.
Kevin Rue-Albrecht (05:25:03) (in thread): > btw, sorry again about my typo inmode_CyTOF. However, I’ve just found out that the cytof example in the vignette still refers togeneExprs...instead offeatExprs. I’ll solve both tonight if no one gets to it during the day
Kevin Rue-Albrecht (05:25:35) (in thread): > oh I don’t mean the creation of the app. I mean the initialisation of the various panels at run time
Charlotte Soneson (05:25:48) (in thread): > Yes, the vignette is indeed a bit outdated. I pushed just at the time of the change, and thought I’d wait until we had converged on a nomenclature.
Charlotte Soneson (05:26:03) (in thread): > Ah, I see. I thought you meant the launch time of the app, depending on theinitialPanels
Kevin Rue-Albrecht (05:26:15) (in thread): > I’ve timed the whole app generation to the moment where plot get displayed and it all happens lighting fast
Kevin Rue-Albrecht (05:26:27) (in thread): > it seems that it’s only the rendering that causes the pause
Kevin Rue-Albrecht (05:26:46) (in thread): > plus, all the plots/tables show simultaneously
Charlotte Soneson (05:26:49) (in thread): > Yep, seems reasonable
Charlotte Soneson (05:27:28) (in thread): > But if some panel types were heavier than others, I suppose you would also see that if you launch an app with only that panel (and measure the time until the plot is shown)
Kevin Rue-Albrecht (05:27:38) (in thread): > which seems to point at a within-Shiny stack of tasks, rather than something in our control
Kevin Rue-Albrecht (05:27:49) (in thread): > absolutely
Kevin Rue-Albrecht (05:28:17) (in thread): > that’s the thing, it’s not like the app shows 3 panels, lags on the 4th one, and then shows the last 2
Charlotte Soneson (05:28:30) (in thread): > No
Kevin Rue-Albrecht (05:28:35) (in thread): > it’s just the whole app that freezes for a couple of seconds, and then all plots show together
Kevin Rue-Albrecht (05:28:46) (in thread): > I think that just says it’s not in our control
Charlotte Soneson (05:28:57) (in thread): > Yeah, I tend to agree with that feeling
Aaron Lun (06:33:22): > Can we mergeheatwithmasteronce the heatmaps are done.
Aaron Lun (06:33:38): > We can fix the tests on a new branch; butheatis getting pretty damn big.
Charlotte Soneson (06:35:08): > I will try to get to the capping later today, but I have some other things I need to do first. Is there anything else pending there?
Charlotte Soneson (06:35:38): > And do we need to fix the tests first (or comment out the ones that don’t work) in order to be able to even mergeheatintomasterdue to the protection…
Aaron Lun (06:37:09): > Ah crap.
Aaron Lun (06:37:12): > Forgot about that.
Aaron Lun (06:37:26): > All right, let’s do the bare minimum necessary to get it functional again.
Aaron Lun (06:53:00): > Currently working on brush link tests.
Kevin Rue-Albrecht (06:55:10): > Perfect. I’ll clean up theCyTOFbranch tonight following on the thread with Charlotte. I think that’ll give a nice demo of how to use the functions indefaults.R
Kevin Rue-Albrecht (06:55:29): > btw: that doesn’t impact on theheatbranch
Kevin Rue-Albrecht (06:56:35): > bonus: it’ll probably make the code chunks simpler to write in the vignette section about CyTOF data
Federico Marini (07:19:58) (in thread): > Chimin’ in here: I’d assign the vignette/doc curation to myself, as I could not help that much on this round of implementation
Kevin Rue-Albrecht (07:20:22) (in thread): > absolutely, no worry
Federico Marini (07:20:51): > Just demoed the lasso part to a colleague
Kevin Rue-Albrecht (07:20:57) (in thread): > as a heads up, don’t bother too much with the CyTOF section until I clean up the branch of the same name
Federico Marini (07:20:59): > jaws dropped:slightly_smiling_face:
Kevin Rue-Albrecht (07:21:10): > hahaha, good to hear
Federico Marini (07:21:23): > one point - but that is probably hard
Federico Marini (07:21:45): > shiny has the nice feature of having the brushed area “movable” once it is created
Federico Marini (07:21:53): > I guess we can’t with our polygon?
Aaron Lun (07:36:58): > no
Kevin Rue-Albrecht (07:55:51): > awwww:stuck_out_tongue_winking_eye:
Aaron Lun (08:10:04): > Monster PR going intomastersoon.
Aaron Lun (08:10:23): > I will spend tomorrow documenting some of the internal functions.
Federico Marini (08:10:24): > noticed that live
Federico Marini (08:10:45): > shall we give another twitter shoutout for the lasso feature?
Aaron Lun (08:11:35): > Sure, if you want. Heatmaps, lasso and row data plots are the new features, from what I remember.
Federico Marini (08:12:14): > yep, enough to remind everyone that there’s new meat on the grill - or any other relevant metaphor:slightly_smiling_face:
Federico Marini (08:15:50): > reminder for the open big goals/implementation, from the minutes of our latest conf call: > - rowdata,:white_check_mark:- heatmap,:white_check_mark:- aesthetics:white_check_mark:- names, titles, brushed in/out, % n of N points
Federico Marini (08:16:07): > Only the refresh of the documentation would be strictly needed
Federico Marini (08:17:01): > i.e. general info (README, DESCRIPTION) , github tags too? + > Rd & vignette + > intro tour
Aaron Lun (08:17:05): > Yes, so we can push for submission at the EOM.
Federico Marini (08:18:10): > the hover info is not so feasible since we do not enter plotly, and the usage via server is somewhat covered - I did not test it yet, but sounded correct the way it was proposed by slowkow on GH
Charlotte Soneson (08:34:25): > Maybe this one would warrant a version bump…:slightly_smiling_face:
Aaron Lun (08:35:27): > Yes it would.
Charlotte Soneson (09:10:14): > I pushed a commit with color range setup for centered data, where one of the provided color scales are used. To work properly it assumes that the lower bound is <0 and the upper bound is >0, I think that makes sense (since I explicitly set the color at 0 to be the middle color in the scale). I don’t know where is the best way to check this: should thetextInputreset if one tries something else, or should it be some other type of input?
Aaron Lun (09:19:30): > will have a look at this in a second, got to deal with something first.
Kevin Rue-Albrecht (09:24:00): > We should probably soon start including updates to theNEWSfile as part of our development cycle. Probably only from the moment we submit to BioC, though
Aaron Lun (09:24:24): > At every PR, I would say.
Kevin Rue-Albrecht (09:25:16): > I’m repeating myself from this weekend (call it a self-reminder): The warning messages aboutscale_sizecollision between the lasso path and the boxes of the griddotplot still bugs me. I wanna look into that to keep a console as clean as possible
Aaron Lun (10:56:41): > looking at the capped color bar; the cap does succesffully restrict the color range, but the actual numerical values on the color bar do not change. Is this intended?
Kevin Rue-Albrecht (10:57:51): > Without checking visually, that sounds like the intended behaviour
Aaron Lun (10:58:03): > I would have expected the colour bar itself to contract.
Kevin Rue-Albrecht (10:58:04): > It highlights the capping to the user
Aaron Lun (10:58:15): > Hm. I guess that’s true.
Aaron Lun (10:58:18): > Okay, that’s fine then.
Kevin Rue-Albrecht (10:58:53): > If I get it correctly, the numerical values show the full range, while the capping preserves the color beyond the caps, is that right?
Kevin Rue-Albrecht (10:59:23): > because that’s what I would have expected
Aaron Lun (10:59:41): > Yes, that’s the case.
Kevin Rue-Albrecht (11:00:30): > I have to admit that I can imagine some (corner?) cases where that would compress the color range to a tiny fraction of the color bar
Kevin Rue-Albrecht (11:01:04): > Dunno how frequent/frustrating those cases are
Aaron Lun (11:02:23): > Probably fine, I would say.
Aaron Lun (11:03:44) (in thread): > Just add ashiny::validate, I think. I’ve only used theupdate*Inputsfunctions in situations where the invalidity affects the state of the app (and thus downstream observers), rather than just a single plot.
Aaron Lun (11:10:43): > Also, the legend plot gets cluttered pretty quickly. Is there a way to force the legends to align horizontally, rather than switching to a new line (which pushes some legends out of the plot’s viewport)?
Aaron Lun (11:48:13): > Added support for reclustering genes.
Aaron Lun (12:00:19): > Supported heatmap in the code tracker.
Aaron Lun (12:02:15): > @Charlotte SonesonThe heatmap output code could do with some prettifying re. line widths.
Aaron Lun (12:08:23): > How easy would it be to add zoom for the heatmap?
Aaron Lun (12:08:49): > It would only affect the heatmap itself; the annotation bars would be considered part of the axes, for example.
Federico Marini (12:12:18): > would the activation of the brush already grant the zoom?
Federico Marini (12:12:48): > not the brush brush like receiving the brush, but the simple brush&zoom
Aaron Lun (12:13:18): > The shiny side is not the problem; it’s making sure that the ggplot side can respond to a new set of coordinates.
Federico Marini (12:13:33): > with the doubleclick on the brushed area
Aaron Lun (12:13:44): > Or more precisely, to figure out where the new set of coordinates actually refer to with respect to the heatmap.
Aaron Lun (12:14:03): > To translate the x/y into genes/cells; this is not so obvious in the grid layout, due to the annotation on the top.
Federico Marini (12:14:46): > true.
Federico Marini (12:15:02): > hm. I don’t know how feasible/hard this could be
Federico Marini (12:15:21): > I had in pcaExplorer a static plus an interactive heatmap
Federico Marini (12:15:52): > but it can work differently now that we do it in ggplot2?
Aaron Lun (12:16:05): > ¯*(ツ)*/¯
Federico Marini (12:16:09): > (oh well, and I was not using the brushing info either)
Federico Marini (12:16:25): > i was merely changing the view on more or less zoom
Federico Marini (12:17:11): > side rant: would that be so hard for NCBI to have a damn proper correspondence between SRA and GEO
Federico Marini (12:17:28): > (spoiler alert, answer isyes, most likely )
Aaron Lun (12:18:20): > Well, Sanger and EBI’s computer systems don’t like talking to each other.
Aaron Lun (12:18:27): > And the buildings are 50 meters apart.
Aaron Lun (12:18:39): > It’s easier just to dump data to a hard drive and walk over.
Federico Marini (12:19:11): > eheheheh
Federico Marini (12:19:22): > ok, it’s more a Schadenfreude eheheheh
Federico Marini (12:21:11): > I feel Charlotte is feeling the same pain again:stuck_out_tongue:
Charlotte Soneson (12:21:23): > I agree with the comments on the heatmap code and the legend plots are fixable. I really just gotta ride out this migraine first and then I’ll fix that…:confused:
Aaron Lun (12:22:07): > Yes, I too have this weird rash on my legs.
Aaron Lun (12:55:00): > Pretty restrained convo on the BioC devel mailing list
Aaron Lun (12:56:46): > Adding internal documentation; how do we avoid exporting the docs?
Federico Marini (13:04:00): > would it be fine to remove the roxygen comment and leave it as if we roxygen’ed?
Aaron Lun (13:05:52): > Huh?
Aaron Lun (13:06:10): > Currently documenting.spawn_brush_chartinbrushing.R.
Aaron Lun (13:06:17): > devtools::document() complains that .spwan_brush_chart.Rd is an invalid name, which is fair enough.
Federico Marini (13:06:59): > oh
Federico Marini (13:07:13): > you want to create the .Rd
Federico Marini (13:07:28): > I thought we do document, but only in text, above the func itself?
Federico Marini (13:07:35): > sorry if I misunderstood this
Aaron Lun (13:07:50): > There’s no guarantee that it’ll stay up to date if we don’t Roxygenize regularly.
Aaron Lun (13:07:58): > Internal functions being what they are.
Aaron Lun (13:08:40): > Of course, we can just tell it not to export it… but I don’t know how to do it.
Aaron Lun (13:09:22): > It would be nice if there was a flag to check the Roxygenized stuff, but not create the actual file.
Aaron Lun (13:10:56): > Perhaps@keywords internal
Kevin Rue-Albrecht (13:11:25): > Isn’t it just about removing the @export tag?
Aaron Lun (13:12:40): > That’s for putting the function in the NAMESPACE
Kevin Rue-Albrecht (13:13:19): > Yep but a function that’s not exported yet has a man page would raise a flag in CMD check no?
Aaron Lun (13:13:32): > yes.
Kevin Rue-Albrecht (13:13:38): > So I’d expect export to coordinate both
Aaron Lun (13:13:50): > The lack of exporting is not the problem.
Aaron Lun (13:14:11): > I’d likedocument()to check the Roxygen flags, even though the manpage doesn’t actually get built.
Aaron Lun (13:14:20): > This would encourage us to keep at least the arguments up to date.
Kevin Rue-Albrecht (13:14:27): > Ahh wow
Kevin Rue-Albrecht (13:14:36): > Eh i don’t know about that one
Aaron Lun (13:14:37): > Otherwise, like most parts of the genome, the documentation would just degrade over time.
Kevin Rue-Albrecht (13:15:08): > Well.. just like us, most of the source would be non coding
Aaron Lun (13:15:17): > junk.
Aaron Lun (13:15:23): > Let’s call a spade a spade here.
Aaron Lun (13:15:30): > Despite whatever ENCODE says.
Kevin Rue-Albrecht (13:15:48): > Not even regulatory? Aww
Federico Marini (13:15:57): > dan graur GROWLING now:smile:
Aaron Lun (13:16:33): > How about this; we getdocumentto create the documentation, but we don’t commit it.
Aaron Lun (13:16:46): > This should give us the best of both worlds.
Kevin Rue-Albrecht (13:17:31): > Oh yeah. A bit of gitignore can’t hurt that much
Aaron Lun (13:17:52): > Let me finish off the brushing example, and we can see whether or not we agree on this.
Aaron Lun (14:04:59): > Okay, have a look atheat. Try as best as you can to putimportFromas close to where it’s used, rather than dumping everything iniSEE-pkg.R.
Aaron Lun (14:08:36): > Doesn’t really matter if we doble-up on importFrom statements, either.
Charlotte Soneson (14:22:31): > What about putting the legends horisontally? I don’t think we can safeguard against all possible numbers and types of annotations, at some point people are just going to have to change the size of their panel if things don’t fit…
Charlotte Soneson (14:22:41): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-19 at 20.22.11.png - File (PNG): Screen Shot 2018-02-19 at 20.22.11.png
Kevin Rue-Albrecht (14:28:16): > That looks cool:slightly_smiling_face:Also, I can’t wait to reply to the first users who suggest that we ‘grid-organise’ legends, to fit more in the available space:wink:
Kevin Rue-Albrecht (14:29:05): > PS: the branch network is getting fun to look athttps://github.com/csoneson/iSEE/network
Kevin Rue-Albrecht (15:19:02) (in thread): > btw, I’m changing the interface to themode_gating
Kevin Rue-Albrecht (15:19:42) (in thread): > introducing adata.frameto define Y/X features made me realise that the number of rows will definen_linked
Charlotte Soneson (15:20:00) (in thread): > Right, cool
Aaron Lun (15:26:16): > Looks pretty good to me.
Aaron Lun (15:27:04): > Yes, the branch network has always been a source of some amusement for me.
Aaron Lun (15:28:29): > Hey, is the “NA” in core type a “NA” string, or actually missing?
Kevin Rue-Albrecht (15:29:28): > actually missing
Kevin Rue-Albrecht (15:29:36): > remember the issue about brushing?
Kevin Rue-Albrecht (15:30:15): > also, ggplot is pretty smart about that: it tends to give a default grey color to trueNAvalues
Kevin Rue-Albrecht (15:30:45): > a default that can be controlled separately from real levels, obviously:wink:
Aaron Lun (15:31:02): > Hm. Okay. I’ll let it slide this time…
Kevin Rue-Albrecht (15:31:26): > now that you point it out, i can see it for Secondary.Type as well, same colour
Kevin Rue-Albrecht (15:52:33): > here we go,mode_gatingdocumented and exported
Aaron Lun (16:39:49): > what’s with thesample()for defaultfeatures? Looks a bit odd to me.
Kevin Rue-Albrecht (16:40:14): > yeah.. feedback welcome. i just wanted to throw in something for demo
Kevin Rue-Albrecht (16:40:31): > probably best left as a mandatory argument
Aaron Lun (16:40:48): > Yeah, I would say so too.
Aaron Lun (16:44:48): > I’m going to demo the latestheattomorrow - hopefully everything is functional!
Kevin Rue-Albrecht (17:48:12): > just playing with the gating mode, i wonder: when a bunch of plots are serially linked all with restrict, if say (1) gets brushed, (2) gets restricted, but (3) that depends on (2) [by restrict, remember] still showsall the data points even those filtered out of 2 by 1 (even though 2 isn’t brushed yet)
Kevin Rue-Albrecht (17:48:47): > In the absence of brush on (2), should (3) be updated to show only points present in (2)?
Kevin Rue-Albrecht (17:50:03): > in a way, I would say “no”, as the current behaviour is, because despite the “restrict” relation 2->3, as long as 2 isn’t brushed, 3 has no reason to hide any point,even those that aren’t present in 2 because of 1 , right?
Kevin Rue-Albrecht (17:58:19): > yeah, nevermind, it’s actually brilliant the way it is
Kevin Rue-Albrecht (17:58:55): > @Aaron Lun: it will all be much prettier in my next push, where the example in the man page select the top variable genes; it’s also more deterministic
Aaron Lun (18:30:35): > Killed this giant fucking mosquito in my room
Aaron Lun (18:30:43): > It was probably giving me the rash
Aaron Lun (18:31:04): > It was so large I thought it was a moth, so I didn’t kill it when I first saw it a few days ago
Kevin Rue-Albrecht (18:31:44): > more meat to grill if you let them feed for a while
Aaron Lun (18:31:58): > I’ll, uh, keep that in mind.
Kevin Rue-Albrecht (18:34:53): > unrelated: i just caught up on that bioc-devel thread. damn he’s trying hard to bring us back to R≥2.3
2018-02-20
Federico Marini (02:51:42): > Jeez - too many people want to grab a gun and shoot themselves in their feet. Funny how different persons came up with very similar points, are the two colleagues?
Federico Marini (04:10:52): > naive question, when running the example of isee now I cannot see the DT from the row stats table
Federico Marini (04:11:00): > yet, the table is linked to other plots
Federico Marini (04:11:12): > are you also having this?
Federico Marini (04:12:42): > ok, i think i got it
Federico Marini (04:12:59): > we removed this “fake” column that let the table be displayed, i guess
Kevin Rue-Albrecht (04:13:31): > Morning idea: brush on the sequencing lane, see the dimRed based on read counts restricted to cDNA fragments in that area:innocent:
Kevin Rue-Albrecht (04:13:55): > What « useless »?:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (04:20:03): > bit ofRsamtools here, bit ofRsubread there …
Kevin Rue-Albrecht (04:21:34): > on a more serious note, I think I just finally figured out a ‘fix’ to thescale_sizeconflict between the lasso and griddotplot on my walk to work this morning. I’ll implement/test tonight
Aaron Lun (04:22:10): > @Federico MariniThe fake column is back.
Aaron Lun (04:22:34): > @Kevin Rue-AlbrechtDid you fix the colour-by-feature name validation error? It still shows up for me.
Kevin Rue-Albrecht (04:23:27): > eh.. did i fix it on a branch different thanheat?:face_with_raised_eyebrow:
Kevin Rue-Albrecht (04:23:33): > lemme check
Aaron Lun (04:24:46): > I’m still seeingchosen_gene %in% rownames(se)in.process_colorby_choice_for_row_plots
Kevin Rue-Albrecht (04:24:50): > oh no wait, i haven’t fixed anything about color.. only the X axis
Aaron Lun (04:24:57): > Okay, I’ll do it then.
Kevin Rue-Albrecht (04:25:11): > i fixed it here:https://github.com/csoneson/iSEE/commit/54463be172624efea68e3a1773b9727ce26c666d#diff-8912ca2ee6504bef8f23a001cfb07051L132
Kevin Rue-Albrecht (04:25:25): > i guess you’re looking at something nearby
Kevin Rue-Albrecht (04:26:33): > right, i see it too. sorry
Aaron Lun (06:31:42): > Internal default functions have been documented.@Charlotte Soneson- need to add some documentation for heatmap defaults.
Kevin Rue-Albrecht (06:33:20): > Nice one. I’ll go through as much of the plotting functions then tonight, if there is nothing more urgent first (?)
Aaron Lun (06:35:36): > Not from my end. I didn’t get to demo it, we ran out of time. Again.
Kevin Rue-Albrecht (06:36:19): > oh right, didn’t realise what you meant by demo, thought you mean “test”. Group meeting? Seminar? bigger?
Aaron Lun (06:44:11): > Group meeting
Charlotte Soneson (07:47:15): > Ok, I’ll have a look at the heatmap defaults documentation
Aaron Lun (07:54:23): > Should be as simple as adding a new @section .
Aaron Lun (08:39:53): > Almost finished documenting all of the functions I have primary responsibility for. Onlymisc.Ris left.
Kevin Rue-Albrecht (08:41:36): > Nice:slightly_smiling_face:Speaking or responsibility: who’ll take care of users per R 3.5?:innocent:
Kevin Rue-Albrecht (08:43:24): > I like the guy’s last reply, ‘teaching’ Vince how Bioc works:joy:
Aaron Lun (08:56:30): > lol
Aaron Lun (08:56:44): > won’t someone think of the children?
Aaron Lun (09:37:28): > God, the misc functions are so mind-numbingly boring.
Kevin Rue-Albrecht (09:38:18): > not sure whether I’m willing to share the fun-time ofplotting.Rwith anyone:stuck_out_tongue:
Aaron Lun (09:40:20): > I have been doing this for 5 hours straight, so a bit of fraying around the edges is to be expected.
Kevin Rue-Albrecht (09:40:55): > I think it’ll all be worth it, and we’ll be glad to have this in a few months
Aaron Lun (10:20:14): > AW GOD
Aaron Lun (10:20:19): > GOD THIS IS PAINFUL
Federico Marini (10:53:19) (in thread): > Go go Lori - I expect this is not yet over with that guy:disappointed:
Kevin Rue-Albrecht (10:54:40) (in thread): > I anticipate his end goad is to support Windows Vista
Federico Marini (10:55:31) (in thread): > WinXP FTW
Kevin Rue-Albrecht (10:55:57) (in thread): > I always liked that one. Waited until Vista got replaced to upgrade
Aaron Lun (11:04:31): > I’m done. I can’t do anymore today.
Kevin Rue-Albrecht (11:06:10): > I’m still impressed by the time you find for it every day
Aaron Lun (11:06:18): > I’m at the EBI so I don’t want to do any real work.
Aaron Lun (11:06:35): > Also I finished the beachmat revisions, so I’m in a decent mood.
Kevin Rue-Albrecht (11:07:42): > I hear they’ve got pretty good job packages, for that kind of freedom:smirk:
Aaron Lun (11:21:15): > Wondering whether the text on the x-axis in violin plots and grid plots should be vertical… Same for the continuous legends. Currently they just overlap each other on small screens.
Aaron Lun (11:21:21): > e.g., secondary type.
Kevin Rue-Albrecht (11:21:53): > arf, yeah
Kevin Rue-Albrecht (11:22:42): > maybe not strictly vertical, sthg liketheme(axis.text.x = element_text(angle = 60))
Aaron Lun (11:23:09): > Angled text is pretty bad when it gets even moderately compact.
Kevin Rue-Albrecht (11:24:53): > ok. (there is alsoelement_text(... hjust=H, vjust=V)) that can help sometimes, although I’d rather not research the magic formula to computeHandVdepending on the number and character length of the labels…
Aaron Lun (11:27:09): > I guessangle=90should do the job, then?
Kevin Rue-Albrecht (11:27:25): > yep, let’s go for it
Aaron Lun (11:29:07): > If you’re documentingplotting.R, I’ll leave it to you to fiddle with.
Kevin Rue-Albrecht (11:29:27): > I’m just wondering whether we could enforce a sort of label trimming to a max length (text = paste0(letters, collapse = "");gsub("^(.{10}).*", "\\1...", text))
Aaron Lun (11:30:19): > … I suppose we could. Though this is really the user’s responsibility. If they give us gibberishly long strings, that’s their own fault.
Aaron Lun (11:30:31): > It’s pretty easier to just dosubstrbefore heading into the app, if they need to.
Kevin Rue-Albrecht (11:30:38): > yup
Aaron Lun (11:31:46): > And panel height is fairly cheap, so you can just make panels taller if the names are long.
Aaron Lun (11:32:05): > Panel width is admittedly more difficult as increasing width will kick other panels off the same row.
Aaron Lun (11:32:55): > Also realized I have primary responsibility fordynamicUI.R- dammit.
Aaron Lun (11:34:09): > Will deal with that tomorrow. I no longer have the appetite for it tonight.
Kevin Rue-Albrecht (13:53:25): > :tada:just realised: we passcheckagain:tada:
Aaron Lun (13:53:35): > Yeah.
Kevin Rue-Albrecht (13:53:43): > nicely done
Kevin Rue-Albrecht (13:55:51): > alright, if it’s still there I’ll look into this ‘vertical X labels’ thing, and also test the fix for thescale_sizeclash that I keep ranting about
Kevin Rue-Albrecht (13:56:16): > that’ll warm me up for dealing with theplotting.Rdoc
Aaron Lun (13:56:18): > Yep
Aaron Lun (13:56:22): > I haven’t touched the plots.
Aaron Lun (13:56:45): > Have a look at some of the documented functions, though, to get a feel of the style of the internal docs.
Kevin Rue-Albrecht (13:58:40): > yep, i went straight foriSEE-extras
Aaron Lun (14:24:10): > Formode_gating; I suggest using camelcase for exported functions, because that’s what we’ve being doing so far.
Kevin Rue-Albrecht (14:27:28): > oh right, sorry
Kevin Rue-Albrecht (14:28:12): > i actually meant to brainstorm with<!channel>whether there was better naming convention for ‘modes’
Aaron Lun (14:28:47): > If you plan on adding more of them, probably best to have a consistent prefix
Aaron Lun (14:28:49): > likemodeGating
Aaron Lun (14:28:57): > orsetupGating
Aaron Lun (14:29:06): > orinitializeGating
Kevin Rue-Albrecht (14:29:16): > gatingModesounded more natural, but I’m attracted to the idea of a (consistent prefix, as you just wrote), to facilitate browsing using autocompletion
Kevin Rue-Albrecht (14:29:38): > nah, notinitialize, that’s going to be misleading with other methods
Charlotte Soneson (14:29:40): > I likemodeGating
Kevin Rue-Albrecht (14:32:52): > (also, I’m in the process of adding the corresponding unit test formodeGating, btw)
Kevin Rue-Albrecht (18:14:33): > damn i knewplotting.Rwould be … an exercise of navigating the code back and forth between functions ^^
Aaron Lun (19:12:33): > It shouldn’t be that hard, I would have thought - the hierarchy of calls is fairly clear, and you can just pass off...as “further arguments to be passed to X”.
Kevin Rue-Albrecht (19:30:38): > i just see you message now that I was about to post: “thank god for the refactoring: once past the first function, a good dose of copy-paste makes the following ones pretty simple”
Kevin Rue-Albrecht (19:33:31): > I’ll see how much will I have left when I’m done with the basic documentation of each function, but I’ve long been tempted to document the...of.create_plot. It’s really the hub of plotting arguments
Kevin Rue-Albrecht (20:57:09): > almost there, i can’t leave it at 5 functions from the end ^^
2018-02-21
Kevin Rue-Albrecht (03:39:49): > @Aaron Lun: just crossed my mind, looking back at the remaining open issues, in this case the ‘info on hover’ one: would it make sense to have a widget (radio button or selectInput) to switch the behaviour of single-click between “lasso” and “label ID”? > As it says on the tin, I’d expect the click to add aggrepel::geom_repel_text()layer that annotates the data points nearest to the click (within a maximal range, maybe). > Double-click behaviour would then have to be further divided depending on the switch status, to clear either an existing lasso or existing point labels. Although double-click could also clear the other if the first one isn’t there (to avoid having to constantly switch the input). > If we were going this way, I’d suggest an app-wide switch in a modal, to avoid overcrowding thebrush parametersbox, and also because I wouldn’t expect users to often use the two types of click behaviour simultaneously on multiple plot panels. Rather, I’d imagine a period of time in the sessions lifetime where users would be labelling data points in the various plots, without further brushing at that point.
Aaron Lun (04:18:02): > Dunno.
Aaron Lun (04:18:15): > I guess we could.
Aaron Lun (04:18:54): > But we’re probably overloading the click operator; the double-click operator has way to much behaviour already.
Kevin Rue-Albrecht (04:20:36): > I had this feeling too while describing the idea.
Kevin Rue-Albrecht (04:21:31): > Plus, given the choice, I’d prefer the transient effect of a hover to a permanent label
Aaron Lun (04:28:25): > Wow. All of you guys really like your manual word wrap, don’t you.
Aaron Lun (04:31:07): > In any case, make sure you distinguish between “Shiny brush” (i.e., the thing that Shiny makes) and “brushing” in general (i.e., any selection between plots, be it with the Shiny brush or lasso). This is particularly pertinent in the documentation forprocess_brushby_choice- see.spawn_brush_chartfor details.
Aaron Lun (04:32:35): > Just being precise with the terminology should be sufficient - on your first mention of general brushing in a given documentation file, note that it includes both the Shiny brush and lasso selection. Otherwise, if you are referring explicitly to Shiny brushing, mention “Shiny brush”.
Kevin Rue-Albrecht (04:46:10): > absolutely, I vaguely remember catching a few “Shiny brush”/“brushing” in my half-sleep last night, but i may well have been a bit inconsistent between the first few functions and the last ones
Kevin Rue-Albrecht (04:55:11): > Also, I’ve dropped thescale_sizeof lasso waypoints for a smarter use of point shapes. I started looking into your idea of drawing the actual rectangle that would capture a closing click, but that would mean: > - either somehow making.self_lasso_pathaware of the plot bounds, so that it can recomputerange/100- sticking those bounds as additional attributes on the matrix of waypoint coordinates (kinda makes sense, to define waypoints with their bounding box as attribute) > - adding an extra call to a new function iniSEE.Rthat would be given the bounding box coordinates and the first lasso point to generate the appropriate ggplot layer
Aaron Lun (04:57:54): > Don’t worry about that ATM, I think.
Kevin Rue-Albrecht (05:01:14): > To be fair, I’m quite happy with the current state of affair on this aspect. The default point size, even though relatively small, seems a pretty good approximation of that target box.
Charlotte Soneson (05:16:12): > @Kevin Rue-AlbrechtJust wondering: are theiSEE:::necessary in themodeGating? It gives lots of warnings in the check.
Kevin Rue-Albrecht (05:17:30): > oops. maybe not, i’ll check again, but off the top of my head, i think it’s leftovers from developing it in an R script initially
Charlotte Soneson (05:17:53): > yes, I thought so. I can try to remove them then.
Kevin Rue-Albrecht (05:17:57): > thanks for pointing out the warnings, I missed them in the midst of a few documentation-related ones
Kevin Rue-Albrecht (05:18:24): > oh cool, I’m happy to see you fix this if you have time. Thanks
Federico Marini (05:49:41) (in thread): > Good point. Especially when describing the main documentation, we should be homogeneous in this
Federico Marini (05:51:00) (in thread): > I recall at least from the excellent concepts defined in Tamara Munzner’s work that brushing is this area selection
Federico Marini (05:51:16) (in thread): > what we also do is what I would call “panel linking”
Federico Marini (05:51:31) (in thread): > when we go i.e. table-to-plot
Aaron Lun (05:54:52): > Looking at plotting documentation. note thatidis an integer scalar.
Kevin Rue-Albrecht (05:57:11): > aargh, I was looking for that word and couldn’t remember it. I’ll also have to search-and-replace “value” by “scalar” in a few places then
Charlotte Soneson (05:58:59): > I have fixed some documentation issues inheat. But I don’t understand why it complains about a missing link to.panel_generationinINTERNAL_check_plot_feasibility.Rd…
Kevin Rue-Albrecht (05:59:31): > I had this issue a few times last night and solved them, let me check this one
Kevin Rue-Albrecht (06:00:37): > seems to be because.panel_generationisn’t documented yet:https://github.com/csoneson/iSEE/blob/6f28ff400f5b17cfcbf206c1bc240debd0c007e2/R/dynamicUI.R#L42
Aaron Lun (06:04:02): > Also, make sure you refer to@param idas the “current” plot, see my latest updates. This is because it would otherwise be a bit confusing.
Aaron Lun (06:13:00): > And don’t forget the de-centralization of@importFromstatements.
Kevin Rue-Albrecht (06:13:44): > I’ll browse back the channel to list your last few points in a to-do list for later. Do we have a better place to list them, which isn’t as formal as GitHub issues?
Aaron Lun (06:14:07): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (06:14:27): > btw,griddotplotis the only (awww) plot that doesn’t end in_plot. Suggestions?
Aaron Lun (06:14:47): > Rename it if you like, I don’t really care.
Kevin Rue-Albrecht (06:15:50): > well, me neither, but going through the code in one go was akin to a manuscript proof-reading: made me spot those small details/inconsistencies
Aaron Lun (06:15:59): > Oh, there’s a lot of them.
Aaron Lun (06:16:36): > I need to go and replace all instances ofiwithidas well.
Kevin Rue-Albrecht (06:17:19): > I also spotted some variable names that are a bit misleading and should probably be renamed systematically: for instance to differentiate betweencmd(character vector) and maybecmdList(list of character vectors)
Kevin Rue-Albrecht (06:17:53): > That’s mostly what slowed me down in documenting commands, list of commands, and lists of lists of commands
Kevin Rue-Albrecht (06:18:14): > had to track down where they were generated and passed up/down
Aaron Lun (06:19:03): > In any case,@Federico Marini; suggest rolling out the annotation facility into a separate function. That is;iSEEwill take an “annotation function” that returns a HTML tag object upon being supplied a row index and SCE object. You can then write a function that generates this function given the annotation information (i.e., orgdb and etc.) This generalizes it for users where the annotation may not be org’able, or even genes.
Aaron Lun (06:19:24) (in thread): > As long as it’s underscored for internal functions/args.
Aaron Lun (06:19:39) (in thread): > This rule isn’t applied consistently either, so we’ve got to fix that.
Kevin Rue-Albrecht (06:19:46) (in thread): > agreed
Kevin Rue-Albrecht (06:20:03): > That’ll be cool to have indeed
Aaron Lun (06:21:22): > For example, one might prefer to have links to UCSC for genomic regions.
Kevin Rue-Albrecht (06:24:03): > Dunno if it would make things simpler of more complicated, but I’ve been meaning to look intoGSEABase::CollectionTypefor a while now.
Aaron Lun (06:25:21): > It may be possible to supply a set of pre-defined groupings for the heatmap plot.
Kevin Rue-Albrecht (06:25:53): > I’ve leftGOexpress with an ugly home-madedata.frameof annotations, and just can’t find the time/support (as in funding) to update it
Aaron Lun (06:37:00): > I’ve also noticed that everyone seems quite trigger happy with wrapping things in\codein the docs.
Aaron Lun (06:38:20): > As a general rule, this probably isn’t necessary when you’re referring to a class name or object type.
Aaron Lun (06:39:10): > I too used to put\code{SOME CLASS}until I looked at the Bioc core docs - e.g.?GRangesdoesn’t\code{DataFrame}.
Aaron Lun (06:39:38): > \code{list}is particularly unnecessary, unless you’re explicitly referring to the action of making a list.
Federico Marini (06:39:59): > pleading guilty on this - I thought it was helpful to highlight what a more program-related thing is, and what is just purer text
Aaron Lun (06:40:38): > Putting to many things in\codemakes the entire block quite difficult to read, so perhaps we should be more judicious in our choice of what to put in and what to leave out.
Aaron Lun (06:41:23): > I’ll admit that I’m not always consistent with this, but we should try our best nonetheless.
Kevin Rue-Albrecht (06:41:30): > pleading guilty too
Aaron Lun (06:42:17): > A good guide is the latex biocstyle, where class names are rendered with italics rather than monospace.
Federico Marini (06:44:04): > quick question for the heatmap team: the suggest feature order is nothing but adding the clustering of rows, right?
Kevin Rue-Albrecht (06:44:22): > yeah, I put a lot oflistand `character in code, where I thought worth to highlight, but probably best left in regular font to avoid distracting/overloading the reader with too much details
Aaron Lun (06:45:43) (in thread): > Yes, it will rearrange the elements in the selectize according to the clustering.
Federico Marini (06:47:37) (in thread): > ok, so thisand that
Federico Marini (06:47:58) (in thread): > asking re. the tour elements that are missing
Federico Marini (07:42:13): > I fixed a couple of open things with the docs
Federico Marini (07:42:28): > soon to be done: finalizing the steps of the intro tour
Federico Marini (07:42:53): > the changes which are there will be soon merged with master - so that you do not duplicate the effort:wink:
Aaron Lun (07:43:30): > okay
Federico Marini (07:44:15): > I guess we all will throw a good eye on the consistency issues - which otherwise will be pointed to from the bioc reviewers:stuck_out_tongue:
Aaron Lun (07:44:59): > We can put style comments on the wiki, if someone wants to get started on that.
Federico Marini (07:45:28): > what I also see in some packages would be a contribution code
Federico Marini (07:45:38): > as plain text in the pkg
Federico Marini (07:45:54): > but that is more focused on the content than on the style
Federico Marini (09:30:33): > @Federico Mariniuploaded a file:Screen Shot 2018-02-21 at 15.30.10.png - File (PNG): Screen Shot 2018-02-21 at 15.30.10.png
Federico Marini (09:31:17): > Would we need some label to this selectable input? If so, which one would it be?
Aaron Lun (09:32:39): > ?
Aaron Lun (09:32:46): > Thought it would be pretty obvious that it’s the assay choice.
Aaron Lun (09:33:11): > In all cases, I deliberately set the label to NULL to compact the UI.
Federico Marini (09:33:26): > I am also for no labl
Federico Marini (09:33:39): > seems pretty self explanatory
Federico Marini (09:33:49): > but you know, users will be users:stuck_out_tongue:
Aaron Lun (09:34:05): > Well, if they can’t figure it out, there’s really no helping them.
Federico Marini (09:34:19): > amen to that
Aaron Lun (09:34:20): > I’m sure iPhones don’t have a label above the home button saying “Home”
Federico Marini (09:34:38): > hhihi
Federico Marini (09:34:42): > BTW, iPhones
Federico Marini (09:35:06): > let’s be ready for someone asking (demanding?) mobile device support
Federico Marini (09:35:07): > :smile:
Aaron Lun (09:37:52): > They’d need incredible motor control to handle the lassos.
Kevin Rue-Albrecht (09:39:24): > Actually, can widget have tooltips in Shiny? I can’t remember
Kevin Rue-Albrecht (09:39:58): > If so, we could maybe drop some of the most common/obvious widget labels
Kevin Rue-Albrecht (09:40:53): > PS: I sent a screenshot using Fede’s hosted app from my phone a while back. Doesn’t look too bad!
Kevin Rue-Albrecht (09:41:37): > (Just coming out of a 1:40 lab meeting, got an hour gap to our journal club:sleepy:)
2018-02-22
Kevin Rue-Albrecht (06:30:55): > :musical_score:Siiiiilent slaaaack, hooooollly slaaaack:musical_note:
Kevin Rue-Albrecht (06:40:56) (in thread): > going back through the messages to make my to-do list (finally got rid of the presentation to give this morning, freedoooom!)
Aaron Lun (06:41:32): > Bogged down with BK trees and k-mer distances.
Aaron Lun (06:41:36): > Hardcore bioinformatics.
Kevin Rue-Albrecht (06:42:41) (in thread): > i agree with@Federico Marinihere: from the first issue opened on GitHub, we’ve been referring to the panel-linking feature as ‘brushing’ between ourselves, and I think this transpired into the user-visible man pages. We’ll need to look at this more closely, if not already dealt with.
Kevin Rue-Albrecht (06:43:35): > :musical_score:Single cells, single cells, single in their well. Oh what fun it is to yield, so little RNA-hey!:musical_note:
Kevin Rue-Albrecht (06:44:00): > :musical_score:Guided by the flow, into different wells, stick a TSO, reverse complement:musical_note:
Aaron Lun (06:46:43): > I think you need a break.
Kevin Rue-Albrecht (06:48:53) (in thread): > just spotted this idea, which I think is a great use of the GitHub wiki.
Kevin Rue-Albrecht (06:50:25) (in thread): > do you have an example package (e.g. link to github code). I’m curious what you mean
Kevin Rue-Albrecht (06:52:35): > you don’t know how much you’re right ^^ although i still haven’t finished that song that i started a few months ago. Playing with my limits both of English and knowledge of library prep kits, to be fair:yum:
Federico Marini (06:54:31) (in thread): > this one is the first one that came to my mind now
Federico Marini (06:54:34) (in thread): > https://github.com/tidyverse/purrr/blob/master/CONDUCT.md - Attachment (GitHub): tidyverse/purrr > purrr - A functional programming toolkit for R
Federico Marini (06:55:19) (in thread): > some were quite more detailed, in terms of detailing the style too - i.e. the contributors are expected to stick to that, be them just us, or people proposing a PR
Federico Marini (06:55:43) (in thread): > but it is not so top priority, I think
Kevin Rue-Albrecht (06:56:51) (in thread): > oh right, that thing. > yeah, i’ve seen that GitHub ‘Insights’ panel called ‘Community’, e.g.https://github.com/tidyverse/purrr/community weirdly enough theirs doesn’t get picked up by GitHub, I suppose one has to use their ‘Propose’ button on the same page to place the file where expected
Kevin Rue-Albrecht (06:57:37) (in thread): > I haven’t made use of this GitHub feature myself yet.iSEE could be a good place to start
Aaron Lun (07:59:15): > Anyway, when you’re done, we’re due for a fresh PR.
Kevin Rue-Albrecht (08:20:09): > yep. just to be clear: currently active branches for documentation fixes areplotting2(me) andheat(charlotte and federico) , right?
Charlotte Soneson (08:21:46): > As far as I can remember I am only working on heat, yes.
Kevin Rue-Albrecht (08:27:54): > oh right, sorry I saw Fede’s name only because you merged his commits intoheat
Charlotte Soneson (08:28:38): > Ah no, I just meant that I am not working on any other branch:slightly_smiling_face:I think Aaron has also pushed some things intoheat.
Kevin Rue-Albrecht (08:34:59): > riiight, i get you now
Charlotte Soneson (13:40:51): > something weird seems to be going on in the.createColorPanelForColumnPlotsfunction in theheatbranch: the last argument has been changed tono_rows=FALSE(logical scalar), but as far as I can see it is still called withfeasibility, which is a list. Any insights as to which one to keep? It passes the check/travis testing, but nothing actually runs…
Charlotte Soneson (13:41:57): > Which made me think of another thing: if we want to increase the robustness of the code, I guess we should do all our function calls with named arguments rather than relying on the positional information…
Aaron Lun (13:46:08): > Oh, forgot to change it.
Aaron Lun (13:46:23): > Uh - get rid offeasibilityand add a check for whether there are any rows inse.
Charlotte Soneson (13:49:28): > ok
Charlotte Soneson (13:51:01): > you mean columns?
Charlotte Soneson (13:51:11): > or rows ofcolData(se)?
Aaron Lun (13:51:47): > No, number of features. It needs to knkow this in order to determien whether or not to provide colour-by=-feature options.
Charlotte Soneson (13:51:54): > Ah ok, sorry
Aaron Lun (13:51:54): > If there are no features, those options will be disabled.
Charlotte Soneson (14:40:26): > There should now be support for color capping also for non-centered data inheat
Kevin Rue-Albrecht (14:46:29): > i’m back into cleaningplotting.Rtonight
Kevin Rue-Albrecht (14:47:16): > @Kevin Rue-Albrechtshared a file:Plotting.R (and shared) to-do list - File (Canvas): Plotting.R (and shared) to-do list
Unknown User (14:47:16): > @Kevin Rue-Albrecht commented on @Kevin Rue-Albrecht’s file https://community-bioc.slack.com/docs/T35G93A5T/F04BQMU91TR : my to-do list for this evening - File (Canvas): Plotting.R (and shared) to-do list
Kevin Rue-Albrecht (14:48:32): > (although looking back at this list, I notice that i’ve also listed a few extras that I’m not dealing with like stuff inheat)
Kevin Rue-Albrecht (14:50:56): > damn I was trigger-happy indeed: 40 hits for\code{list}
Kevin Rue-Albrecht (15:06:52): > checking out theExperimentColorMapdoc, as well, how do you call alogical(1), i.e., a TRUE/FALSE argument?
Kevin Rue-Albrecht (15:17:06): > How does that look:https://github.com/csoneson/iSEE/wiki/Documentation-guidelines ?
Kevin Rue-Albrecht (18:07:23): > I think I stumbled onto a weird behaviour with regard to self-brush (although I’m not sure how much this feature will actually be used): > - first of all, set redDim1 (for instance, I assume all plots are affected the same way at this point) to self-brush with transparency > - brush some points:it doesn’t apply the brush effect - if you then set the brush to color:it does refresh and apply the colored brush effect - if you double-click, the brush disappears (good), but the colored effect remains (not good ) > - even setting the brush effect to ‘—’ does not remove the color
Kevin Rue-Albrecht (18:08:06): > - even setting the brush effect to another plot does not remove the color
Kevin Rue-Albrecht (18:08:28): > - one has to brush nothing on the other plot to finally reset redDim1
Kevin Rue-Albrecht (18:10:25): > - at that point you can restart playing with the self-brush issue, and show that it’s not jus atransparency issue, but that color brushes also don’t get apply on self-brush if you don’t toggle the self-brush effectafter brushing data points
Kevin Rue-Albrecht (18:13:49): > I’ve addressed most of my objectives for tonight, with the exception of: > - de-centralization of @importFrom statements. > - misleading variable names
Aaron Lun (18:20:26): > Re brush effect: probably one of the checks in the brush observer was too aggressive.
Aaron Lun (18:20:57): > I put in some protection to avoid unnecessary replotting when the brush was unchanged, or was absent, etc.
Aaron Lun (18:21:40): > But it may have done something unexpected, especially after our latest round of refactoring with lassos.
Aaron Lun (18:24:47): > … wait. Does it draw the brush box? Because then it’s probably an issue of forgetting to force re-rendering
Kevin Rue-Albrecht (18:24:56): > it does draw the brush box
Kevin Rue-Albrecht (18:25:34): > and at least the first lasso point (i haven’t bothered drawing an entire lasso, and already closed R for today)
Aaron Lun (18:25:58): > Yes, theno_rerendersetting is probably the culprit.
Aaron Lun (18:26:24): > Easily fixed by checking whetherpObjects$memory[[mode0]][i0, .brushByPlot] == plot.name.
Aaron Lun (18:26:44): > Though perhaps I should just get rid of the staged re-rendering entirely.
Kevin Rue-Albrecht (18:26:58): > alright, i hope you’re up for it, because i’m off to catch up on some sleep now
Aaron Lun (18:27:07): > I’ll do it tomorrow.
Kevin Rue-Albrecht (18:27:13): > good with me ^^
2018-02-23
Aaron Lun (04:21:28): > I’m going around raising issues and assigning jobs for all of us.
Kevin Rue-Albrecht (04:21:36): > perfect!
Aaron Lun (04:28:29): > Okay<!channel>, we all have a bit of work to do. I forgot Feb has fewer days than the other months so it’s a bit more of a rush than I thought, but let’s do what we can.
Kevin Rue-Albrecht (04:45:44): > Further aesthetic parameters #146:point_size,font_size,legend_positionprobably should be plot-specific (hence in a collapsibleBox -> assign to@Federico Marini?), I’d argue that default color for ‘colorless’ point should probably be an app-wide setting
Kevin Rue-Albrecht (04:46:30): > might be the the first one, but I’n expecting more to crop up in the future
Kevin Rue-Albrecht (04:48:29): > in somewhat lower priority, I’d also encourage another unit-test writing hackhaton (possible while the package is in Bioc review)
Federico Marini (10:05:58): > Is there a reason for having## Not run:as top line of iSEE-main.R?
Kevin Rue-Albrecht (10:06:07): > codecov
Federico Marini (10:06:08): > or just remnant of copy-pastes
Federico Marini (10:06:10): > ah
Kevin Rue-Albrecht (10:06:36): > but we should remove it, in fact
Kevin Rue-Albrecht (10:07:36): > initially i was worried that we wouldn’t be able to test a function that launches an interactive app (I did it a long time ago and wondered whyR CMD checkwas hanging … )
Kevin Rue-Albrecht (10:08:05): > but in our case the function returns the app, so if we catch the result the app isn’t launched
Kevin Rue-Albrecht (10:08:40): > … which means we can ‘unit test’ the function, and therefore there is no point excluding it from code coverage
Federico Marini (10:15:05): > k
Kevin Rue-Albrecht (10:16:47): > before I forget: if you do it, don’t forget to remove the pendant on## Not run:at the end of the file. > nevermind: I just checked and someone deleted it already
Kevin Rue-Albrecht (10:17:02): > so yeah the top one is a leftover of that other deletion then
Kevin Rue-Albrecht (10:18:01): > that’s actually good news: i’ve always thought that 40% was the coverage withiSEE()excluded, which is a wild understimation of our actual package coverage
Kevin Rue-Albrecht (10:19:12): > … considering thatiSEE()is 1,330 lines long
Kevin Rue-Albrecht (10:20:15): > for the record, we’re not doing too badly, once the shiny app itself is excluded: > > * Checking function lengths....................... > The longest function is 1330 lines long > The longest 5 functions are: > iSEE() (R/iSEE-main.R, line 88): 1330 lines > iSEE_server() (R/iSEE-main.R, line 254): 1157 lines > .panel_generation() (R/dynamicUI.R, line 42): 265 lines > .track_it_all() (R/codetrack.R, line 21): 142 lines > .make_heatMapPlot() (R/heatmap.R, line 1): 137 lines >
Kevin Rue-Albrecht (10:20:31): > 265 lines for the longest _actual_function
Aaron Lun (10:31:28): > Still pretty fucking long, tho
Aaron Lun (10:31:42): > BTW you can get a masssive spike in code coverage just by runningpanel_generation.
Aaron Lun (10:32:11): > this will run many of the internal UI functions. I would guess probably +10%
Federico Marini (11:47:44): > <!channel>theannotation_factorybranch is there for you - i might not be able to react promptly over the weekend, please feel free to edit yourselves small things if you spot somethng out
Aaron Lun (11:48:51): > Looks pretty neat
Federico Marini (11:50:25): > grazie mille:slightly_smiling_face:
Federico Marini (11:50:51): > (pretty empty of people in the building, glad to see someone interacting with me at least here eheh)
Kevin Rue-Albrecht (11:51:26): > (I can start singing on the channel again…:stuck_out_tongue:)
Charlotte Soneson (11:52:06): > Everyone is in the Biergarten celebrating the unexpected victory?
Aaron Lun (11:52:23): > Huh
Aaron Lun (11:52:27): > What?
Aaron Lun (11:52:30): > Oh
Aaron Lun (11:52:33): > right. Snow stuff
Charlotte Soneson (11:52:52): > Ice, in this case
Aaron Lun (11:52:53): > You europeans and your cold sports.
Federico Marini (11:54:06): > eheheh, no clue. but still, quite some people stick to the saying “freitag ab eins macht jeder seins”
Charlotte Soneson (11:54:29): > I thought it was “kein Bier vor vier”
Federico Marini (11:54:35): > well
Federico Marini (11:54:50): > here people do disappear at lunchtime on Fr
2018-02-24
Aaron Lun (11:07:20): > All mentions of color will now be American. None of thisunonsense.
Kevin Rue-Albrecht (11:07:29): > :stuck_out_tongue:
Kevin Rue-Albrecht (11:07:59): > sorry.. i still have PTSD from my PhD supervisor ^^
Aaron Lun (11:58:43): > BTW no one touchplotting.R; it is going through some SERIOUS refactoring. No line untouched, and many tests will probably break.
Kevin Rue-Albrecht (11:58:56): > ok i’m not working on it today
Aaron Lun (13:57:38): > Well, that was a failure.
Aaron Lun (13:57:44): > Turned out to be much more difficult than I thought.
Aaron Lun (13:57:58): > In the end, it didn’t seem worth it.
Aaron Lun (13:58:45): > Nonetheless, I was half-way through the refactoring, so it wasn’t all for nothing - if you look atplotting.Rnow, you’ll see that the plotting code is now fully separated from the generation ofplot.data.
Aaron Lun (13:59:25): > This should make it much easier to follow -.create_plotnow should only addggplot -related commands
Kevin Rue-Albrecht (13:59:55): > that sounds really nice, thanks!
Kevin Rue-Albrecht (14:02:26): > unrelated: i’m headed into one of those massive timeout on Travis.. I haven’t pushed code there in a loooong time, and now that I pushed a bug fix, it’s updating alot of packages ^^
Aaron Lun (14:54:37): > 400 commit monster in da house.
2018-02-25
Kevin Rue-Albrecht (09:46:06): > Btw, sorry for my inactivity this weekend. Taking a bit of heat here in Oxford. Apparently, restricting my iSEE work to evening and weekend is still impacting too much on the group’s activities:expressionless:Can’t wait to see the app used without credit for the development effort.. > Is ‘Additional documentation notes to fix #143’ likely to raise issues for the Bioc reviewer, if we submitted without closing it? I hope to handle it an evening this week
Aaron Lun (09:46:35): > Can push back a week, if you like.
Aaron Lun (09:46:47): > I have a stats course to give so will be out of action from Wednesday onwards.
Kevin Rue-Albrecht (09:47:18): > Hmm.. is there any other issue than 143 holding us back?
Aaron Lun (09:48:05): > I will need some assistance with 147 later.
Aaron Lun (09:48:56): > 146, I mean.
Kevin Rue-Albrecht (09:49:14): > ah, i was getting confused looking at 147 ^^
Charlotte Soneson (09:49:18): > I’m working on the heatmap zooming, but it also turned out to be more complicated than I imagined. For some reason my brushing doesn’t return the actual coordinates of the box, but rather values in [0, 1], as opposed to what it does for the other plot types. Haven’t managed yet to figure out why this is happening and if there’s anything to do about it. but that’s where I’m at.
Aaron Lun (09:49:37): > okay
Charlotte Soneson (09:49:41): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-25 at 09.23.46.png and commented: brush output for different plot types: - File (PNG): Screen Shot 2018-02-25 at 09.23.46.png
Kevin Rue-Albrecht (09:50:21): > hm.. maybegeom_rasteruses relative coordinates in [0,1] for performance
Aaron Lun (09:50:22): > Maybe it’s because it’s set up in a grid?
Kevin Rue-Albrecht (09:50:51): > or more precisely what Aaron said
Charlotte Soneson (09:51:03): > But the funny thing is that if I addcoord_cartesianwith theactual coordinates (not in [0, 1]) to a heatmap outside of the app, everything works fine
Charlotte Soneson (09:51:32): > So it knows about the values, but somehow doesn’t get them out of the brushing
Kevin Rue-Albrecht (09:51:48): > oh.. i wonder if therescale-ing things to a final (visual) [0,1] range, like what happens forscale_size
Kevin Rue-Albrecht (09:53:08): > I remember having trouble understanding how to talk toscale_size_*to control the mapping from ‘data’ values from a range [min, max] map the scale ‘plotted’ sizes in a range [0,1]
Kevin Rue-Albrecht (09:53:47):
a numeric vector of length 2 that specifies the minimum and maximum size of the plotting symbol after transformation.
Charlotte Soneson (09:54:19): > Hmm…ok
Kevin Rue-Albrecht (09:54:20): > arf.. one of those ‘obvious’ things.. once you know how they work
Aaron Lun (09:55:20): > Don’tq uite understand what you mean by addingcoord_cartesianand everything working fine.
Charlotte Soneson (09:56:51): > So, if I take the heatmap code from the app and just add+coord_cartesian(xlim = c(5.5, 10.5), expand = FALSE), for example, it zooms in to show 5 cells.
Charlotte Soneson (09:57:06): > As it should
Aaron Lun (09:57:29): > Right.
Aaron Lun (09:57:43): > So it’s strictly a problem with defining the coordinates of the brush.
Charlotte Soneson (09:57:58): > Yes, I think so
Kevin Rue-Albrecht (09:58:01): > so.. the brush returns values in the range [0,1] instead, right?
Aaron Lun (09:58:06): > Iscowplot_gridconfusing matters?
Charlotte Soneson (09:59:07): > Ah, that’s a good point, didn’t think about that. I don’t know whether that has an effect, but it might.
Kevin Rue-Albrecht (10:01:04): > we’re not just talking about multiplying/dividing coordinates by the number of rows/columns in the heat map, once we figure out which components (brush,plot,inputs) work with which range?
Charlotte Soneson (10:02:50): > Perhaps for the cells, but it looks like the y limits would be for the whole grid (including all annotations and the colorbar). And even for the cells I suspect it includes the y-axis label etc.
Kevin Rue-Albrecht (10:03:38): > uff, I mean ouch
Charlotte Soneson (10:04:02): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-25 at 16.03.29.png and commented: For example, now if I try to brush the entire heatmap I get this - File (PNG): Screen Shot 2018-02-25 at 16.03.29.png
Kevin Rue-Albrecht (10:04:57): > damn, that looks like a pain:confused:
Charlotte Soneson (10:07:51): > Yes:confused:I thought it was just me missing some setting somewhere, which would have been much easier
Aaron Lun (10:08:18): > My money’s on cowplot_grid fucking things up.
Charlotte Soneson (10:09:09): > Yes, mine too now…it probably just places everything nicely in a [0,1]x[0,1] grid and forgets about the actual values.
Charlotte Soneson (10:09:39): > But without it, the annotations would be a real pain
Kevin Rue-Albrecht (10:10:08): > If I get that right, it would mean figuring out the outter margins around all the panels, and the offset added by each new annotation bar, to only then convert brushes in/out of the plot. Is that an accurate description, or am I missing something?
Charlotte Soneson (10:11:33): > I guess that would be enough.
Charlotte Soneson (10:15:08): > The$rangeslot of the brush also also encodes theactual size of the panel, so if I resize my browser window this will change. So I guess that’s not going to help here.
Aaron Lun (10:18:01): > Is there a way to do this in pure ggplot - e.g., add another raster on top of the first with the annotations?
Aaron Lun (10:18:14): > This should be pretty easy - it’s just anothergeom_rastercall, right?
Charlotte Soneson (10:18:36): > The problem there is with the color scales - you can only have one per plot. This is why I generate one plot per raster now
Aaron Lun (10:18:51): > ah fuck
Aaron Lun (10:19:03): > dammit. this would be easy with base plot.
Kevin Rue-Albrecht (10:19:47): > that, and the fact that multiple ggplot layers are laid outexactly on top of each other. I wouldn’t know how to create multiplealigned panels in pure ggplot
Aaron Lun (10:20:38): > What about two plots as separate UI elements, right on top of each other?
Aaron Lun (10:20:55): > Guess the gene name width would be difficult to control, thouh.
Charlotte Soneson (10:20:57): > I don’t think they would be aligned either
Charlotte Soneson (10:21:14): > Yes. And the annotation name can be longer than the gene names as well
Kevin Rue-Albrecht (10:22:25): > to be perfectly aligned, they’d need to both show the same legend, title, etc. I don’t think there’s any ‘nudge’ effect to substitute that effect of aesthetics and legends
Charlotte Soneson (10:26:21): > I agree. One option would be to go with a plotly-based thing likeiheatmapr(https://ropensci.github.io/iheatmapr/index.html ). However, I think that would be considerably slower since it is completely interactive (I tried it once and it was really struggling to make a heatmap with hundreds of genes and tens of samples)
Aaron Lun (10:27:57): > Hm.
Aaron Lun (10:28:21): > Does cowplot_grid return any information at all?
Charlotte Soneson (10:34:42): > Well, it is a ggplot object, so there is information in there, but it is not structured as a regular ggplot (e.g, thegg$datais just a 2x2 matrix: (0, 1, 0, 1), instead of the plot data)
Charlotte Soneson (10:36:20): > gg$layerscontains a lot of information, but it is almost unreadable compared to a typical ggplot.
Charlotte Soneson (10:38:12): > You can get out the relative y-coordinate at which the two panels (heatmap, annotations) meet
Aaron Lun (10:39:04): > Hm. Okay.
Kevin Rue-Albrecht (10:39:07): > might be worth checking how responsive the cowplot maintainer is:Maintainer: Claus O. Wilke <wilke@austin.utexas.edu>if we can save us time answering some of our questions
Kevin Rue-Albrecht (10:40:52): > not sure how to read the latest update, but it could bePackaged: 2017-12-17that is pretty recent
Charlotte Soneson (10:47:32): > Ok, I think it is pretty clear that it iscowplot. I made a small app with two plots: one generated with onlyggplot2, one with the sameggplot2call wrapped incowplot::plot_grid:
Charlotte Soneson (10:47:41): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-02-25 at 16.46.17.png - File (PNG): Screen Shot 2018-02-25 at 16.46.17.png
Kevin Rue-Albrecht (10:47:55): > -_-
Aaron Lun (10:51:40): > yep, looks fucked up to me.
Aaron Lun (11:06:35): > Also, anyone looking for an easy job can try to integrate general visual parameteres into the plotting functions, e.g., point size, font size, legend position, etc.
Kevin Rue-Albrecht (11:26:17) (in thread): > Yep, i was brainstorming that one before starting writing my usual first draft code, for once. > Are those parameters already implemented as inputs anywhere (e.g. a modal)?
Kevin Rue-Albrecht (11:28:31) (in thread): > I haven’t implemented much of the input UI so far, but I suppose I could copy-paste from another modal to get started.
Aaron Lun (11:28:55) (in thread): > All of them are in the UI now.
Kevin Rue-Albrecht (11:28:56) (in thread): > The alternative, that I’d like to avoid for many reasons, would be to have those arguments as part of theiSEE()call
Kevin Rue-Albrecht (11:29:00) (in thread): > ahhh ok cool
Aaron Lun (11:29:29) (in thread): > There are unrelated issues with the latestheat, regarding extraction of theassayNamesto use inassayColorMap; it is hard to separate them.
Kevin Rue-Albrecht (11:30:03): > btw, I just pulled again just now, but apparently my latest pull onheatearlier today caused the following error > > > app <- iSEE(sce) > Error in `[[<-`(`**tmp**`, .plotParamPanelOpen, value = FALSE) : > object '.plotParamPanelOpen' not found >
Aaron Lun (11:30:23): > That shouldn’t exist anymore.
Aaron Lun (11:30:28): > Re-install and try again.
Kevin Rue-Albrecht (11:31:06): > Cool. I was just building the latest pull again with that hope
Kevin Rue-Albrecht (11:31:18): > yep, all good now
Kevin Rue-Albrecht (11:33:57): > woooow.. nice job on the visual parameters indeed, they seem all ready to go
Kevin Rue-Albrecht (11:36:40): > Ok then. I can’t really do it right in the next couple of days, to sort out some urgent local work, but I’d say the heat map situation probably needs at least another week of work, so I’ll try to match that with some work in the evening sometime this week on both issue 143 and the visual parametes. Both pretty straightforward
Aaron Lun (11:40:30) (in thread): > Goddammit! The fact that you can give names toassayColorMapis a real pain in the arse.
Kevin Rue-Albrecht (11:40:43) (in thread): > is it?
Aaron Lun (11:41:09) (in thread): > Yes, because it means that the creation of the color scale must be able to accessseandall_memory.
Aaron Lun (11:41:28) (in thread): > Which means that I need to pass these arguments all the way down the chain, just for the colormap.
Kevin Rue-Albrecht (11:41:43) (in thread): > hmm:confused:
Aaron Lun (11:42:05) (in thread): > It would be ideal if a “rationalized” ECM guarantees that the index and name accessors are the same.
Aaron Lun (11:42:23) (in thread): > rationalized meaning that it’s matched up to the SE object.
Kevin Rue-Albrecht (11:42:48) (in thread): > eh.. hence my dream of seeing ECM integrated toSummarizedExperiment
Kevin Rue-Albrecht (11:43:04) (in thread): > (some day)
Aaron Lun (11:43:23) (in thread): > I really doubt that’s going to happen. But in the meantime, what do we do to check ECM validity?
Kevin Rue-Albrecht (11:43:42) (in thread): > I put it at the start ofiSEEI believe
Kevin Rue-Albrecht (11:43:57) (in thread): > isColorMapCompatible(colormap, se, error = TRUE)
Aaron Lun (11:44:00) (in thread): > Just piggy-back off that and make sure every entry of@assaysis of the same length asassayNames(se).
Kevin Rue-Albrecht (11:44:31) (in thread): > i was just thinking that instead/in addition to that, we could.sanitizeECM
Aaron Lun (11:44:46) (in thread): > This needs to be user-visible, therwise the code tracker won’t work.
Kevin Rue-Albrecht (11:44:57) (in thread): > maybe 1) check overall validity, 2) sanitize
Kevin Rue-Albrecht (11:47:41) (in thread): > basically, what you’d like, tell me if i’m wrong, is giving a valid ECM that may only define a subset of assays, and getting back a ’rationalized` ECM where the missing assays color maps have been filled in with their appropriate default, right?
Kevin Rue-Albrecht (11:48:11) (in thread): > (so that the number and the name return the same thing)
Aaron Lun (11:50:09) (in thread): > That would be ideal and the desired user-level behaviour anyway.
Aaron Lun (11:53:33) (in thread): > Well, I’m going to rewrite the code assuming that’s the case.
Aaron Lun (11:53:47) (in thread): > Passing so many arguments down is untenable.
Aaron Lun (12:12:40) (in thread): > In the meantime, I have “solved” the problem by adding more...arguments.
Aaron Lun (12:25:34) (in thread): > Probably just call itsynchronizeAssaysor something.
Aaron Lun (12:26:31) (in thread): > Or indeed, we can just put it all on the user. Tell them that if they plan to setassayColorMapwith names, the order of the names must be the same as the order in the SCE object.
Aaron Lun (12:27:00) (in thread): > A bit unforgiving, but hey, life is tough.
Kevin Rue-Albrecht (12:55:44) (in thread): > got abruptly offline for a bit - sorry. I’ll add thatsynchronizeAssayson my list, doesn’t sound too difficult either, I just need the time to sit down and get it done
Aaron Lun (17:42:53) (in thread): > A bit messy, but that should be enough, right?
2018-02-26
Charlotte Soneson (00:52:21) (in thread): > No, I don’t think so. You still don’t know the coordinates of the actual plot area within each panel (the coordinate you get will include margins etc)
Aaron Lun (04:28:31) (in thread): > We could probably guess the y-axis coordinates - the margins aren’t that big, right? The x-axis is harder to guess.
Charlotte Soneson (04:30:45) (in thread): > No, they are not, and there are (most of the time) not so many values on the y-axis (we only need it to the closest 0.5 resolution).
Aaron Lun (04:34:39) (in thread): > Okay. So if we restrict ourselves to y-axis selection and zooming, and combine it with brushing on the cells, then we can mostly achieve what we wanted to anyway, and pretend that it was all on purpose.
Charlotte Soneson (04:35:25) (in thread): > You mean brushing in the heatmap or in the other plots?
Aaron Lun (04:38:05) (in thread): > In other plots, to select samples in the heatmap.
Charlotte Soneson (04:40:01) (in thread): > Ok, yes, that would be easy to integrate. But in that case, isn’t it better to have the same approach for the genes (i.e., selecting in other panels, or in theselectizeInputof the heatmap)? To avoid questions about why the zoom doesn’t work both ways (and since in my opinion gene selection is already easier than cell selection due to the multiple ways of inputting them and the limited number).
Aaron Lun (04:41:34) (in thread): > Zooming is always nice, right? If you make the entire heatmap but you just want to inspect a subset, but you don’t want to have to delete the rows (and then add them back in later).
Charlotte Soneson (04:42:56) (in thread): > Yeah, sure. I will see how it works with the y-axis zooming.
Federico Marini (16:25:38): > <!channel>couple of things:
Federico Marini (16:25:55): > I did update to the current state of master for the demo server
Federico Marini (16:26:26): > got some errors, checked the log: we probably have to add two more importfrom from ggplot2
Federico Marini (16:26:46): > i.e.geom_rasterandelement_text
Federico Marini (16:27:02): > for the rest, seems to be working fine:wink:
Federico Marini (16:28:00): > this would be if we want to show off a little the link, again:http://shiny.imbei.uni-mainz.de:3838/iSEE
Federico Marini (16:28:43): > I was thinking recently on advertising one more round, this time with something like “ready to take your samples with a lasso”:stuck_out_tongue:
Kevin Rue-Albrecht (16:29:11): > “catch’em while they’re (pl)ot”
Federico Marini (16:30:01): > drop it like it’s (a heatmap) plot?
Kevin Rue-Albrecht (16:31:56): > :musical_score:“hit me with you best plot”:musical_note:(https://www.youtube.com/watch?v=0JRgHol94Xc )
Aaron Lun (16:41:08): > really digging out the classic hits, aren’t you.
Kevin Rue-Albrecht (16:41:48): > not a big fan of what’s released these days, tbh
Aaron Lun (16:43:00): > Lol
Aaron Lun (16:43:11): > Anyway, I am FINALLY going to demo this tomorrow.
Aaron Lun (16:43:18): > Unless I get hit by a bus.
Kevin Rue-Albrecht (16:43:42): > ahhh looking forward to hear more feedback
Federico Marini (16:47:50): > ismasterfine for you Aaron?
Aaron Lun (16:48:10): > I’ll be usingheat, and just do some handwaving about the features that aren’t fully integrated yet.
Federico Marini (16:49:02): > okiedokie
2018-02-27
Aaron Lun (06:15:00): > Mostly good, though these guys can’t be satisfied. It was like, ‘can you run a PCA on the subset’? And I was like, no, stop being a lazy bum.
Kevin Rue-Albrecht (06:16:06): > ah ok, so it’s not only here that people want do bioinformatics without any more programming, like at all ^^
Kevin Rue-Albrecht (06:17:51): > I was just going to say that we could work on exporting the differentplot.data… but we’re basically giving people the whole script to regenerate them all inall.coordinates
Kevin Rue-Albrecht (06:18:49): > then it’s just a matter of training users to find where each bit of information is stored, but that PCA is basically only a handful of commands away
Federico Marini (07:10:56) (in thread): > But hey, apart from the lazy bums we will never be able to satisfy entirely
Federico Marini (07:11:08) (in thread): > this means we cover pretty much all minus a few needs
Federico Marini (07:11:28) (in thread): > I think this is quite remarkable - and not just because we did it:stuck_out_tongue:
Federico Marini (07:12:10): > General consideration: maybe we could also think of Genome biology as a target for the manuscript
Federico Marini (07:12:18): > they are strongly focused on sc
Federico Marini (07:12:34): > and our friend Charlotte has pretty much an abo over there
Federico Marini (07:12:35): > :slightly_smiling_face:
Kevin Rue-Albrecht (08:23:02): > That’s true. > To be fair Oxford Bioinfo has a high IF, but it may not be worth the pain of compressing iSEE in two pages
Kevin Rue-Albrecht (08:23:16): > A larger/more flexible format may do us more good
Aaron Lun (08:24:54): > Note that there are few bugs inheat; ironing them out now.
Charlotte Soneson (08:26:46): > Anything affecting the actual heatmaps? I finally have a functional y zoom for heatmaps (in a new branch), just need to clean it up a bit and then I’ll push it.
Aaron Lun (08:30:01): > Nope.
Aaron Lun (08:39:43): > Done.
Charlotte Soneson (09:45:14): > Ok, heatmap (y-axis) zooming pushed toheatzoom
Aaron Lun (09:47:07): > Sweet, will check it out in a bit.
Aaron Lun (09:48:41): > Need to get past my neurosis about using t-tests for single-cell data first.
Charlotte Soneson (09:49:02): > Haha, you can use edgeRQLF, it is ok:slightly_smiling_face:
Aaron Lun (09:49:25): > I do pairwise comparisons between all pairs of clusters, so t-tests are undeniably fast and convenient.
Aaron Lun (09:49:40): > The issue is how to collate the results across blocking levels, i.e., batches.
Aaron Lun (09:50:16): > I’ve been getting the t-test p-values and combining them with Stouffer’s method, which seems to work okay. But I’m not 100% sure.
Charlotte Soneson (09:50:48): > Doing a t-test within each batch?
Aaron Lun (09:51:00): > Yes, within each batch, between cell types.
Aaron Lun (09:52:13): > It seems sensible, except in the edge case of one cell from each group per blocking level. In this case, linear models would be fine but it’s not possible to do t-tests. There’s also a fudge whenever there is only one cell in a group, where we switch from Welch to Student.
Aaron Lun (09:52:27): > Results in some discontinuities in behaviour that shouldn’t really be there…
Charlotte Soneson (09:52:52): > Sure. I guess you cluster everything together after some batch removal?
Aaron Lun (09:53:00): > Yeah
Aaron Lun (09:53:20): > Well, that’s the theory.
Charlotte Soneson (09:55:09): > Out of curiosity, do you get different results if you test based on the full, batch-corrected data?
Charlotte Soneson (10:00:56): > Btw, some tests are failing with the new structure of the plot commands (e.g.,cmd_listinstead ofcmd)
Aaron Lun (10:03:31): > Yes, I’m not surprised.
Kevin Rue-Albrecht (10:05:39) (in thread): > That’s probably my duty to deal with asap:slightly_smiling_face:
Aaron Lun (10:05:53): > Probably needs a look over to update everything. I had my hands full with the internal docs.
Aaron Lun (10:07:43) (in thread): > Haven’t tried that. Batch correction by regression prior to clustering has its own problems when the clusters are not split evenly across batches; and MNN correction does not (easily) give values that you can directly interpret in a DE analysis.
Aaron Lun (10:10:14) (in thread): > I guess the anticonservativeness associated with DE on the batch-corrected values would be minor at highn ; assuming that the batch correction was accurate, though.
Charlotte Soneson (10:11:47) (in thread): > Yes, perhaps. And the accuracy of the batch correction will have a pretty big effect on the clustering already.
Aaron Lun (10:12:55): > Heatmap zoom looks good, but the gene names run past the color legend and through the annotations upon zoom.
Aaron Lun (10:13:42): > Also, it seems to re-render when a brush is made on the heatmap - not sure why - and it crashes when trying to import from an empty table. Both of these are mine to fix, probably.
Charlotte Soneson (10:14:10) (in thread): > Ah yes, I noticed that at some point, had forgotten about that. I am not sure why it happens, will have a look.
Charlotte Soneson (10:15:45): > The re-render is probably me setting some parameter wrong. Didn’t try the import from an empty table…but I noticed that the gene selection box empties when I ask to suggest ordering.
Aaron Lun (10:16:40): > And the brush refuses to die! Lol.
Charlotte Soneson (10:16:52): > What?
Kevin Rue-Albrecht (10:17:00): > btw, unrelated but while i think of it: a lot of people are throwing “Morpheus” (https://software.broadinstitute.org/morpheus/ ) at me in comparison to iSEE. I think we already mentioned it before on the channel, but I’m not sure anymore
Aaron Lun (10:17:19): > What’s its selling point?
Kevin Rue-Albrecht (10:17:57): > you can upload your data matrix in there, and dynamically browse and compute certain types of plots, including PCA I believe
Aaron Lun (10:18:23): > … or you could do that in R.
Kevin Rue-Albrecht (10:18:26): > I was put off by the UI when I first opened it, so I didn’t dig very deep
Kevin Rue-Albrecht (10:18:59): > there are a bunch of famous demo data sets to play with
Aaron Lun (10:19:31): > The heatmap is not particularly informative to me.
Aaron Lun (10:19:38): > I mean, it’s got every gene in there.
Kevin Rue-Albrecht (10:19:45): > my feeling too
Aaron Lun (10:20:41): > I think it’s just trying to make a replacement for R for people who can’t be bothered to use R.
Aaron Lun (10:20:57): > I mean, click on hierarchical clustering.
Kevin Rue-Albrecht (10:21:07): > hehehe
Kevin Rue-Albrecht (10:21:14): > I wouldn’t have put it better
Aaron Lun (10:21:40): > default distance metric is wrong - 1 - pearson’s is not a distance.
Aaron Lun (10:21:45): > (Its 0.5*sqrt(1- p))
Aaron Lun (10:22:12): > Limited set of linkage methods - and in any case, complete would be a better default.
Kevin Rue-Albrecht (10:22:25): > I just wonder what’s worse: wrong code, or wrong documentation
Aaron Lun (13:37:45): > @Charlotte Sonesonjust modifying your observer set-up a bit; heatmaps now have no brush observers, but have their own custom zoom observer.
Charlotte Soneson (13:39:03): > Great, thanks:slightly_smiling_face:
Aaron Lun (13:48:20): > As a general rule, only the point-based plots can really share the code for defining observers; heatmaps and row statistics tables need their own dedicated observer-defining code.
Aaron Lun (14:42:13): > Also note the clustering doesn’t work; we need to cache the expression submatrix, not theplot.data.
Aaron Lun (14:48:58): > Probably also need to think about downsampling - I noticed considerable delays when trying to draw a lasso on arow
Aaron Lun (14:49:35): > I know how to do this quite efficiently by aggregating over a fine grid; the question is how to turn this on and off…
Aaron Lun (14:52:53): > @Kevin Rue-AlbrechtI suggest working off theheatzoombranch when you get back around to doing this; too many changes elsewhere.
Aaron Lun (14:55:54): > Downsampling won’t need a knn search, either, so it’ll be strictly linear, and super-fast. I’ll have to think of how to make it aware of the plot size, tho.
Kevin Rue-Albrecht (16:09:42): > Thanks for theheatzoombranch note: indeed, I was getting uncertain where to pick things from
Federico Marini (16:45:17): > To avoid confusion, I’d merge theannotation_factoryalready into master
Aaron Lun (17:43:12): > sounds sensible.
Aaron Lun (17:43:19): > Also, I HAD A VISION.
Aaron Lun (17:43:46): > We can provide support for custom plotting functions, in the same manner that we provide support for custom annotation.
Aaron Lun (17:44:31): > Power users can supply a function that accepts a SCE object and some column indices, and produces a ggplot and some coordinates.
Aaron Lun (17:45:22): > This can then integrate directly into a custom plotting panel.
Aaron Lun (17:45:55): > This would kill many birds, especially for people who can’t shut up about making PCA plots on subsets of data.
Aaron Lun (17:46:14): > They can just define their own function that takes the SCE and some column indices and makes a PCA.
Aaron Lun (17:47:31): > I’m not going to implement this before the BioC submission, but just FYI to answer any crazies who want to do weird shit.
Federico Marini (17:49:35): > prophet Aaron, I support your vision:slightly_smiling_face:
Federico Marini (17:52:05): > btw, we are almost making art with our branch network plot
Kevin Rue-Albrecht (17:57:14): > :star-struck:
Aaron Lun (18:16:38): > Could someone mergemasterback intoheatzoomand deleteannotation_factory.
Kevin Rue-Albrecht (18:21:41): > not sure whether I’m the best placed to do it, after dropping out a couple of days, but I’ve pulled both branches, got a one-line conflict in NAMESPACE when merging (> importFrom(AnnotationDbi,species)), and I’mR CMD checking now
Kevin Rue-Albrecht (18:22:53): > if all goes well, I’ll push back and deleteannotation_factory, otherwise i’ll leave it to someone who’s more up to speed with the last few commits
Kevin Rue-Albrecht (18:23:32): > @Kevin Rue-Albrechtuploaded a file:R CMD check - File (PNG): R CMD check
Kevin Rue-Albrecht (18:25:28): > is that expected? can’t see it in the latest Travis run (neitherheatzoomnormaster)
Kevin Rue-Albrecht (18:26:22): > local check:1 error | 2 warnings | 1 note. Error are expected due to unit tests out of date
Kevin Rue-Albrecht (18:26:53): > I guess i’ll push then
Aaron Lun (18:44:03): > Well, nonte of theINTERNAL_*.Rdfiles should be committed.
Aaron Lun (18:44:19): > But we should probably fix those warnings nonetheless.
2018-02-28
Federico Marini (03:38:16) (in thread): > That is my extraimportFrom, located in annotation.R
Charlotte Soneson (03:59:59): > So I think the problem with y axis ticks/labels extending below the heatmap can be solved by usingscale_y_discreteinstead ofcoord_cartesian. However, in this case we instead end up withggplotwarnings since it has to drop all NA values introduced byscale_y_discrete…Not sure which is worst…
Kevin Rue-Albrecht (04:08:03): > it’s a big “if”, but wecould usesuppressWarnings()with care, if we’re sure enough that we’re not expecting other (helpful) warnings
Kevin Rue-Albrecht (04:11:52) (in thread): > ok, can I leave that to you then? I got a fairly large ‘raw’ R script to turn into markdown report today:sleepy:
Charlotte Soneson (04:12:20): > Sure. We could maybe wrap just the execution of the relevant (plot) commands in the app but leave it in the tracked code. Would be nicer if the problems withcoord_cartesiancould be solved somehow though:slightly_smiling_face:
Kevin Rue-Albrecht (04:14:07): > I’ve thought about this already for the.griddotplot(now.square_plot) issue withwidthandheight, but I generally prefer the bold honesty of showing all messages and warnings to users
Aaron Lun (04:29:19): > I’m happy to not use coord_cartesian, and to literally subset the expression matrix beforehand, if that helps.
Aaron Lun (04:29:51): > We’re not using the expression matrix for anything else downstream, so it should be fine.
Charlotte Soneson (04:36:56): > Yeah, seems to work too. I’ll push it in a minute.
Charlotte Soneson (04:47:14): > Ok, it’s there. One thing that we can’t do in the heatmaps is “successive zooming”, i.e. zooming in the zoomed plot, since it only records the coordinates and translates them to indices in the original matrix. Maybe this is fine, otherwise I guess we need to store the currentplot.datain memory somehow.
Aaron Lun (05:27:50): > Right… we could cache that…
Aaron Lun (05:28:15): > What do you need for this?
Charlotte Soneson (05:30:42): > So, what’s happening now is that I’m getting the y-coordinates from the selection, mapping them back to the “local” coordinates for the heatmap panel (basically 1:nbrgenes), and then subset theplot.datato to only the rows where the gene is one ofrownames(value.mat)[selected coordinates]. So I guess we’d just need to change 1:nbrgenes to 1:nbr_remaining_genes, and store the remaining gene names.
Charlotte Soneson (05:31:26): > Or maybe just store the selected rows ofvalue.matand recreate everything from there.
Aaron Lun (06:48:58): > We can store the remaining gene names in the custom zoom observer for the heatmap. In fact, I would suggest doing the translation to the genesin the heatmap zoom observer directly, so we only need to hold on to the remaining gene names.
Aaron Lun (06:56:54): > So basically, the.zoomDatafor heatmaps would store the remaining gene names.
Aaron Lun (06:57:04): > Or the X:Y thereof.
Charlotte Soneson (09:15:07): > Yes, I think that sounds reasonable.
Aaron Lun (10:02:39): > TheiSEEinput now sanitizes the SCE object in a separate function (see.sanitize_SE_inputiniSEE-extras.R). This includes flattening nested DataFrames; adding row/column names, if absent; addingsizeFactorsandisSpikeinformation into the column- and row-level metadata, respectively; and removing non-atomic types that cannot be put in adata.framefor ggplotting.@Federico Marini, this requires us to putse_cmds(seeiSEE.R) in the code tracker so that users can reproduce the renaming (if any commands are involved).
Kevin Rue-Albrecht (10:06:15): > nice.. hadn’t thought about the non-atomic types
Aaron Lun (10:11:35): > There is a better solution that avoids redefining things, but it can’t be implemented without a few changes to the plotting functions, and I’m unwilling to do that ATM.
Aaron Lun (10:11:52): > This will do for now, I guess.
Aaron Lun (12:09:21): > ouch snowman-making induced frostbite
Federico Marini (14:58:49) (in thread): > if I got your implementation right, it is just the matter of passing that to.track_it_allas parameter, andsprintfing that too, right afterse <- se
Aaron Lun (16:05:03) (in thread): > Yes.
Federico Marini (17:22:59) (in thread): > thanks then for making my part easier:wink:
Federico Marini (17:23:36) (in thread): > it is now on heatzoom, together with the correct extra part for handling the error when we do not convert the gene id
2018-03-01
Charlotte Soneson (03:54:22): > I think the repeated zooming should work now (heatzoombranch).@Aaron Lun: since there is no brush for the heatmap I guess we no longer need the brush commands in the heatmap function (l.157-167), right? Or am I missing something? I commented them out for now and it seems to work.
Aaron Lun (04:19:37): > I thought I already deleted the brush observer.
Aaron Lun (04:20:35) (in thread): > Cool
Aaron Lun (04:42:17): > Oh, sorry, you’re talking about heatmap.R.
Aaron Lun (04:42:31): > Uh. That’s right, the box doesn’t get redrawn.
Charlotte Soneson (04:47:34): > Ok, then I’ll remove it
Aaron Lun (07:25:58): > I’ll spend this afternoon linking the heatmaps with incoming brushes.
Aaron Lun (07:26:16): > Do we always restrict?
Aaron Lun (07:26:46): > Otherwise we’d need to add a new annotation bar.
Charlotte Soneson (07:27:33): > Maybe it would be useful to have a way of just highlighting the cells (just to see where they fall in the ordering), but on the other hand you can plot the annotation in the colData plot and highlight them there.
Aaron Lun (07:29:22): > Well, it wouldn’t hurt to have coloring options, as long as you can make a new annotation bar. This may require the use of.safe_field_nameto make sure it doesn’t clash with any existing fields. I don’t think the transparency option makes much sense, unless you tell me that it’s possible. (If it is, it would make life easier for me, as I can just recycle the UI elements).
Charlotte Soneson (07:30:24): > I don’t know whether you can use alpha for the fill ingeom_raster, but I suppose that is possible
Aaron Lun (07:30:40): > Well, I’ll just do what’s easiest for me, and you can tell me what to take out afterwards.
Aaron Lun (07:31:25): > BTW,"ZoomData"will need separate documentation for heatmaps indefaults.Rd.
Aaron Lun (07:31:46): > I presume it’s always a contiguous array from X:Y?
Aaron Lun (07:32:02): > where X:Y refers to the subset of the features in the full heatmap?
Charlotte Soneson (07:32:08): > Yes, it should be.
Aaron Lun (07:32:28): > Okay - just edit the zoom section when you have the time.
Aaron Lun (08:53:59): > Brush UI added for heatmaps.
Aaron Lun (09:01:25): > Suggest using.get_brush_selection()to identify the columns.
Aaron Lun (09:01:54): > I haven’t added brush observers; will do so soon, it’s my turn to do this teaching course.
Aaron Lun (09:16:49): > joined up.
Aaron Lun (09:33:31): > @Charlotte SonesonNote that the UI is currently broken as centering/scaling commands are now checkboxes with TRUE/FALSE options.
Aaron Lun (09:33:54): > This makes more sense than radio selections, but needs some updating toheatmap.R.
Kevin Rue-Albrecht (09:35:20): > feeling so trapped in work over here:sweat:, can’t wait to join back in the fun
Charlotte Soneson (09:57:55) (in thread): > Ok, I’ll fix that.
Aaron Lun (10:34:28): > I will now reflect on downsampling.
Charlotte Soneson (11:39:33) (in thread): > I changed the heatmap code to take the logical values as input. However, the plots are not triggered to rerender if you unclick a box (or for that matter, if you reclick it again after unclicking it once).
Charlotte Soneson (12:03:50) (in thread): > Ok, turns out that it’s thereq(input[[cur_field]])in theobserveEventthat causes this.
Aaron Lun (12:11:28) (in thread): > Ah, shit.
Charlotte Soneson (12:11:49) (in thread): > I just asked it not to check this for these two inputs
Charlotte Soneson (12:12:08) (in thread): > It will not give any ugly red text if it is not selected anyway
Aaron Lun (12:13:21) (in thread): > I wonder if it renders the heatmap twice upon re-rendering.
Charlotte Soneson (12:14:07) (in thread): > Ok, why?
Aaron Lun (12:18:20) (in thread): > Thereqwas part of the protection against double-rendering, to avoid it trying to re-render the plot when the UI re-rendered via.panel_generation(e.g., when deleting/adding/moving panels). The idea was that re-rendering would initially generate aNULLin theinput, which is replaced by the actual value upon re-rendering. This would result in re-triggering of the observer and undesirable re-rendering of existing plots, and thereqwas designed to protect against that. However, I realize thatobserveEventhasignoreNULL=TRUEby default, soNULLs should never even get inside the observer. If you deleted the observerreqs iniSEE, do we get double-rendering of the heatmap (stick a print command in therenderPlotcall for heatmap to find out)? If it doesn’t, maybe we can delete allreqcalls iniSEE-main.R.
Charlotte Soneson (12:22:42) (in thread): > Regardless of thereq()in the heatmapobserveEventit renders only once at startup, and doesn’t rerender if I move the plot around. And this even withignoreNULL=FALSE(which I guess we need for the logical values to trigger rerendering
Charlotte Soneson (12:23:39) (in thread): > Well, actually it seems to work withignoreNULL=TRUEas well
Aaron Lun (12:25:29) (in thread): > Oh good.
Aaron Lun (12:26:03) (in thread): > WellignoreNULL=TRUEis the default, and we should keep it that way mostly.
Charlotte Soneson (12:26:30) (in thread): > Yes. I first thought that was the reason why the deselection didn’t trigger any update
Aaron Lun (12:26:41) (in thread): > So we can probably delete thereqcalls in the heatmap observers.
Aaron Lun (12:26:55) (in thread): > And in all observers, probably. At least forobserveEvent;observemay require some more care.
Charlotte Soneson (12:27:13) (in thread): > Ok. I’ll try this tonight.
Aaron Lun (12:29:05): > Will need to compact the UI elements to fit an extra “Downsample” checkbox.
Charlotte Soneson (14:23:25) (in thread): > Soreqdoes make a difference (albeit not for the heatmap).
Charlotte Soneson (14:24:14) (in thread): > Withreq, at startup: > > [1] "I'm here in redDimPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in colDataPlot" > [1] "I'm here in rowDataPlot" > [1] "I'm here in heatmap" > [1] "I'm here in featExprPlot" >
Charlotte Soneson (14:24:33) (in thread): > Withoutreq: > > [1] "I'm here in redDimPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in colDataPlot" > [1] "I'm here in rowDataPlot" > [1] "I'm here in heatmap" > [1] "I'm here in featExprPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in redDimPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in colDataPlot" > [1] "I'm here in rowDataPlot" > [1] "I'm here in featExprPlot" > [1] "I'm here in featExprPlot" >
Charlotte Soneson (15:55:13): > legend position, point alpha and point size inputs are now integrated inheatzoom.
Charlotte Soneson (15:55:34): > What font size do we want to control with the Font size UI? Axis labels or titles?
Kevin Rue-Albrecht (17:01:41): > thanks for doing that Charlotte, sorry that I’m just taken away these days, or I should say evenings
Kevin Rue-Albrecht (17:04:02): > I would keep the titles to a fixed size, they should usually be fairly concise, I’d expect users to fiddle more withaxis.text(x and y), to reduce them and enlarge the plotting area
Aaron Lun (18:09:36): > Agreed
Aaron Lun (18:09:57) (in thread): > Ich bien confused.
Aaron Lun (18:10:21) (in thread): > Ah - probably due to the updateSelectize!
Aaron Lun (18:11:35) (in thread): > You can tell because the featExprPlot get hit particularly hard.
2018-03-02
Kevin Rue-Albrecht (03:03:25): > @Charlotte Soneson: i think you can markhttps://github.com/csoneson/iSEE/issues/146 as completed, right? Sorry again I didn’t get to it in the last few days
Charlotte Soneson (03:04:14): > Still have to fix a couple of things (font size, default color). Then I’ll close it:slightly_smiling_face:
Kevin Rue-Albrecht (03:05:40): > Ok no worries. I’m regularly pulling the updates onheatzoom, but didn’t spin up the app to check. It’s crazy how fast we’re moving, between the four of us!
Kevin Rue-Albrecht (04:09:12): > Btw, I demo-ed iSEE to a wet-lab collaborator who happens to be used to FlowJo, and she picked up on something that I’ve mentioned in the past: the “add new [panel]” buttons at the top left are a bit counter-productive, in the meaning that there is now a number of them and they take up some space that is not frequently used. Is it possible to have a dropdown of them in the left panel? Something akin to “File > New window”?
Kevin Rue-Albrecht (04:10:12): > (except in this case it would collapse all of them into one “Add new panel” button, that would open a dropdown of buttons, one for each panel type
Aaron Lun (04:12:02): > Yes, it is possible to have the button open a modal that allows the user to choose the type of panel to add.
Kevin Rue-Albrecht (04:13:56): > Cool. I think that would unclutter the UI a bit.
Kevin Rue-Albrecht (04:23:47): > Btw, I’ll spend some time either tonight or this weekend to deal with issue ‘Additional documentation notes to fix #143’. I’ve got yet another presentation to prepare for a lab meeting on Wednesday, but I think I deserve enjoying a slice of iSEE after the past week head down in analyses:upside_down_face:
Kevin Rue-Albrecht (04:29:00): > @Federico Marini: just looking at the vignette now, I was wondering if it’d be more helpful to add the FAQ point “Copy to clipboard” directly at the top of the aceEditor modal, and removing it from the vignette.
Aaron Lun (04:29:20): > But people can just control-a….
Kevin Rue-Albrecht (04:29:31): > exacly my point
Kevin Rue-Albrecht (04:30:26): > Instead of writing it in the vignette–that people never check at the moment theyactually need it–, having a short statement at the top of the modal
Kevin Rue-Albrecht (04:31:17): > ..or maybe it’s too much
Aaron Lun (04:32:52): > Oh, right. Yep.
Kevin Rue-Albrecht (04:33:16): > .. actually.. i’m just thinking now… at that point we might as well give them the button.. it takes less space than a statement:joy:
Kevin Rue-Albrecht (04:33:45): > especially if the statement has to cover multiple OSes (Command_A, Ctrl-A, whatnot-A)
Kevin Rue-Albrecht (04:39:50): > Completely independent question, I can’t help but ask at this point: how did guys strike a deal with your respective PIs to find time and freedom foriSEE ? I understand that Steve got a bit overprotective of my time recently, considering that we have a few projects going on, and that it gets sometimes hard justifying to collaborators that I spend time of something that benefits me more than the group (at least on a short term). > I suppose some of you already somewhat ‘earned’ their freedom through a decent number of first-author publications, is that the trick?
Aaron Lun (05:04:06): > Yes.
Aaron Lun (05:04:21): > I pump out papers like a factory.
Aaron Lun (05:04:26): > It hurts sometimes.
Aaron Lun (05:07:36): > I also work 12/7, which helps.
Aaron Lun (05:08:21) (in thread): > Don’t want to have to set up unnecessary observers, and dependencies (xclip requires system libraries).
Aaron Lun (05:09:20): > And John (and probably Mark) are also pretty well-established, so I suppose that makes them less anxious.
Kevin Rue-Albrecht (05:11:34): > Ok, I understand there are a few factors at play. > I noticed your 12/7 indeed. I guess the fact that I invested a bit of that to help wrap up manuscript from previous positions didn’t play in my favour either.
Kevin Rue-Albrecht (05:12:25) (in thread): > Agreed. Too much pain for too little gain
Aaron Lun (05:15:48): > Despite what PIs might think, they don’t have your weekends or after-hours time. So I think it’s fair enough to work on your own projects then.
Aaron Lun (05:16:16): > I happen to work on some official projects during this time, but that’s a choice, rather than an expectation.
Kevin Rue-Albrecht (05:17:49): > Exactly, that was one of the hot topics recently, which to be honest hasn’t really been resolved to my satisfaction yet, but I think we’re moving in the right direction to find a balanced situation acceptable on both sides
Aaron Lun (05:20:56): > As long as I do something that’s useful to the group, John doesn’t really care if his name is on it.
Aaron Lun (05:21:03): > For the greater glory!
Kevin Rue-Albrecht (05:21:38): > I openly admit that software projects likeiSEE can turn into infinite time sinks that occasionally distract from ‘main’ projects through the excitement of new features or the frustration of bug fixes, but I also think it’s only fair to have a ‘recreational’ programming hobby, more so if it makes me more familiar with concepts relevant to daily projects
Kevin Rue-Albrecht (05:21:43): > hahaha yeah
Kevin Rue-Albrecht (05:24:44): > Well Steve’s openly admits that iSEEwill be useful to the group, but then I must also admit that he’s also absolutely right to be wary of the time that I invest in itat the expense of projects that me and him are accountable for. > A balancing act for me and him, obviously
Aaron Lun (05:28:19): > Anyway, I’m at a teaching course right now, so I won’t be making any commits, because I amSICK
Aaron Lun (05:28:22): > SICK of it.
Aaron Lun (05:28:25): > It disgusts me.
Aaron Lun (05:28:31): > I can’t write code on this shit.
Kevin Rue-Albrecht (05:28:48): > hahaha it can get painful at times
Kevin Rue-Albrecht (05:29:11): > catch up later
Federico Marini (08:36:31): > Picking up on your question on dealing with time,@Kevin Rue-Albrecht. > I can’t say I did write that much lately for my stuff, but in my case it is a mix of the following: > - the parties that are or were funding me always got back what they expected, in time -> i.e. ahead of time done, then I got some freedom > - for the fellowship I started, I have some headstart on WP1 -> but who doesn’t. > - I am as of now still taking care of some older stuff which I am happily soon delegating to my successor-me, and this plays as a bonus on managing some time in a protected way for me > - also relevant, we currently are in an interim phase for bioinformatics and/or the head of the institute. this means again lil more freedom and less tasks to take over, mostly due to this vacancy
Federico Marini (08:37:00): > plus, well, developing tools that later could save my time is a well done investiment
Federico Marini (08:37:02): > :slightly_smiling_face:
Federico Marini (08:39:45): > to be honest, sometimes I do envy you guys having strong & established bosses, which might be a constant forge of ideas
Federico Marini (08:40:15): > whereas here, well, sometimes there’S not so much, how to say…, “healthy pressure”
Federico Marini (08:41:12): > to get things out on time i.e. now or at least very soon. That’s one of the reason why I was so enthusiastically jumping in the blue and started working on this very-much-side project with you
Kevin Rue-Albrecht (09:03:37): > Thank Federico, it’s all good feedback and experience to hear back from you all.
Charlotte Soneson (09:17:59): > I don’t know if I can contribute much additional wisdom…Mark generally seems happy for us to do things that are useful to the community and to the group in general. I have plenty of collaborations as well, but as long as I deliver there I think he doesn’t care muchwhen I do what. And the knowledge gained via projects like this can be useful for other projects in the group in the future too.
Federico Marini (09:24:23): > Dear Santa,I know it is ahead of time. Please bring me an institute director that thinks like Mark.Yours sincerely,Federico
Charlotte Soneson (09:26:51): > Haha, shall I ask him if he wants to move to Mainz?
Federico Marini (09:30:19): > We have already had an australian colleague who liked it a lot here
Federico Marini (09:30:27): > if this is a prior good enough:smile:
Charlotte Soneson (09:31:30): > Mark’s Canadian, but sure…:wink::stuck_out_tongue_winking_eye:
Federico Marini (09:32:18): > ehehehe
Federico Marini (09:32:29): > it’s friday for me too:smile:
Federico Marini (09:32:47): > or I am just giving you low informative priors
Kevin Rue-Albrecht (09:42:05): > Thanks all. That’s really helpful comments to put things a bit more in perspective for me. > Without going into details, as Steve pointed out to me, I’ve been delivering on local collaborations, but I’ll have to slow down a bit on the iSEE side, if I want to pick up the pace and properly lead our own projects.
Aaron Lun (10:36:20): > I would suggest flying off the handle.
Aaron Lun (10:36:25): > I often do so.
Charlotte Soneson (11:51:17): > I pushed a commit toheatzoomthat changes the font size of axis titles and labels by multiplying default sizes (10 for text, 12 for title) by the provided font size scaling factor. For the “default” point color, I am wondering if we could useupdate_geom_defaults("point", list(colour = "red"))rather than setting the color for each plot separately (and thus keeping track of whether things are brushed or colored based on something else). I wonder where we should put it though. And it also needs to go in the code tracker.
Charlotte Soneson (11:52:22): > But I guess it has to go in the plot code, since it can be different for each panel
Charlotte Soneson (12:07:31): > Anyway, I did it like this and it seems to work
Aaron Lun (13:04:51): > Looking at it now, I’m back on a real computer.
Aaron Lun (13:07:16): > I’m thinking of changing them to text input boxes, to allow for very large or very small point/font sizes.
Aaron Lun (14:41:26): > On a different note, I’ve streamlined the row table links to avoid defining a separate choice.
Aaron Lun (14:49:19) (in thread): > Done.
Aaron Lun (14:56:45): > Also switched to text input.
Federico Marini (15:06:11): > would a numericInput be a better choice?
Federico Marini (15:06:25): > instead of plain text
Federico Marini (15:06:48): > i think this would check automatically that numbers are input
Aaron Lun (15:07:45): > Oh. Yes, that would be good.
Aaron Lun (15:07:57): > On an unrelated note, attempting to brush on the heatmap blew up my computer.
Federico Marini (15:08:03): > uh
Federico Marini (15:08:14): > I’ll try on mymac
Federico Marini (15:08:15): > :smile:
Aaron Lun (15:09:06): > boooo
Aaron Lun (15:09:15): > I hope the entire screen just explodes
Federico Marini (15:09:32): > ahah
Federico Marini (15:09:40): > screencast live to prove it:stuck_out_tongue:
Aaron Lun (15:09:42): > Make sure you press the import button.
Aaron Lun (15:10:02): > This is what happened to me; I imported the first 100 features from the row table, and then tried to brush.
Aaron Lun (15:10:57): > The brush got created but then my computer stopped.
Federico Marini (15:11:24): > whole computer or just R?
Aaron Lun (15:11:38): > Whole computer.
Aaron Lun (15:11:41): > Must have run out of memory.
Federico Marini (15:11:43): > asking because I am also writing the material for a course:stuck_out_tongue:
Federico Marini (15:11:53): > let me save first then
Aaron Lun (15:12:06): > And I’m running R command line, so there’s no Rstudio interference.
Federico Marini (15:13:04): > you mean just click and hold on the heatmap now?
Federico Marini (15:13:24): > or also actively passing somewhere else?
Aaron Lun (15:13:31): > Huh?
Aaron Lun (15:14:31): > Well, now it’s gotten even worse; it refuses to make heatmaps with more than one gene.
Aaron Lun (15:18:22): > okay… now it’s back, and brushing is okay.
Aaron Lun (15:18:27): > Not sure what the hell happened there.
Federico Marini (15:20:16): > well i’m back
Federico Marini (15:20:19): > from rebooting:smile:
Aaron Lun (15:20:24): > Oh?
Aaron Lun (15:20:28): > Did it crash?
Federico Marini (15:20:32): > ran out of mem
Aaron Lun (15:20:36): > Interesting.
Federico Marini (15:20:50): > @Federico Mariniuploaded a file:Screen Shot 2018-03-02 at 21.15.04.png - File (PNG): Screen Shot 2018-03-02 at 21.15.04.png
Aaron Lun (15:20:51): > I, on the other hand, am now fine.
Aaron Lun (15:20:58): > Hm.
Federico Marini (15:21:10): > could be due to having excel open, still.
Federico Marini (15:21:27): > ok i am joking
Aaron Lun (15:21:41): > About the rebooting?
Federico Marini (15:22:04): > no i forced it
Federico Marini (15:22:13): > but I was monitoring the ram usage
Federico Marini (15:22:16): > and it boosted up
Aaron Lun (15:22:29): > during the brushing.
Federico Marini (15:22:32): > before becoming unresponsive
Federico Marini (15:22:33): > yep
Aaron Lun (15:22:37): > Ah shit.
Aaron Lun (15:23:49): > I wonder why that is?
Federico Marini (15:25:37): > no clue
Federico Marini (15:25:52): > somehow the plot error gets too big
Aaron Lun (15:26:11): > there’s a plot error?
Federico Marini (15:26:16): > i don’t know if it would help to have an observer that just checks the object size?
Federico Marini (15:26:29): > and messages it out to console
Aaron Lun (15:26:55): > There is no way that the objects themselves are responsible for this.
Aaron Lun (15:27:02): > Nothing should even hit more than a couple of MB.
Aaron Lun (15:27:11): > ggplot, or the underlying data.
Federico Marini (15:27:36): > so where is the black hole coming from?:disappointed:
Federico Marini (15:27:58): > we don’T do anything too funky..
Aaron Lun (15:28:25): > To be clear, did you have problems when you brushed, or when you tried to zoom?
Federico Marini (15:30:03): > i managed to zoom
Federico Marini (15:30:04): > in
Federico Marini (15:30:07): > and back out
Federico Marini (15:30:20): > probably without noticing whether it was due to that
Aaron Lun (15:30:40): > okay, so when did it become unresponsive?
Federico Marini (15:31:03): > shortly after me checkign the ram usage
Aaron Lun (15:31:12): > Hm.
Federico Marini (15:31:16): > i tried also to pass the brush from another object
Aaron Lun (15:31:30): > Well, that shouldn’t be working at all yet.
Federico Marini (15:31:33): > e.g. from red dim, picking only some samples
Federico Marini (15:31:35): > indeed
Aaron Lun (15:34:11): > Okay, it’s fine for me, I’ve been tracking the memory usage and it seems okay - though it oscillates by +/- 500 MB, don’t know where that’s coming from.
Federico Marini (15:34:39): > damn
Aaron Lun (15:35:03): > We could throw in somegc()calls, maybe that would help.
Federico Marini (15:37:12): > could help, but still i am little puzzled where this could stem
Aaron Lun (15:45:22): > ¯*(ツ)*/¯
Aaron Lun (15:45:37): > @Charlotte Soneson@Kevin Rue-Albrechtdo you see issues with excessive RAM usage when brushing the heatmap?
Aaron Lun (15:46:31): > Specifically: runexample(iSEE), set up the heatmap to import from the row table, import the first 100 rows, and try brushing on the heatmp. This caused my computer to freeze and eventually caused@Federico Marini’s Rstudio to become unresponsive.
Aaron Lun (15:47:06): > Unfortunately, I can’t reproduce it anymore, so if you guys don’t see it, I’ll consider it a one-off.
Charlotte Soneson (15:49:10): > It seems to work fine for me. I see no real fluctuations in memory usage and the brushing/zooming works fine.
Charlotte Soneson (15:52:40): > What do you say about the fact that we need to have at least one annotation bar for the heatmap? I know there is a reason, but do you think it is worth trying to solve it somehow or should we keep it like this?
Aaron Lun (15:52:59): > If we didn’t have an annotation bar, how would it be sorted?
Aaron Lun (15:53:15): > It’d just be a heatmap with a whole bunch of stuff, basically.
Charlotte Soneson (15:54:08): > Sure…it just looks a bit weird that you remove the column name from theselectizeInputbut it is still there in the plot.
Aaron Lun (15:55:40): > Yes, this was what I was talking about when I was saying shiny was too laissez-faire with the NULL vs empty strings vs FALSEs in deciding whether or not an input is valid.
Aaron Lun (15:56:06): > In the case without any column names at all, we would have probably thrown avalidateerror anyway. So I didn’t see the need to work around it.
Aaron Lun (15:56:34): > I mean, what do people expect to see?
Charlotte Soneson (15:58:03): > Maybe telling the user that it is still there since you need to have at least one annotation (i.e., having avalidate) would lead to less questions of the type “It doesn’t react when I do this”
Aaron Lun (16:03:05): > Well, you can try to do it. You’ll have to remove thereqin the observers, though. This was the major problem, because the server-side selectize initializes to NULL and this was initially causing issues with double-rendering. Perhaps it’s not a problem anymore, if you make sureobserveEvent’signoreNULLis set toTRUE.
Aaron Lun (16:11:33): > Or specifically - it wasn’t initializing to NULL, it was initializing to an empty string, which was even worse as this bypassed theignoreNULL. So we’d always get double-rendering. This is actually the case when selectizeInput is empty; it doesn’t give a NULL, it gives an empty string. Real pain in the ass.
Aaron Lun (16:41:48): > Anyway, our priority is get this merged tomaster, fix all the tests, update the docs, and get this into BioC-devel.
Aaron Lun (16:42:00): > I will clean up the internal docs tomorrow - I am exhausted right now.
Kevin Rue-Albrecht (17:53:07) (in thread): > I don’t see any particular issue brushing on the heat map. Running through RStudio. What do you guys use to track memory usage?
2018-03-03
Aaron Lun (06:22:36) (in thread): > I’m going viafreeon the command line.
Kevin Rue-Albrecht (06:28:29) (in thread): > thanks
Aaron Lun (08:23:26): > I’ve merged the beast
Aaron Lun (08:26:28): > Just need to link the brushes with the heatmap now.
Aedin Culhane (09:35:16): > @Aedin Culhane has joined the channel
Federico Marini (09:37:05): > Hi@Aedin Culhane, welcome!
Aaron Lun (09:49:43): > Anyone want to do some brush linking?
Kevin Rue-Albrecht (09:50:18): > can’t do today: handling a last minute rush for a manuscript
Federico Marini (09:51:32): > wish I could. I have the following situation with girlfriend with cold, and 2/3 kinds with gastroenteritic flu
Federico Marini (09:52:14): > i.e. I will try to take care of them without getting myself sick:disappointed:
Federico Marini (09:52:37): > f*ing february - not a single full week…
Federico Marini (09:55:24): > btw
Federico Marini (09:55:41): > the memory monster happened again
Aaron Lun (09:58:09): > Dammit
Aaron Lun (09:58:47): > Can you track memory usage throughout the app?
Kevin Rue-Albrecht (10:01:42): > I recently dug up thishttps://gist.github.com/netj/526585/36515c55a3b25232bddfdd51e43cadc1a5f296de to track peak memory usage for specific command lines. > Might be hackable, if not directly applicable, to Shiny apps (especially if R is run from the command line, like Aaron does)
Kevin Rue-Albrecht (10:01:52): > @Kevin Rue-Albrechtshared a file:memusg - File (Plain Text): memusg
Aaron Lun (10:02:24): > I suspect a bit of alt-tabbing should be sufficient in this case.
Federico Marini (10:02:39): > actions taken: > > library(iSEE) > example(iSEE,ask=F) > > added the 100 feats > then, brush on heatmap just selecting. double click to zoom in. zoomed out. then asked to reorder the features, and I see the usage goes from 1.3 G to 17G
Aaron Lun (10:02:53): > YES
Aaron Lun (10:02:58): > I know what it is now.
Federico Marini (10:03:04): > alttabbing
Aaron Lun (10:03:12): > It’s because the re-ordering is fucked
Aaron Lun (10:03:23): > it’s trying to build a distance matrix on the melt’d dataframe.
Aaron Lun (10:03:27): > which is obviously huge.
Federico Marini (10:03:30): > no script, but thanks@Kevin Rue-Albrecht, it can be useful in other situations
Federico Marini (10:03:41): > :kiss:to the problem solver:slightly_smiling_face:
Federico Marini (10:03:49): > careful, I can infect:stuck_out_tongue:
Aaron Lun (10:04:06): > @Charlotte Sonesoncan you return the expression matrix from the heatmap rather than the meltedplot.data?
Federico Marini (10:04:59): > I’ll join you later - on duty with kids:slightly_smiling_face:
Charlotte Soneson (10:15:33): > Ok, I changed.make_heatMapPlotto returnvalue.mat(which is correctly scaled and centered) instead ofplot.data. I pushed to a new branchheatbrush. I can not promise that I can look at the brushing today, I need to finish another thing first.
Aaron Lun (11:42:03): > That seems to have solved the memory problem.
Aaron Lun (11:42:37): > Though not before I forgot to switch branches, resulting in another freeze-reboot cycle.
Federico Marini (14:57:49): > Confirmed, it works:wink:
Federico Marini (14:59:05): > One suggestion, if possible: can we make the features widget “scrollable”? when many genes are in it, it b oosts terribly the space taken by the widget
Kevin Rue-Albrecht (15:08:19): > not built-in that i remember.. but there may be some JavaScript magic to enable that
Aaron Lun (17:29:41): > Probably doable with some selectize options, though you’ll have to trawl through them.
Aaron Lun (17:33:02): > https://github.com/selectize/selectize.js/blob/master/docs/usage.md - Attachment (GitHub): selectize/selectize.js > selectize.js - Selectize is the hybrid of a textbox and box. It's jQuery based and it has autocomplete and native-feeling keyboard navigation; useful for tagging, contact lists, …
Aaron Lun (17:34:30): > Nothing obvious comes to mind, though.
2018-03-04
Charlotte Soneson (08:02:19): > heatbrushnow supports restrict and transparent brushing. I don’t know about the coloring, it seems thatgeom_rasterdoesn’t supportcolor(whilegeom_tileandgeom_rectangledo), wo if we want that we may need to add another layer.
Charlotte Soneson (08:03:53): > There’s probably a better way of managing the selected cells, currently it will just doplot.data$X in c(.....)where the vector can be very long, which will not look good in the code tracker.
Aaron Lun (09:13:25): > Right. That was probably incomplete advice on my part, I forgot about the need for code tracking.
Aaron Lun (09:31:58): > Transparencies… suck.
Aaron Lun (09:32:17): > I can’t even imagine a case where anyone would want to use that.
Aaron Lun (09:35:18): > I would go with restrict or an extra annotation bar with colours indicating whether it is a selected cell or not.
Aaron Lun (09:35:30): > In the meantime, I will try to improve the brushing.
Aaron Lun (09:41:26): > Can we make the annotation bar transparent instead?
Charlotte Soneson (09:41:49): > Sure
Aaron Lun (09:43:36): > Okay, give me a bit, I’m just fiddling with the final brush details.
Aaron Lun (10:27:02): > Okay, done.
Aaron Lun (10:50:38): > Note that transparency of the heatmap is sort of useless anyway if the middle colour is white.
Charlotte Soneson (11:20:56): > Transparency is now shown in annotations instead of heatmap. I moved the commands around a bit to keep the factor levels of annotations after the restriction.
Aaron Lun (11:45:06): > That’s fine, but make sure you move brush_cmds after theevalcall on setup_cmds, otherwise we’ll have a spaghetti of evaluation order. Also update the code tracker.
Aaron Lun (11:46:09): > Furthermore, I’m doing an imminent push that will break the heatmap as center/scale will no longer be TRUEs, but rather a character vector containing “Centered” or “Scaled”
Aaron Lun (11:46:18): > On the plus side, the heatmap will now respect NULL inputs.
Aaron Lun (11:51:15): > This means that it will briefly render with the validation message before actually generating the heatmap.
Aaron Lun (11:53:25): > Actually - that’s a bad idea, it won’t work properly with the server-side selectize updating.
Aaron Lun (12:03:53): > Checking it and it will definitely trigger double-rendering due to the updates.
Aaron Lun (12:07:55): > So NULL inputs are only allowed for column data (where selectize updates don’t happen).
Aaron Lun (12:20:58) (in thread): > Also known as “spaghetti source”.
Aaron Lun (12:21:15) (in thread): > Especially prevalent in any language with Goto’s.
2018-03-05
Aaron Lun (04:35:10): > @Charlotte SonesonThe issues I mentioned above have been addressed. Note that the zooming only really needs to hold the first and last index, as you can just print “X:Y” in the code tracker.
Charlotte Soneson (04:49:39): > Ok. Anything still I should fix? I’m in a course all day today so I can’t open up the app and check now.
Aaron Lun (04:51:08): > Probably the color bar.
Charlotte Soneson (04:51:51): > In what sense?
Aaron Lun (04:54:57): > Oh - was it added already?
Aaron Lun (04:55:04): > Brushing by color.
Charlotte Soneson (04:55:22): > Ah ok, I thought you meant the legend color bar.
Charlotte Soneson (04:56:06): > So you would like to add another annotation bar with the coloring if we are brushing with color effect?
Aaron Lun (04:56:15): > Yep.
2018-03-06
Charlotte Soneson (04:42:46): > @Aaron LunI was going to fix the color bar, but pulling the latest updates fromheatbrush, I get an Error: object ‘pchoice_field’ not found. Any ideas where this may come from before I start diving into the details?
Aaron Lun (05:07:04): > It’s an issue with the row data visual parameter box, should be fixed now.
Charlotte Soneson (05:08:44): > Yep, seems fine now.
Charlotte Soneson (06:41:27): > Ok, color brushing should work now
Aaron Lun (06:58:29): > Looks nice.
Charlotte Soneson (07:00:53): > One thing that just came to mind: how will the heatmap play with sparse matrices? In particular the scaling, where I guess it has to convert the input to a dense matrix?
Kevin Rue-Albrecht (07:03:39): > what’s the largest matrix that any of us has visualised as a heat map in the app so far?
Kevin Rue-Albrecht (07:06:27): > I haven’t played much with the heat map view myself, mostly because I’m not sure yet how I’d like to use it. > But I wonder more generally whether we’d expect users to feed massive sparse matrices that could cause that kind of trouble
Kevin Rue-Albrecht (07:07:26): > In most cases, I’d expect reasonably sized ‘slices’ of sparse matrices being passed to the heat map, which should really be a problem to convert to dense matrix
Charlotte Soneson (07:08:31): > Yeah, I guess so (it will always be less than 100 genes with the current settings). Maybe it’s not a problem:slightly_smiling_face:
Kevin Rue-Albrecht (07:13:55): > Without any Googling or deep thinking, we might put amake_sparse_heatmap()function on a ‘to-think’ list, if we identify a need for a specific behaviour.
Aaron Lun (07:25:45): > Well centering makes this point moot, as it is a sparsity destroying operation.
Kevin Rue-Albrecht (07:26:22): > duh. right, sorry
Aaron Lun (07:32:43): > Note that, when you’re referring to list elements in memory, don’t run unlist: > > unlist(pObjects$memory$heatMapPlot[i0,][[.heatMapColData]])) > > It is much easier to do something like this: > > pObjects$memory$heatMapPlot[,.heatMapColData][[i0]] > > which basically takes the.heatMapColDatacolumn (which is a list) and then takes thei0th entry of it.
Aaron Lun (07:35:03): > The reason whyheatMapPlot[i0,][[.heatMapColData]]gives you a list of length 1 is because the entry is technically stored as a list, and if you do a straight[]of a list, you get another list. So you need[[]].
Federico Marini (07:37:06): > Hey<!channel>, this is probably still inofficial, but:http://bioconductor.org/developers/release-schedule/-> now in its draft version
Aaron Lun (07:37:30): > Yes, I’ve noticed. Let’s merge theheatbrushbranch, bump up the codecov and get this done.
Federico Marini (07:38:21): > Is the UI going to be further affected? If not, I can finalize the tour (some elements have been renamed or added/edited)
Aaron Lun (07:39:15): > … probably not before submission.
Kevin Rue-Albrecht (07:39:56): > After the lab meeting tomorrow, I’ll be clear to help polish the last few bits and pieces to get us over the line for package submission
Kevin Rue-Albrecht (07:40:21): > I just won’t be able to contribute major features in the near future
Federico Marini (07:40:46): > Fine for me. I will be able to work in small bursts, since the daycare mum is still sick for the rest of the week, but we are really closing in
Aaron Lun (07:41:16) (in thread): > When brushing to color, the color bar doesn’t appear until I’ve actually brushed something in the transmitting plot. This is the correct behaviour, but it means that the number of annotation bars is overestimated when deciding how to zoom. Is it possible to just get the number of annotation bars from the plot object itself? Also happy to support caching of other information, if this is necessary.
Federico Marini (07:42:06): > most likely we have to update a little more the vignette too, but I think I can take care of that, too. In case I see things get worse on the whole sickness side I’ll inform you
Charlotte Soneson (07:50:29) (in thread): > Yes, that’s true. IniSEE-main.R, I only check that there is a transmitting plot and that the effect is color, not that anything is brushed. Maybe one can add a check for the existence of brushed points in there? I thought about it before and the other way seemed to me to actually modify the selection of annotations for the heatmap (in memory), but that would mean that it wouldn’t agree anymore with what the user sees in the interface.
Aaron Lun (07:51:38) (in thread): > It would be easiest for the plot to remember how many annotations were added. Is this information somewhere in the plot object? Otherwise we can cache it elsewhere.
Charlotte Soneson (07:54:52) (in thread): > In the final plot object, my guess is no, since this is after cowplotting.
Charlotte Soneson (07:55:22) (in thread): > Btw, are you trying the very latest version? I fixed one thing with the global-to-local conversion after the first push
Aaron Lun (08:05:25) (in thread): > Global - to -local?
Aaron Lun (08:06:03) (in thread): > In any case, I forgot that we already have the functions to “sneak peek” at the transmitting plot; see.transmitted_brushand its uses in main.
Charlotte Soneson (08:06:11) (in thread): > sorry:slightly_smiling_face:global (as in the coordinates in the [0,1] coordinate system of the cowplot) to local (as in the actual y-coordinate in the heatmap)
Aaron Lun (08:06:45) (in thread): > Yes, I was using the latest; it did seem a bit more accurate.
Charlotte Soneson (08:10:01) (in thread): > it will never be 100% accurate with the current setup - the coordinates are just estimates. But (apart from potentially the case of no selected points in the transmitting plot) it seems to be reasonably accurate.
Charlotte Soneson (08:26:17) (in thread): > Ok, I added a check for brush transmission
Aaron Lun (08:32:57) (in thread): > yep, seems sensible.
Aaron Lun (08:50:52) (in thread): > Best to merge it withmasterwhile everything’s still working.
Charlotte Soneson (08:52:43) (in thread): > Good point, I created a pull req
Charlotte Soneson (09:04:51): > heatbrushbranch now merged intomaster
Charlotte Soneson (11:52:35) (in thread): > space invader?:slightly_smiling_face:
Aaron Lun (12:01:08) (in thread): > Working through the list.
Charlotte Soneson (12:02:33) (in thread): > Ah, good. There are still a few to go…
Aaron Lun (12:11:48): > SettingrunApp(test.mode=TRUE, launch.browser=FALSE)may allow us to immediately cover the entirety ofiSEE.R.
Kevin Rue-Albrecht (12:26:18): > wasn’t it just about assigning the output ofdummy_value <- iSEE(...)?
Aaron Lun (12:26:51): > This runs through theiSEEbut doesn’t actually test the UI or server functions (I think).
Aaron Lun (12:27:06): > TherunAppwill, I think, execute those functions.
Aaron Lun (12:27:31): > Of course, we couldn’t do it normally, because then CHECK would stall as a browser opens.
Kevin Rue-Albrecht (12:28:58): > oh right, indeed, what I suggest would only provide coverage. > I am curious to see howtest.mode=TRUEbehaves (I’ve looked up the man page and it sounds spot on)
Aaron Lun (12:47:00): > Just taught someone about inception-style DataFrames. (FOGHORN)
Federico Marini (13:52:52): > Today I had to refresh a prof about experimental design. In a way that gave me goosebumps. His aim was to correct for batch effects by confounding it totally with treatment dosage
Federico Marini (13:52:57): > :face_palm:
Aaron Lun (13:53:54): > lol
Federico Marini (13:54:14): > he was about to fire up sequencing for 72 samples
Federico Marini (13:54:47): > I dunno what to do. I tell people to turn to us in advance, and then such things are just about to happen
Federico Marini (13:55:17): > I mean, I could refuse myself to look at such data - being the whole thing pointless
Federico Marini (13:55:32): > but then they bitch about us as the ones that are just complaining:smile:
Kevin Rue-Albrecht (15:09:42): > hehehe I don’t know if that beat the postdoc who once agreed to an experimental design with four conditions, a couple of replicates per condition, and came back two weeks later: “here are the first two samples: one sample for each condition: can you give me a fold-change, to see if we’re on the right track?”
Federico Marini (16:27:14): > FO SHO
2018-03-07
Federico Marini (08:01:08): > did you already try iSEE on your (John’s) gastrulation dataset,@Aaron Lun?
Aaron Lun (08:01:19): > Nope.
Federico Marini (08:01:20): > as a nice 100k cells test
Federico Marini (08:01:40): > ran into a tweet now which was mentioning it
Aaron Lun (08:05:15): > Yeah, due to the conference that’s happening around this time.
Federico Marini (09:58:15): > huh. I wanted to get a newer Rdevel
Federico Marini (09:58:25): > but it seems that now there is no 3.5.0?
Federico Marini (09:58:44): > and the so-called r devel is still a 3.4.4?
Kevin Rue-Albrecht (10:14:54): > how do you mean? I have the following: > > > R.version > _ > platform x86_64-apple-darwin15.6.0 > arch x86_64 > os darwin15.6.0 > system x86_64, darwin15.6.0 > status Under development (unstable) > major 3 > minor 5.0 > year 2018 > month 01 > day 26 > svn rev 74163 > language R > version.string R Under development (unstable) (2018-01-26 r74163) > nickname Unsuffered Consequences > > R.version.string > [1] "R Under development (unstable) (2018-01-26 r74163)" >
Federico Marini (10:15:12): > yep. but if you get it now
Federico Marini (10:15:19): > it is seen as 3.4.4
Kevin Rue-Albrecht (10:15:24): > ah..
Federico Marini (10:15:27): > i have a december version
Kevin Rue-Albrecht (10:15:30): > the joys of R-devel I guess
Federico Marini (10:15:33): > just wanted a newer one:smile:
Federico Marini (10:15:37): > joy joy
Kevin Rue-Albrecht (10:15:48): > Back to the future… or not in this case ^^
Kevin Rue-Albrecht (10:16:26): > ah no, actually, it is: Back (3.4) to the future (newer release)
Kevin Rue-Albrecht (10:16:45): > i’m a bit brain fried after the lab meeting ^^
Aaron Lun (12:28:54): > Do you know what needs to be done?
Aaron Lun (12:29:21): > You might even be the lucky soul who hits 1000 commits.
Kevin Rue-Albrecht (12:31:31): > Fede or me? > I have still on my to-do list: > - de-centralization of@importFromstatements. > - misleading variable names > - unit tests > - check doc for typos
Aaron Lun (12:33:21): > And don’t forget the assayColorMap sync, though you can leave this till the end.
Federico Marini (12:34:18): > I’ll tweak the tour and the vignette
Federico Marini (12:34:36): > and commit like a rabid wombat to get to 1k
Federico Marini (12:34:36): > :smile:
Kevin Rue-Albrecht (12:35:19) (in thread): > Ah right. To clarify this one again, what we want is to fill in the undefined assay color maps so that the named ones appear at their true index position, right?
Kevin Rue-Albrecht (12:36:01): > ahahaha it’s going to be like that game where the one taking the last stick on the table loses
Kevin Rue-Albrecht (12:36:29): > holding the commit until the other sends in the 999th ^^
Federico Marini (12:37:13): > As a word of warning for previous shiny developers, i.e.@Kevin Rue-Albrechtand@Charlotte Soneson: I am noticing somewhat strange behaviors of validate/need on reactive values. I can not describe it properly since I did not dig in deep detail. Still, the problem only appears in the devel setting
Kevin Rue-Albrecht (12:38:17): > what do you mean by ‘devel setting’? in R-devel?
Aaron Lun (12:38:41) (in thread): > Yes.
Federico Marini (12:38:56): > yes
Federico Marini (12:39:13): > or maybe some packages that trigger these oddities
Federico Marini (12:39:46): > for me they occur when I have parts of the app that had these nice-error msgs
Federico Marini (12:40:04): > because the reactive objects they should pick up are not there
Kevin Rue-Albrecht (12:42:36): > I think I saw some discussion between@Aaron Lunand@Charlotte Sonesonthis week about adding somevalidate, no?
Aaron Lun (12:42:49): > This was done.
Kevin Rue-Albrecht (12:43:25): > Maybe that’s what Fede is seeing?
Aaron Lun (12:44:08): > Well, it could be anything, so we’d need to know more.
Aaron Lun (12:44:51): > It is possible that some of the validate messages will briefly show upon updates if I missed a double-rendering somewhere.
Kevin Rue-Albrecht (12:45:39): > indeed, the interaction between all the features has created a bit of a monster to handle, so we’ll have to require a certain level of details from users to document and track that kind of behaviour
Federico Marini (12:47:12): > no, here it is fine
Federico Marini (12:47:21): > the msg was for people having other shiny apps
Federico Marini (12:47:37): > that say, “used to work” and might end up not working - at least as expected
Kevin Rue-Albrecht (12:48:08): > eh.. i’m not following anymore
Federico Marini (12:48:53): > normal validate/need checks that used to work - now they show up as
Federico Marini (12:49:53): > “Error in get(as.character(FUN), mode = “function”, envir = envir) : > object ‘as.SimpleList’ of mode ‘function’ was not found”
Federico Marini (12:50:15): > in error-red, which is not so pleasant
Aaron Lun (12:51:47): > SimpleListis generally fucked, seehttps://github.com/Bioconductor/S4Vectors/issues/10. - Attachment (GitHub): Absent elementMetadata upon SimpleList creation · Issue #10 · Bioconductor/S4Vectors
Kevin Rue-Albrecht (12:52:02): > ok.. but you’re talking about ‘other shiny apps’ right? or is that affecting iSEE ?
Federico Marini (12:52:09): > no
Federico Marini (12:52:14): > my other creatures:smile:
Federico Marini (12:52:21): > thanks aaron for pointing it out
Federico Marini (12:52:29): > maybe i’ll just wait the due time
Kevin Rue-Albrecht (12:52:34): > Ahhh, ok, I’m breathing again now. I thought we had another beast to deal with
Federico Marini (12:53:42): > damn BLeeding edge
Kevin Rue-Albrecht (12:54:06): > bound to gets cuts occasionally:confused:
Federico Marini (13:58:15): > oh, we still need the logo - if we want it in the release version
Kevin Rue-Albrecht (14:00:58): > I’m runningcheckand get this note: is this going to be part of thecleanupbranch? > > * checking R code for possible problems ... NOTE > .create_visual_box_for_column_plots: no visible global function > definition for 'checkboxGroupInput' > .create_visual_box_for_row_plots: no visible global function definition > for 'checkboxGroupInput' > .panel_generation: no visible global function definition for > 'checkboxGroupInput' > Undefined global functions or variables: > checkboxGroupInput >
Aaron Lun (14:01:43): > Yeah, just throw in some importFroms, I forget about them.
Kevin Rue-Albrecht (14:22:35): > no worries, I’ll grep them to place the imports here and there then
Kevin Rue-Albrecht (14:22:45): > one step closer to the 1k:stuck_out_tongue:
Kevin Rue-Albrecht (14:23:24): > typing sound coming from Fede’s keyboard
Federico Marini (14:24:33): > ahahah
Kevin Rue-Albrecht (15:57:38): > silly point, but considering that the functions inplotting.R‘write’geom_rectand other ggplot calls, but don’t actually call the function themselves, is it still fine if I add theimportFrom ggplot geom_rectin the roxygen of those functions? > The alternative would be to add theimportFromwhere the commands are evaluated (.create_plot), which kinda defeats the point of decentralising theimportFrom
2018-03-08
Federico Marini (02:47:35): > the dropdown menu of newPanelChoice is not “anchorable” as a introjs tour element?!
Kevin Rue-Albrecht (04:04:01): > Eh.. dunno. > Ive taken a stroll through plotting.R and ExperimentColorMap last night to start cleaning up. I hope there aren’t much changes planned in there today: I’m planning to calmly commit and push things in steps on Saturday.
Aaron Lun (04:27:47) (in thread): > Yes, do put the importFroms there.
Aaron Lun (04:28:14) (in thread): > ¯*(ツ)*/¯
Aaron Lun (04:28:58) (in thread): > Just anchor to the button instead, I suppose.
Kevin Rue-Albrecht (04:33:54) (in thread): > Cool. I’l just have to homogeneise the location of theimportFrom: I realised only later than many of the functions were already covered, with theimportFromas the last item in the roxygen doc
Aaron Lun (04:45:28) (in thread): > Not sure what you mean, but I’ll just say that it’s fine to duplicate @importFrom statements.
Aaron Lun (04:45:39) (in thread): > Taken from the horse’s (Hadley’s) mouth.
Kevin Rue-Albrecht (04:47:26) (in thread): > I just meant that theimportFromappear as the last instructions in the blocks of roxygen documentation above the various functions. > I only realised that was the case after I started documenting other functions with theimportFrominstructions above theauthorinstruction
Kevin Rue-Albrecht (04:47:50) (in thread): > not a big deal, really, but I’ll take the 5 min to homogeneise that inplotting.R
Kevin Rue-Albrecht (04:48:26) (in thread): > It has absolutely no impact on the package itself. It’s just for our own sake
Kevin Rue-Albrecht (04:49:07) (in thread): > As an example, a consequence was that I started addingimportFromtags to functions that already had it, but just not where I was looking for it
Kevin Rue-Albrecht (04:49:52) (in thread): > (those roxygen blocks got pretty big, and knowing where to look for things will likely save significant time in the grand scheme of things)
Aaron Lun (09:04:59) (in thread): > Yeah okay
Federico Marini (15:01:04): > shall we remove from this line > r_check_args: “–no-build-vignettes –no-manual –timings” > in travis.yml the part with > no-build-vignettes?
Federico Marini (15:01:18): > this way we can keep it checked on the CI
Federico Marini (15:01:59): > knitting it locally returned > > Warning messages: > 1: In .local(object, ...) : using library sizes as size factors > 2: In .local(object, ...) : using library sizes as size factors > 3: In .override_defaults(tmp, cur_args) : > unknown field 'YAxis' in user-specified settings > 4: In .override_defaults(tmp, cur_args) : > unknown field 'YAxisGeneText' in user-specified settings >
Kevin Rue-Albrecht (15:18:40): > is there a link between thecheckoptions and the warnings? I wouldn’t think so
Kevin Rue-Albrecht (15:19:10): > it’s probably me who put thosecheckoptions: i always do it to match the options of the BioC nightly build, i might be right or wrong to do so
Kevin Rue-Albrecht (15:19:34): > if I remember correctly, it’s something to do about avoiding to build the vignette twice
Aaron Lun (15:20:37): > The first two warnings can be ignored. The second two warnings are funny and should be checked out.
Aaron Lun (15:20:52): > There are no longer any YAxis or YAxisGeneText fields.
Kevin Rue-Albrecht (15:21:32): > wild guess: it might be something in themodeGatingif I hard-coded anything
Aaron Lun (15:22:38): > Probs
Aaron Lun (15:22:52): > OKay, I’m going home. Absolutely pooped from MS thatdid not write itself.
Federico Marini (16:48:28): > noting it down here:
Federico Marini (16:49:00): > we do not have yet a section describing theExperimentColorMapclass
Federico Marini (16:49:22): > I was thinking of putting it at the end of the description, before the use cases
Kevin Rue-Albrecht (16:49:44): > you mean in the vignette? I thought I saw a section about ECM in there
Federico Marini (16:50:07): > probably@Kevin Rue-Albrechtis the most appropriate candidate for coming up with a small para?
Federico Marini (16:50:08): > yes
Federico Marini (16:50:37): > but it is “only in use”. do we want a couple of lines in the text-text part of the app description?
Kevin Rue-Albrecht (16:50:58): > ahh.. indeed
Kevin Rue-Albrecht (16:51:09): > i’ll pile that on the to-do list for this weekend
Kevin Rue-Albrecht (16:51:35): > hopefully a good burst of productivity should solve that
Federico Marini (16:52:00): > :slightly_smiling_face:
Federico Marini (16:52:30): > I’ll go through the use cases and then call it a day
Federico Marini (16:55:23): > uh. we still need (if we want) an abstract for the vignette - or we leave the simple intro
Federico Marini (16:55:29): > and the logo, the logo
Kevin Rue-Albrecht (16:55:37): > loco logo
2018-03-09
Aaron Lun (05:20:32): > No need for an abstract, I think.
Aaron Lun (05:38:40): > Did anyone try therunAppthing? This should immediately give us >50% coverage.
Kevin Rue-Albrecht (05:40:40): > I meant to, but started at the other end of my to-do list
Aaron Lun (05:40:48): > okay, cool.
Aaron Lun (05:41:01): > Which reminds me, I’m meant to submit another paper today.
Aaron Lun (05:41:12): > But co-authors didn’t respond so will give them another 5 days.
Federico Marini (10:57:39) (in thread): > I got it working
Federico Marini (10:57:58) (in thread): > some DOM black magic:stuck_out_tongue:
Federico Marini (11:46:35): > introjs tour is finished and “fixed” for better anchoring also at the dropdown elements
Federico Marini (11:47:04): > it is not flawless as we like it, but it is a big improvement wrt to the old version
Federico Marini (11:55:02): > talking about symbolism and us getting close to commit 1k
Federico Marini (11:55:03): > https://travis-ci.org/csoneson/iSEE/builds/348709423
Federico Marini (11:55:10): > this was a pretty good fit
Kevin Rue-Albrecht (13:18:31): > when isdocufixexpected to be merged?
Federico Marini (13:34:07): > anytime, actually
Federico Marini (13:34:17): > you can take a look if i missed something
Federico Marini (13:34:27): > i wanted to add the small ecm para
Federico Marini (13:34:35): > so i’d say end of weekend?
Kevin Rue-Albrecht (13:35:08): > i’m literally diving into the ECM now
Kevin Rue-Albrecht (13:35:32): > But you shouldn’t wait on me: I’ll add the paragraph as part of larger updates to the ECM
Federico Marini (13:35:36): > ok
Federico Marini (13:35:43): > perfect
Kevin Rue-Albrecht (13:36:57): > Most importantly asynchronizeAssaysthat Aaron brought up a few times, and that I just couldn’t do, even though I’ve had the implementation plan in my mind for a while
Kevin Rue-Albrecht (13:37:47): > but also, I’ve been wanting to throw in some more accessors to make it easier to extract/replace pieces of the ECM without the need to create/overwrite a new one
Kevin Rue-Albrecht (13:38:45): > I’ve literally addedassaysandassayNamesseconds ago
Kevin Rue-Albrecht (13:38:49): > (on the ECM)
Aaron Lun (14:52:02): > Talk to me, people.
Aaron Lun (15:11:08): > … anyone?
Federico Marini (15:12:13): > here:slightly_smiling_face:
Aaron Lun (15:12:51): > Ah good.
Federico Marini (15:15:45): > I’m noting down the minor things that are still open - do we need sometime next week for a short skype check?
Aaron Lun (15:16:24): > Sounds like a good idea.
Federico Marini (15:16:49): > side note, very very side: the creator ofrintrojsis the owner of this:
Federico Marini (15:16:49): > https://www.cannadata.com/#preview - Attachment (cannadata.com): CannaData > The best software to grow your Cannabis business
Federico Marini (15:17:19): > looking for solutions on the tour i browsed a lil more of what he is in
Aaron Lun (15:17:40): > lol
Federico Marini (15:19:14): > funny, I always thought his package can elevate shiny apps. but not on this way of getting high:smile:
Federico Marini (15:25:14): > anyway, a sneek peek into my notes: > - logo > - final checks > - coverage upping > - authorship & maintainer(s) > - manuscript plans
Federico Marini (15:26:22): > of the open issues, hovering will be another time/pkg
Federico Marini (15:26:47): > remote running I did not try that yet
Federico Marini (15:27:56): > docu notes, we are already up to it soon
Aaron Lun (16:15:13): > looks sensible. Coordinate with@Kevin Rue-Albrechtabout the docs.
Kevin Rue-Albrecht (16:15:51): > I just got home and was just cleaning up the outdated local branch (which for some reason wouldn’t get deleted by--prune)
Kevin Rue-Albrecht (16:16:46): > I’m going to branch fresh fromdocufixand cherry pick my local changes on top of that
Kevin Rue-Albrecht (16:17:12): > @Federico Marinican i let you do the PR and merge ofdocufixwhen you’re ready?
Federico Marini (16:22:44): > sure
Kevin Rue-Albrecht (16:30:39): > drumrolls.. I’ve written the test that coversiSEE(), and i’m runningcovr::package_coverage()now
Kevin Rue-Albrecht (16:37:38): > hang on, did i miss something of isiSEEalready tested from somewhere? (I can’t find a call to it in current unit tests)
Kevin Rue-Albrecht (16:38:14): > because the first few lines and theuiare already covered, even before I add my test
Kevin Rue-Albrecht (16:38:55): > Anyway, I’ll test that now - Attachment: Attachment > Setting runApp(test.mode=TRUE, launch.browser=FALSE) may allow us to immediately cover the entirety of iSEE.R.
Kevin Rue-Albrecht (16:40:39): > :expressionless:nevermind, it was me who wrote a test formodeGatingthat itself calls iSEE
Aaron Lun (17:22:33): > I didn’t realize that the UI gets covered, .panel_generation shouldn’t execute until the server function runs. Perhaps there’s already atest-ui.R?
Aaron Lun (17:23:01): > And the server function shouldn’t run until we actually go throughrunApp.
Kevin Rue-Albrecht (18:15:10): > well, at the end ofiSEE, it is possible thatshinyApp(ui = iSEE_ui, server = iSEE_server)somehow executesiSEE_ui, but definitely not theiSEE_server
Kevin Rue-Albrecht (18:16:17): > > > shiny:::uiHttpHandler > function (ui, uiPattern = "^/$") > { > force(ui) >
Kevin Rue-Albrecht (18:16:22): > there goes
Kevin Rue-Albrecht (18:18:30): > I had a brief look atshinytest, but it seems only compatible with ‘folder-based’ apps: > > > recordTest(app) > Error in recordTest(app) : > Recording tests for shiny.appobj objects is not supported. >
Kevin Rue-Albrecht (18:21:07): > i’ll leave that one out for now, and focus on cleaning up the things i can more immediately take care of
2018-03-10
Kevin Rue-Albrecht (04:09:08): > quick ‘Ask the Audience’ moment: i’m thinking of substituting theassayColorMap(x, i, ...)and friends for the existingassay(x, i, ...)which were the inspiration for them in the first place.
Kevin Rue-Albrecht (04:10:51): > Ah no wait. that only makes sense forassay(x, i, ...). > the current genericscolData(x)androwData(x)don’t takei
Kevin Rue-Albrecht (04:11:53): > in that case, I’ll just set a method forassay(ecm, i, ...)and make the currentassayColorMapa synonym for it
Aaron Lun (07:19:56): > If you want to mimic the generics, just havecolData(x)return a list that can be subsetted.
Aaron Lun (07:20:07): > Or allowecm$blahto mimiccolData(x)[["blah"]].
Kevin Rue-Albrecht (07:20:31): > i’m close to committing the first bit
Kevin Rue-Albrecht (07:21:32): > I actually never went to implement behaviour for$. Is that simplysetMethod('$', ...)?
Kevin Rue-Albrecht (07:23:58): > I can dig up an example from some Bioc-central package, e.g. how they implement$for SummarizedExperiment objects
Kevin Rue-Albrecht (07:57:57): > I’m a bit puzzled how far I should go into ‘synchronising’ ECM and SCE
Kevin Rue-Albrecht (07:58:48): > everyone happy if i throw an error when people supply a mix of named and unnamed assays? it’s just not possible to deal with
Aaron Lun (08:02:20): > This is the aftermath of the fight for SE assays regarding whether names should be enforced.
Aaron Lun (08:02:23): > That was lost, so here we are.
Kevin Rue-Albrecht (08:02:41): > hm.. i think i’ll throw an error only if ECM assays are not named
Kevin Rue-Albrecht (08:03:13): > i meanstopifnot(all(names(ecm@assays))!="")
Aaron Lun (08:03:25): > Seems sensible.
Kevin Rue-Albrecht (08:03:49): > because this way, I can set all the unnamed SCE assays to the default colormap
Kevin Rue-Albrecht (08:04:17): > which -correct me if I’m wrong- would be the user-level behaviour anyway
Aaron Lun (08:04:32): > Probably.
Kevin Rue-Albrecht (08:05:15): > i mean theonlyscenario where one could possibly rely on positional layout to pair an ECM and an SCE would be where none of the assays are named in either side
Kevin Rue-Albrecht (08:06:06): > in that case, the only check would be thatlength(assayNames(..))matches, which means color maps would be mapped 1->1, 2->2, …
Kevin Rue-Albrecht (08:06:23): > introduce names anywhere and we’re doomed
Kevin Rue-Albrecht (08:06:41): > (or at least this approach is anyway)
Aaron Lun (08:12:13): > The only safe programmatic approach for assays is to rely on positional indices only.
Aaron Lun (08:12:42): > This is why we are using positional indices insideiSEE.
Aaron Lun (08:13:19): > You could also enforce this by requiring integer assignments toassayColorMap.
Aaron Lun (08:13:40): > And ignoring all names. Or if names are to be assigned, a corresponding SE object must also be supplied.
Aaron Lun (08:17:13): > And by “safe”, I mean returns sensible results without any error.
Aaron Lun (08:17:31): > Most incorrect uses ofassay(se, "")trigger an error, which is usually fine as people can fix it immediately.
Aaron Lun (08:17:42): > For us, we would just crash out straight away, which is BAD(TM).
Kevin Rue-Albrecht (08:18:56): > at the moment, indeed, the ECM simply returns a default color map for incorrect uses ofassayColorMap(x,i)
Kevin Rue-Albrecht (08:19:22): > which, to be fair, was the desired behaviour so far
Kevin Rue-Albrecht (08:19:50): > (return a default color map if the user didn’t specify one)
Kevin Rue-Albrecht (08:20:50): > then, I guess priority 1 is now forsynchroniseAssays-in the presence of named assays- to reorder those in theECMto match those in theSCE, so that numeric indexing is the same in both objects
Aaron Lun (08:21:36): > The real risk is that we forget to sync the colormap to the SE; insideiSEE, we use positional indexing for assays, resulting in the incorrect colormap being used. This becomes particularly problematic for code tracking if people do not copy-paste the entire script, as we would need to sync the ECM at the top.
Kevin Rue-Albrecht (08:21:49): > ok
Aaron Lun (08:21:52): > There’s a lot of other variables in the code tracker but their absence causes a direct failure.
Aaron Lun (08:21:57): > So I’m less concerned about them.
Aaron Lun (08:22:56): > It may be possible to drag the names along by modifyingparam_choicesinplotting.R.
Kevin Rue-Albrecht (08:22:58): > ok, but if I do thesynchroniseAssaysas described above, then I can amend thenumericsignature ofassayColorMapto throw an error if the index is out of bounds
Kevin Rue-Albrecht (08:23:21): > (the character too, actually)
Aaron Lun (08:23:25): > Well, you’d never know if it never got synced.
Kevin Rue-Albrecht (08:24:25): > no, but 1) in the app we know we do it ourselves, 2) it wouldn’t cost much time to users to run it again
Aaron Lun (08:24:52): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (08:25:20): > obviously, the function should preserve everything if the two input objects are already synchronised
Aaron Lun (08:33:12): > Another option would be forcibly enslave the ECM to the SCE.
Aaron Lun (08:33:30): > i.e., the ECM is stored inside the SCE, and one can only interact with the ECM via the SCE.
Kevin Rue-Albrecht (08:34:30): > is it possible to override the SCE validation method, without having to define a child class?
Aaron Lun (08:34:44): > Not sure.
Kevin Rue-Albrecht (08:35:23): > My guess would be not possible. But ideally that’s what would ensure that the object is synced at all time
Kevin Rue-Albrecht (08:36:29): > when I was still thinking about the ECM, the idea of aColoredExperimentcrossed my mind. But at the time I thought that was overkill to just add some colors:confused:
Kevin Rue-Albrecht (08:38:01): > btw, with the latest push, we can now avoid using@to access the various slots of anECM. I’ve got most accessors in place
Kevin Rue-Albrecht (08:39:14): > although as we’re discussing, the behaviour for missing values or invalid indexing needs to be kept in check. Feel free to add unit tests to declare scenarios that should/n’t happen
Kevin Rue-Albrecht (08:40:22): > in the meantime, I’m writing the first version ofsynchronizeAssays
Kevin Rue-Albrecht (10:02:57): > Alright. Latest commit synchronises ECM and SCE where all the assays are named. I’m moving on to the other scenarios as follows: > - inall cases: drop from the ECM: unnamed assays and named assays absent from the SCE > - ifnone of the SCE assays are named: if the ECM has the same number of assays, return it (after dropping the assay names?). Otherwise, throw an error. > - if only a subset of the SCE assays are named: if all the ECM assays are named, fetch return the matched assay color map, and fill the blanks with defaults. Otherwise, throw an error.
Kevin Rue-Albrecht (10:04:24): > Basically, that will only support unnamed/partially named assays in ECM in one case: > if (all the assays in the SCE are unnamed) & (number of assays is equal in ECM and SCE)
Kevin Rue-Albrecht (10:05:13): > The battle of assay names may have been lost forSummarizedExperiment, but this can be our way to politely enforce it for ECM
Aaron Lun (10:05:52): > I don’t care as long asassayColorMap(ecm, some_number)is valid.
Aaron Lun (10:06:05): > That is necessary to finish the separation between church and state
Aaron Lun (10:06:12): > I mean, between getting the data and plotting the data.
Kevin Rue-Albrecht (10:08:20): > I hear your gospel
Aaron Lun (10:12:50): > Make sure you add a synchronize command to the code tracker as well, then.
Kevin Rue-Albrecht (10:15:45): > Right. Been a while since I looked in there. I’ll give you a shout if I have a doubt
Kevin Rue-Albrecht (10:54:17): > I’m just stepping away from the computer for a bit, but I’m halfway through the rest of the synchronisation. I’ll resume it in the next couple of hours
Kevin Rue-Albrecht (11:00:39): > To be fair, I’m expecting two main scenarios for the assay color maps: 1) users who change the global default for all assay, 2) those who provide a few named override for specific assays. > And from the moment all ECM assays are names, synchronisation is easy peasy
Aaron Lun (11:29:51): > @Federico MariniAre you done with your docfixes? If so, merge them in.
Federico Marini (11:30:19): > yep
Federico Marini (11:31:10): > I’ll wait for your cleanups first?
Aaron Lun (11:31:19): > Yeah
Aaron Lun (11:56:26): > Lol,kevin_docson 999.
Kevin Rue-Albrecht (12:18:51): > Ahhhhhhhruns to the keyboard
Kevin Rue-Albrecht (12:20:23): > No seriously, just bringing home some shopping and I’ll carry on. > In the meantime, if anyone feels like it, I’ve reintroduced the « nocov start/end » exclusion around the app server function, for now. One just needs to remove them both to let code coverage include them in the denominator
Kevin Rue-Albrecht (12:22:42): > From what I could see last night, shinytest only supports apps that are implemented as a folder, not shinyapps objects directly
Federico Marini (13:11:54): > plus, it is not on cran anyway
Kevin Rue-Albrecht (13:29:40): > god this is painful, but I’m almost there
Kevin Rue-Albrecht (13:38:25): > (unit testing: the necessary pain:sob:)
Kevin Rue-Albrecht (13:40:01): > weird discovery: if an ECM is created with only unnamed colormaps, none of them make it through the constructor (no colormap in @assays slot)
Kevin Rue-Albrecht (13:40:25): > if even one of them is named, they all make it to the object
Kevin Rue-Albrecht (13:40:29): > :upside_down_face:
Kevin Rue-Albrecht (13:42:33): > Solved. Seems to be a corner case ofnew()for unnamed arguments
Federico Marini (14:04:07): > kev, you probably don’t need an excuse for an extra commit. but hey, we can bump up the version:musical_note:
Kevin Rue-Albrecht (14:04:14): > hahaha
Kevin Rue-Albrecht (14:05:27): > I was trying to make the commit count: as far as I can see I’ve finished implementingsynchronizeAssays, now i’m just looking where in the code tracker and.sanitize_SE_inputI need to call it for the app
Kevin Rue-Albrecht (14:06:31): > but, yeah: when i see the feature work, I’ll bump up the version. Unfortunately, it’s not going to be commit #1000
Kevin Rue-Albrecht (14:06:39): > maybe 1001, who knows ^^
Federico Marini (14:09:20): > btw. today’s rdevel is indeed r devel
Kevin Rue-Albrecht (14:09:28): > lol
Kevin Rue-Albrecht (14:10:53): > before i unleash Aaron’s wrath: I just noticed how stupid it was to suggesting calling “synchronizeAssaysin.sanitize_SE_input” … which is to sanitizeSEinput
Kevin Rue-Albrecht (14:32:43): > question to the audience: do we still want to throw an error if!isColorMapCompatible, now that we cansynchronizeAssays?
Kevin Rue-Albrecht (14:33:19): > becausesynchronizeAssaysbasically drops ‘incompatible’ things (e.g., colormaps that do not have a matching assay in the SCE)
Kevin Rue-Albrecht (14:33:59): > mah.. nevermind, I’m answering ‘yes’ to myself
Kevin Rue-Albrecht (14:34:24): > there are still many cases when ‘compatible’ ECM are not ’synchronized`
Kevin Rue-Albrecht (14:35:23): > so we’ll throw an error if there are some obvious issues, and we’ll ask users to apply the exportedsynchronizeAssaysfunction before callingiSEE
Kevin Rue-Albrecht (14:36:11): > for the minor issues (compatible assays, but in different orders), we’ll just invisibly synchronize them at the start of the app
Kevin Rue-Albrecht (14:48:46): > bumping
Kevin Rue-Albrecht (14:49:36): > the app runs,isColorMapCompatiblethrows informative (I think) errors , andsynchronizeAssaystakes care of the rest
Kevin Rue-Albrecht (14:50:13): > I’m just running one lastcheckto pick up possible leftovers, but otherwise I’ll push in 5 min
Kevin Rue-Albrecht (14:51:06): > Then I’ll appreciate if you can test it out and send feedback, while I write a paragraph in the vignette
Kevin Rue-Albrecht (14:59:16): > pushed.
Kevin Rue-Albrecht (15:06:11): > @Aaron Lunissue 143: “Consistently use id and id0 instead of i and i0, respectively, throughout iSEE.R, to avoid confusion between the panel ID and more general indexing.” > Grepping the code, it seems that it isconsistentlydoing the opposite of the desired behaviour described. > What matters most in your point: the consistency, or the names?
Kevin Rue-Albrecht (15:25:38): > Please no one touch the vignette, I’m going through it from top to bottom, to get a feel for the writing style, add my two cents here and there, and then add my paragraph where suitable
Aaron Lun (15:39:23) (in thread): > The names.iis used elsewhere (e.g., inbrushing.Rortables.Rto actively refer to an index, which is then used to indexIDs.
Kevin Rue-Albrecht (15:52:50) (in thread): > ok. A separate question was about the ‘brushing’ -> ‘point selection’. > I was wondering whether this statement in.spawn_brush_chartcould be removed altogether, if we consistently refer to ’point selection instead of brushing: > “It is also worth mentioning here that the concept of a “brush” includes selection via both the standard Shiny brush as well as our custom lasso method. Unless we explicitly refer to a Shiny brush, any mentions of brushing are assumed to refer to both lasso and Shiny brushing.”
Federico Marini (17:53:30): > 1k!
Kevin Rue-Albrecht (18:10:38): > :slightly_smiling_face:I’m keen to hear from Aaron if it’s behaving as he expects.
Kevin Rue-Albrecht (20:09:35): > I’m 80% done with the ECM paragraph of the vignette. I’ll just need another hour on the vignette tomorrow morning.
2018-03-11
Federico Marini (04:10:11): > nice touch with the icons - may I suggest we keep it called with an inline block?
Federico Marini (04:11:15): > the result will be the same but it looks somewhat neater on the source side:slightly_smiling_face:
Federico Marini (04:13:17): > we can do it even more extreme and add the whole action button with the style too
Federico Marini (04:13:29): > actionButton(...)
Federico Marini (04:14:02): > probably this makes it even easier to re-localize the functionality?
Federico Marini (04:20:13): > screenshot coming
Federico Marini (04:23:23): > ok probably not - we might need runtime: shiny or similar
Federico Marini (04:25:23): > @Federico Mariniuploaded a file:Screen Shot 2018-03-11 at 09.24.55.png - File (PNG): Screen Shot 2018-03-11 at 09.24.55.png
Kevin Rue-Albrecht (04:34:12): > Hi Fede. It’s up to you, really. I just found the feature here, originally:https://github.com/rstudio/rmarkdown/issues/813 - Attachment (GitHub): Feature Request: Font-Awesome Icons in Rmarkdown Docs · Issue #813 · rstudio/rmarkdown > I’ve noticed that it’s currently quite complicated to insert font-awesome icons in Rmarkdown documents. Could this be made easier somehow? It’s so straight-forward in shiny… Thanks!
Kevin Rue-Albrecht (04:36:44): > I’ll finish the my paragraph now, and also finish review the last part of the vignette. > I’m not planning to touch the first part of the vignette where the icons are, so you should be able to safely tweak things without causing a conflict
Federico Marini (04:40:34): > probably it is way much an overkill the way i proposed
Federico Marini (04:40:48): > your version looks compact & neat - and this is what we should aim for
Kevin Rue-Albrecht (04:42:20): > the button kinda looks nice, to be honest, I was wondering if it was possible to make it smaller (to match the text size better), but yes, it probably is a bit overkill
Federico Marini (04:43:24): > probably - but we cannot escape the runtime: shiny thingy
Federico Marini (04:43:36): > so no real html as the vignette usually is+
Kevin Rue-Albrecht (04:44:08): > i’m not sure I understand what you mean with this runtime point
Federico Marini (04:44:37): > this part should go in the yaml
Federico Marini (04:44:45): > runtime: shiny
Federico Marini (04:45:06): > alternative, which is maybe viable:
Federico Marini (04:45:22): > we could copy the html output of the actionButton call
Federico Marini (04:45:30): > like
Federico Marini (04:45:31): > <button id=“this” type=“button” class=“btn btn-default action-button”>thsas
Federico Marini (04:45:57): > we can hear the others out
Kevin Rue-Albrecht (04:46:17): > ok.. i didn’t know thisruntimething in the YAML. What does it do?
Federico Marini (04:46:27): > I’m fine with either solution - but yes, the visualized icon makes it easy
Federico Marini (04:46:46): > it is to generate shiny html documents ->https://rmarkdown.rstudio.com/authoring_shiny.html - Attachment (rmarkdown.rstudio.com): Interactive Documents > Turn your analyses into high quality documents, reports, presentations and dashboards with R Markdown. Use a productive notebook interface to weave together narrative text and code to produce elegantly formatted output. Use multiple languages including R, Python, and SQL. R Markdown supports a reproducible workflow for dozens of static and dynamic output formats including HTML, PDF, MS Word, Beamer, HTML5 slides, Tufte-style handouts, books, dashboards, shiny applications, scientific articles, websites, and more.
Kevin Rue-Albrecht (04:46:56): > ahhhhh
Kevin Rue-Albrecht (04:47:47): > right, so you would have imagined an actual, clickable, button in the vignette?
Kevin Rue-Albrecht (04:47:58): > that would be overkill, I think
Federico Marini (04:48:53): > -> > > <button id="open_vignette" type="button" class="btn btn-default action-button" style="color: #ffffff; background-color: #0092AC; border-color: #2e6da4"> > <i class="fa fa-book"></i> > Open the vignette > </button> > > , > for one button
Kevin Rue-Albrecht (04:48:53): > if it was only the appearance of the button (i.e., a static image) of the button, fine, but making the whole document interactive is really not necessary in this case.
Federico Marini (04:50:30): > with the html code it works without making it interactive
Federico Marini (04:51:08): > it would just look the same as in the previous screenshot
Kevin Rue-Albrecht (04:52:08): > Ok. Well, to be honest, I’m happy to learn by seeing you doing. If you’re in doubt, you can always fork a branch and tell us where to look for giving you feedback
Federico Marini (04:52:23): > will do later today
Kevin Rue-Albrecht (04:52:27): > cool
Federico Marini (04:52:34): > i’ll fork now and show it up
Kevin Rue-Albrecht (04:52:58): > damn i’m having a hard time keeping the ECM paragraph concise and on point
Kevin Rue-Albrecht (04:53:20): > I’m at the “key advantage” section
Kevin Rue-Albrecht (04:54:51): > i’ve got a few: > - single place to define all colormaps > - simple access interfacecolDataColorMap(colormap, "coldata_name")- transferrable between sessions and projects > - fallback mechanism to default colormaps for undefined colormaps
Federico Marini (05:49:53): > I’d say they’re fine. You had it in your mind ahead of this, so you might have some more thoughts swirling around
Federico Marini (05:50:04): > but in the current version these points will do
Kevin Rue-Albrecht (05:57:46): > thanks. > i’m in section “## Use case III: Exploring mass cytometry data”
Kevin Rue-Albrecht (05:57:50): > i’ll push back soon
Kevin Rue-Albrecht (06:14:35): > Alright. pushed. bit of an update that also include consistent reference to “point selection” rather than “brushing”
Kevin Rue-Albrecht (06:32:21): > i’m pushing a moderated use of 4class{} that is not wrapped in in most cases
Kevin Rue-Albrecht (06:34:54): > Notable exception foriSEE-main.Rfor references to annotationfunctions
Kevin Rue-Albrecht (06:58:40): > warning: major layout cleanup of the doc inplotting.R, no one gets near it for the next hour please
Aaron Lun (07:53:42): > Looking at this now.
Aaron Lun (07:54:13): > Doesn’t Rstudio have line wrap? Why does everything get crammed into the left 50% of the screen?
Aaron Lun (07:54:44): > Looks pretty stupid if you ask me.
Kevin Rue-Albrecht (07:55:32): > well, RStudio has, but it can get a bit painful when opened in a text editor. I do see your point that it does all get unnecessarily compressed to the left
Kevin Rue-Albrecht (07:56:10): > I don’t know what bioc reviewer will say, to be honest
Aaron Lun (07:56:18): > Well, they won’t care
Aaron Lun (07:56:29): > but I find it difficult to read when 1 line != 1 sentence.
Kevin Rue-Albrecht (07:56:43): > all i know is that BiocCheck will flag up a lot of lines >80 and not indentend by 4 space
Aaron Lun (07:56:56): > Yeah, that’s even more stupid.
Aaron Lun (07:57:00): > I suggest ignoring that check.
Kevin Rue-Albrecht (07:57:47): > i tried to leave lines where breaking up the line wasreallymaking it painful to understand the statement
Aaron Lun (07:58:23): > I would expand that limit to 250, personally.
Kevin Rue-Albrecht (07:58:28): > But I don’t mind reverting the others too, if that’s the consensus
Kevin Rue-Albrecht (07:58:40): > yeah. I agree that we’re not in 1980s anymore
Kevin Rue-Albrecht (08:02:06): > well anyway, I’m not touching the doc (or code) anymore today inplotting.R. I’m just writing tests to increase coverage
Aaron Lun (08:05:12): > brushing.Rcode has all been changed to useselect.
Kevin Rue-Albrecht (08:14:36): > @Kevin Rue-Albrechtuploaded a file:joys of unit testing - File (PNG): joys of unit testing
Kevin Rue-Albrecht (08:15:35): > actually, thinking about it, I think this shows that we need to addaes(..., group=interaction(X, Y))somewhere for square plots
Aaron Lun (08:15:42): > Note thatplotting.Rdidn’t finish the job re brush -> select.
Aaron Lun (08:15:49): > We still have.process_brushby_choice.
Kevin Rue-Albrecht (08:16:17): > it seems that at the moment, square plots act more likeinteraction(X, Y, color)
Aaron Lun (08:16:22): > see my new branch brushtoselect.
Kevin Rue-Albrecht (08:16:37): > oh did i miss one? I searched for all occurences of**brush**
Aaron Lun (08:17:29): > Not just in the docs, but also in the code.
Aaron Lun (08:17:47): > I’m going off for lunch now.brushtoselectis halfway done and broken.
Aaron Lun (08:17:55): > Will finish it when I get back.
Kevin Rue-Albrecht (08:18:38): > i did try to deal with all the comments, but I guess it’s possible i missed some occurences
Kevin Rue-Albrecht (08:19:04): > no worrie, as I said, i’m not touching it for now. Unit testing instead for a change
Kevin Rue-Albrecht (08:33:33): > @Kevin Rue-Albrechtuploaded a file:after - File (PNG): after
Federico Marini (08:51:46) (in thread): > that check is quite crazy IMHO
Federico Marini (08:51:56) (in thread): > and I constantly go over that
Federico Marini (08:52:18) (in thread): > 1 sentence 1 line sounds reasonable especially when changing and trackign our changes
Federico Marini (09:02:51): > https://github.com/emitanaka/shinycustomloader- could be a nice funny note
Kevin Rue-Albrecht (09:03:31): > looool, i like it ^^
Federico Marini (09:04:37): > we can have something like the goggles where the plot gradually appears
Kevin Rue-Albrecht (10:58:06): > @Aaron Lunplease seehttps://github.com/csoneson/iSEE/commit/ee6ad96400645e6e8f57ce6276bd732148726064for a fix to rowData plots. I think that relates to something you were talking about recently (passing arguments as...) - Attachment (GitHub): fix color of rowData plots by passing se object · csoneson/iSEE@ee6ad96 > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (10:58:36): > i got errors when unit testing color by feature name in rowData plots
Kevin Rue-Albrecht (10:59:33): > with that fixed, I basically only have the lasso selection left to unit test
Aaron Lun (11:01:32): > That fix should not be necessary in the future assewon’t need to be passed through when the plotting is quarantined from the choice of color.
Kevin Rue-Albrecht (11:03:58): > Right. Then please consider this as a temporary fix
Kevin Rue-Albrecht (11:05:23): > to get the unit test in place
Aaron Lun (11:14:01): > Given that you’re fiddling withtest_plotting.R, I suggest merging inbrushtoselectand fixing the plot test for"select"rather than"brush".
Aaron Lun (11:14:10): > It should be all-clear otherwise.
Kevin Rue-Albrecht (11:14:24): > ah ok
Kevin Rue-Albrecht (11:15:03): > i’m just in the middle of the lasso test. How about I commit, merge, resolve potential conflicts, and fix the “select” plot test?
Aaron Lun (11:15:10): > yes.
Kevin Rue-Albrecht (11:16:12): > remind me, i’m struggling to create a dummy lasso path: it has to be matrix of two column, withattr(lasso_val, "closed") <- TRUE, right?
Kevin Rue-Albrecht (11:16:18): > I’m gettingError in !is.null(lasso_val) && closed : invalid 'y' type in 'x && y'
Aaron Lun (11:16:25): > Yes.
Aaron Lun (11:16:48): > But you can’t set attributes onNULL.
Kevin Rue-Albrecht (11:17:19): > that can’t be the problem, I’ve define a hard coded matrix
Kevin Rue-Albrecht (11:17:26): > > lasso_val <- matrix( > data = c( > min(x_10), min(y_10), > min(x_10), max(y_10), > max(x_10), max(y_10), > max(x_10), min(y_10) > ), > ncol = 2, > byrow = TRUE > ) > attr(lasso_val, "closed") <- TRUE >
Kevin Rue-Albrecht (11:17:36): > (with non-NULL x_10, u_10)
Kevin Rue-Albrecht (11:18:29): > (i’m taking the range of coordinates for the first 10 data points, to define an arbitrary non-empty lasso)
Aaron Lun (11:19:52): > I’ll get back to that in a second. But before that,@Charlotte Soneson; I’ve noticed that we’re usingupdate_geom_defaultsfor setting the default point color. Is there a method to do this that is localized to.create_points? Otherwise the modularity of this section of the code starts to break down.
Kevin Rue-Albrecht (11:20:33): > ah wait, i think it’s that story again that .lassoData is a list of length 1, not the matrix itself
Aaron Lun (11:20:46): > … yes.
Charlotte Soneson (11:25:22): > I don’t know, I guess technically it could be set anywhere, it will not be used unless there is no other coloring specified. I guess we could also specify it explicitly in all situations where it should be used, but I found that to be more cumbersome.
Kevin Rue-Albrecht (11:25:52): > (lasso issue solved)
Federico Marini (11:27:06): > if noone touches the vignette, I’m on it for the button-ing
Federico Marini (11:27:44): > should I also revert it to the 1sentence:1line ignoring the 80char limit?
Aaron Lun (11:27:49) (in thread): > I think we should set it explicitly in every call to.create_points(as this is only where it is used), in the same way that point size and transparency are explicitly set. This has a number of advantages, namely preserving modularity in the codegeneration(for us to understand where the parameter has effects) as well as modularity in codeexamination(so users are not affected by global variables when they try to copy/paste code from the tracker).
Aaron Lun (11:27:58): > Yes, please do.
Aaron Lun (11:28:03): > Fuck the 80 characters, it’s stupid.
Federico Marini (11:29:06): > Ok, I’ll make some tea then:slightly_smiling_face:
Kevin Rue-Albrecht (11:29:32): > i’m just finishing up preparing crepes:ok_hand:
Charlotte Soneson (11:29:55) (in thread): > Ok. I’ll probably not get to fixing this today, but I’ll look at it tomorrow, it should only be to move the command in there.
Aaron Lun (11:32:55) (in thread): > Well, it’s a bit more complicated, unfortunately; by the time you get into.create_points, you’re in the ggplot command chain, so you’ll have to set it as part of the ggplot + something + etc.
Charlotte Soneson (11:33:24) (in thread): > Oh, I see, sorry, I missed that.
Aaron Lun (11:35:05) (in thread): > I mean, having to specify it explicitly as part of eachgeom_pointscommand is a bit of work, but otherwise fine.
Charlotte Soneson (11:36:22) (in thread): > Sure. I found theupdate_geom_defaultsnice since it handled all situations, also e.g. the coloring the “non-brushed” points.
Federico Marini (11:50:20): > kevin kevin:stuck_out_tongue:I found an instance where you should have put the 0km-comma (for you:smile:)
Federico Marini (11:50:45): > I’ll use the new mainzer grammatik:smiley_cat:
Kevin Rue-Albrecht (11:50:53): > ?
Federico Marini (11:51:16): > the oxford comma
Kevin Rue-Albrecht (11:51:27): > ahhh
Federico Marini (12:05:25): > word of warning in general: i found some leftovers of gene expression- related params instead of the current Feature…
Federico Marini (12:05:34): > e.g. in the cytof example
Federico Marini (12:06:00): > and found out the examples contained uneval’d chunks that would throw errors
Federico Marini (12:06:56): > could someone check the cytof example please? -> I have someWarning: Error in update_geom_defaults: could not find function "update_geom_defaults"when openin the app
Federico Marini (12:07:08): > I’ll fix the nomenclature
Federico Marini (12:15:04): > huh, I am also getting aError in cur_memory[, .brushByPlot] : object '.brushByPlot' not found- I branched from kevin_docs
Aaron Lun (12:15:52): > See the comments above.
Federico Marini (12:17:06): > yep:wink:sorry for missing it
Kevin Rue-Albrecht (12:18:16): > alright, i’ve mergedbrushtoselecttokevin_docsand everything looks good except for the unit tests that you mentioned Aaron. I’m getting on that now
Federico Marini (12:39:20): > So, I restored the vignette line-sentence ratio
Kevin Rue-Albrecht (12:39:30): > sorry:innocent:
Federico Marini (12:39:50): > I got the chance to correct a couple of other things along the way
Kevin Rue-Albrecht (12:39:54): > i literally just simultaneously finished updated the unit tests, their names, and even code comments
Federico Marini (12:40:08): > so it was for a good purpose anyway
Kevin Rue-Albrecht (12:42:39): > quick check before I actually commit it: I spotted.brushField <- "Brush"inconstants.R. > A quick grep through the code suggests it’s not used anywhere anymore. Good to remove,@Aaron Lun?
Aaron Lun (12:42:57): > That’s anactualshiny brush.
Kevin Rue-Albrecht (12:43:00): > the only ‘brush’ constant left would be.brushData <- "BrushData"
Aaron Lun (12:43:14): > .brushFieldis necessary for the observer to an actual shiny brush, so don’t delete it.
Aaron Lun (12:43:24): > I’m pretty sure it should be hanging aroundiSEE.R.
Kevin Rue-Albrecht (12:43:26): > ah ok, good that I asked
Kevin Rue-Albrecht (12:43:41): > my grep must have been wrong then, because literally nothing cropped up
Aaron Lun (12:44:13): > grep "brushField" iSEE-main.Rgives me a couple of hits.
Kevin Rue-Albrecht (12:44:15): > ahh yeah i can see it now
Aaron Lun (12:44:25): > same fordynamicUI.R
Kevin Rue-Albrecht (12:44:27): > i grepped the triple:
Kevin Rue-Albrecht (12:44:35): > which didn’t hit
Federico Marini (12:44:39): > I got these: > > Federicos-MacBook-Pro:iSEE fede$ grep -n '.brushField' R/*R > R/constants.R:100:.brushField <- "Brush" > R/dynamicUI.R:158: brush.opts <- brushOpts(.input_FUN(.brushField), resetOnNew=FALSE, > R/iSEE-extras.R:408: brush_id <- paste0(plot_name, "_", .brushField) > R/iSEE-main.R:693: brush_id <- paste0(prefix, .brushField) > R/iSEE-main.R:855: !is.null(input[[paste0(prefix, .brushField)]])) { > R/iSEE-main.R:918: brush_id <- paste0(prefix, .brushField) > R/iSEE-main.R:963: brush_id <- paste0(prefix, .brushField) > > -> line nr might vary
Kevin Rue-Albrecht (12:45:16): > looks all good on my side, waiting forcheckto complete and I’ll push
Federico Marini (12:46:11): > Cool. I’ll follow up. then i’ll try the restyling on my branch further, so you can see it better
Federico Marini (12:46:32): > I’m up for cooking now:slightly_smiling_face:
Kevin Rue-Albrecht (12:52:52): > @Kevin Rue-Albrechtuploaded a file:commit hub - File (PNG): commit hub
Federico Marini (13:37:11): > iskevin_docsto be merged soon?
Kevin Rue-Albrecht (13:37:43): > I was about to, and then realised that my ‘lasso’ test was not covering.self_lasso_path
Kevin Rue-Albrecht (13:38:22): > I’m litterally checking coverage now that I’ve just finished covering all three cases: single-point, open path, and closed lasso
Kevin Rue-Albrecht (13:38:36): > yep, we’re there: 98.46%
Kevin Rue-Albrecht (13:39:12): > just pushed, and when it’s green, i merge
Kevin Rue-Albrecht (13:39:29): > i’ll open a new branch later, for any additional fix
Kevin Rue-Albrecht (14:02:28): > merged
Aaron Lun (14:02:50): > Yeah, >80%. GW
Aaron Lun (14:07:33): > A few dead branches to prune.
Kevin Rue-Albrecht (14:09:05): > yup
Kevin Rue-Albrecht (14:10:11): > you’re not doing anything else with thebrushtoselect?
Aaron Lun (14:10:58): > nope.
Kevin Rue-Albrecht (14:11:00): > (i would deleted only my own, but you said ‘a few’, so I might as well remove the ones that I merged tokevin_docs)
Kevin Rue-Albrecht (14:12:19): > I guess I’ll get rid of theprogressbaras well? did we give up altogether on the pop up showing progress through computation steps?
Aaron Lun (14:14:05): > I think so.
Aaron Lun (14:14:15): > Wasn’t worth the effort.
Aaron Lun (14:14:30): > It all just flashes by for me.
Kevin Rue-Albrecht (14:15:17): > Agreed. Plus, Fede’s earlier link could make the initial start-up friendlier
Kevin Rue-Albrecht (14:16:01): > (https://github.com/emitanaka/shinycustomloader)
Federico Marini (14:49:40): > still it is a very early dev package
Federico Marini (14:49:55): > we canborrowthe idea
Kevin Rue-Albrecht (14:50:25): > we can think about it post Bioc submission, as recreational dev
Kevin Rue-Albrecht (14:51:06): > i also noticed that it seems design for individual plots; not sure whether it’s possible on the entire interface
Kevin Rue-Albrecht (14:51:14): > anyway - fun to think about
Federico Marini (15:04:49): > for some general loader thing, we could use shinyjs
Federico Marini (15:25:01): > FYI,iconizednow shows how I had in mind the buttons
Federico Marini (15:25:40): > just pull it, knit the vignette, and tell me if it is worth to update all button-icons to “real fake buttons”
Kevin Rue-Albrecht (15:33:46): > pulling now
Kevin Rue-Albrecht (15:34:21): > i’ll try to have a sneak peak before dinner. Don’t wanna run into trouble:innocent:
Aaron Lun (15:35:21): > God, if I see one more single-cell imputation paper, I will Flip Out.
Kevin Rue-Albrecht (15:36:02): > is there a pun i’m not catching in the special capitalisation of flip out?
Aaron Lun (15:36:17): > no, I’m just very perturbed that people think imputation is a good idea.
Kevin Rue-Albrecht (15:38:24): > aw.. i thought i’d learn a pun. Anyway, the whole ‘more cells, fewer reads’ had to attract imputation friends
Aaron Lun (15:39:33): > The majority of approaches are circular. Identify subpopulations, do imputation, recluster - well, obviously your subpopulations are going to look better, because your imputation was done within each cluster.
Kevin Rue-Albrecht (15:40:33): > btw,@Federico Marini, i like the buttons, but 1) can they be made a bit smaller in the vignette?, 2) probably best to only show as buttons those that actually look like buttons in the app (i.e., not those that are only used as icons)
Kevin Rue-Albrecht (15:41:14): > although now that I check, I think that is what you did
Kevin Rue-Albrecht (15:43:18) (in thread): > well, papers looking for new subtypes and substates are probably fall into that category as well, considering that the more information we collect, the more people will claim higher resolution of things that are probably not that different (i.e. natural fluctuations in the transcriptome)
Federico Marini (15:50:22) (in thread): > yep:slightly_smiling_face:
Federico Marini (15:51:21) (in thread): > for the size, I’d need to check
Kevin Rue-Albrecht (15:51:27) (in thread): > well then. I really like them, in this way. I was just curious to know if it was possible to scale them e.g. 90%
Federico Marini (16:27:05) (in thread): > they’re “just” html, so I guess so. not my bread&butter, but I’ll check
Federico Marini (16:43:36): > … and the search returned indeed it is just html
Federico Marini (16:44:16): > font-size:80%; padding:4px could do the job, after the colors
Federico Marini (16:47:17): > @Federico Mariniuploaded a file:Screen Shot 2018-03-11 at 21.46.49.pngand commented: 80% w padding, vs default - File (PNG): Screen Shot 2018-03-11 at 21.46.49.png
Kevin Rue-Albrecht (18:05:26): > ok, that looks cool
Kevin Rue-Albrecht (18:05:43): > what’s up with the newline after each button ?
Aaron Lun (18:49:44) (in thread): > I’m less concerned about that, as it generates testable hypotheses that can be shot down (or not). But imputation in the manner described adds nothing to the initial structuring.
Kevin Rue-Albrecht (18:51:50) (in thread): > I haven’t done much imputing myself (knowingly), so I can’t really comment, but I see your point
2018-03-12
Federico Marini (03:49:14): > could not solve it yet
Aaron Lun (05:08:41): > Talk to me, people.
Aaron Lun (05:08:42): > Can we submit this week?
Kevin Rue-Albrecht (05:09:18): > I was happy where I left things yesterday
Kevin Rue-Albrecht (05:10:01): > how’s issue 143 looking? > Minor thing: I haven’t substitutedi*->id*(yet)
Aaron Lun (05:10:58): > Yeah, that’s the remaining thing.
Kevin Rue-Albrecht (05:11:43): > have you had time to play withsynchronizeAssay()yet, to tell me if it behaves exactly as you expected?
Kevin Rue-Albrecht (05:13:51): > I think it’s handling what will make 99% of the scenarios. There may be some corner cases that I haven’t thought about, but I’ve put some fairly strict failsafes in place to throw an error in the one ambiguous scenario (when both SCE and ECM are fully unnamed, and have different number of elements in their assay slot)
Aaron Lun (05:19:07): > I’ll do so later, got to deal with somebeachmatstuff first.
Charlotte Soneson (05:36:12): > I’m looking into the default coloring. Something seems wrong with the brushing though (in currentmaster): all panels have “0 of 0 points selected” to start with, and if I try to select points it crashes (seems to be something in.get_selection_dependents
Kevin Rue-Albrecht (05:37:37): > Oops. I can’t tell where this error would come from, but it makes me realise that I’ve focused primarily on CLI unit tests, and forgot about interactively testing the app last night
Aaron Lun (05:37:50): > Probably more likely due to the refactoring of the observer that checks whether or not there are any brushes in the current plot.
Aaron Lun (05:38:03): > I did it yesterday after noticing that it wouldn’t pick up lasso selections.
Aaron Lun (05:38:29): > I’ll look at it later, still dealing with beachmat.
Charlotte Soneson (06:15:07): > I removed theupdate_geom_defaultscalls in thedefaultcolorbranch. It works as long as there is no brushing, will test if with the brushing when we have fixed the problem above. I had to add an argumentcoloralso tocreate_points, I couldn’t see a way to figure out in there whether there is already coloring information in theaes, which would be overruled if I add anothercolor=in thegeom_point().
Aaron Lun (06:26:02): > Is this just a TRUE/FALSE?
Charlotte Soneson (06:26:23): > Yes
Charlotte Soneson (06:26:57): > I got it in the same way as for.build_aes
Federico Marini (06:30:21): > I’m stuck in a new flu wave which this time also got somewhat to me
Federico Marini (06:30:50): > but getting back to aaron’s question: i think we can wrap up and submit by friday/weekend
Federico Marini (06:33:20): > the open things, I listed them a couple of days ago - and we are on good track to close them up
Kevin Rue-Albrecht (06:33:21): > wow, that flu is just passing as a directed cyclic graph, isn’t it?
Federico Marini (06:33:42): > I wish i had an alert system like we had
Federico Marini (06:34:40): > damn, i have tomorrow a workshop to host, and wed I should go to freiburg
Federico Marini (06:35:12): > for the galaxy conference ->@Charlotte Sonesonare you joining as well?
Charlotte Soneson (06:35:58): > No…I’ve only used Galaxy once in my life (so far):slightly_smiling_face:
Federico Marini (06:38:26): > #metoo:stuck_out_tongue:but it is more for networking
Federico Marini (06:38:35): > and see what these guys are up to
Aaron Lun (07:59:52): > @Charlotte SonesonBugs are fixed. Though couldn’t you figure out whethercolor=was specified in theaeswith agrep?
Charlotte Soneson (08:23:11): > Yeah, I was thinking about that. But I thought it would look less nice and be potentially less robust, if there are other situations where you don’t want to add a default color.
Charlotte Soneson (08:35:57): > Ok, the default color setting seems to work also with the brushing, so I’ll merge it in
Aaron Lun (09:27:02): > Great.
Aaron Lun (09:27:25): > Also - for the love of god, let’s not respect an 80 character line length limit.
Aaron Lun (09:27:39): > I just went through plotting.R, I could barely read anything without putting it back.
Charlotte Soneson (09:29:32): > Actually I kind of like the short lines:slightly_smiling_face:but maybe I have a short attention span
Aaron Lun (09:30:11): > There are cases where a one-line command needs to be split into multiple lines.
Aaron Lun (09:30:16): > But that’s the exception rather than the rule.
Aaron Lun (09:30:24): > I mean, some of the split’s I’m seeing are just ridiculous.
Aaron Lun (09:30:35): > > sprintf( > blah blah balh > ) >
Aaron Lun (09:30:54): > I mean, it all fits on a line with no loss of comprehensibility.
Aaron Lun (09:51:53): > @Kevin Rue-AlbrechtYou forgot to doiSEE:::.self_lasso_path. How on Earth are we passing CHECK?
Aaron Lun (10:09:16): > Anyway, this is fixed, but it is still bemusing how we were passing CHECK before.
Aaron Lun (10:09:45): > Maybe something was wrong with the CI config.
Kevin Rue-Albrecht (10:20:22): > Eh.. how did I forget it?
Kevin Rue-Albrecht (10:20:49): > ah you mean theiSEE:::part ?
Kevin Rue-Albrecht (10:21:42): > eh.. that’s weird indeed, that the test succesfully calls it if it’s not exported
Kevin Rue-Albrecht (10:26:09): > Good fixes to the robustness of the plotting unit tests, btw. Especially the named references tocmdselement instead of positional. I fixed only the ones that were broken, and forgot to updates the other ones in the process
Aaron Lun (14:18:57): > If people can stress-test it tomorrow, we can submit on Wednesday.
Kevin Rue-Albrecht (15:37:06): > while I think of it: one of the things that I tried to homogenise in the vignette was the use of present/future tenses: “the buttonsdoesthis”, “the appwilldo that”, if anyone wants to have a rough read from the top, and see if anything stands out
2018-03-13
Davide Risso (09:58:48): > @Davide Risso has joined the channel
Federico Marini (11:39:15): > … e benvenuto@Davide Risso
Kevin Rue-Albrecht (12:37:58): > Did anyone get a chance to stress test the app today?
Charlotte Soneson (12:50:17): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-03-13 at 17.45.18.pngand commented: I have tried a bit and I haven’t managed to break it yet. I learned that if you set the font size to a negative valueggplot2will turn the labels upside down:upside_down_face: - File (PNG): Screen Shot 2018-03-13 at 17.45.18.png
Aaron Lun (13:05:06): > The more you know…
Kevin Rue-Albrecht (13:06:35): > hahah that’s nice to know though ^^
Aaron Lun (13:09:14): > If anyone’s testing, try to use therenamerbranch.
Aaron Lun (13:09:24): > I did a number of variable changes that … should be okay, but you never know.
Aaron Lun (14:23:31): > Marco
Kevin Rue-Albrecht (14:23:59): > … Polo?
Davide Risso (15:08:26): > @Federico Marinithanks! I’m mostly here just because I didn’t notice this channel until today and I wanted to see what awesome thing you’re cooking!
Davide Risso (15:08:55): > But I have some fairly large SCE objects if you guys want someone to beta test
Aaron Lun (15:22:44): > Beta testers always appreciated. Note that we’re waiting to start the BioC submission before we implement a down-sampling scheme for large data (> 20000 points).ggplotisn’t particularly slow at this number, but it does lag a bit on my desktop (~1s) which isn’t great for responsiveness.
Davide Risso (15:29:57): > OK, good to know! I’ll start with a smallish example and see how many cells I can push before it starts lagging..
Davide Risso (15:30:24): > you guys focused on scatterplots right? Or do you have heatmaps as well?
Davide Risso (15:30:36): > Sorry too lazy to open the vignette I guess:slightly_smiling_face:
Kevin Rue-Albrecht (15:30:58): > To infinity and beyond, has always been the plan:yum:
Kevin Rue-Albrecht (15:31:21): > Heat map is included
Aaron Lun (15:31:23): > lim = mind.
Aaron Lun (15:31:30): > Ah, can’t do math notation here.
Davide Risso (15:31:31): > never mind I found the heatmap
Kevin Rue-Albrecht (15:32:37): > Dot plot, heat maps, and the rising « square plots »
Davide Risso (15:33:49): > no ggridges?:slightly_smiling_face:
Davide Risso (15:34:00): > violin plots are so 2016!
Davide Risso (15:34:03): > :stuck_out_tongue:
Kevin Rue-Albrecht (15:34:52): > You just reminded me of my recent fish plots. I’m home in 2 min, will send one of them
Aaron Lun (15:35:52): > I didn’t know whatggridgeswere. Now that I know, I can’t say my life has been enriched.
Davide Risso (15:37:15): > Well it is the same info of a violin plot in half the ink… plus you don’t need to mentally rotate the plot 90 degrees for them to make sense
Davide Risso (15:37:43): > ok, I’ll stop before I get kicked out of the channel!:grimacing:
Kevin Rue-Albrecht (15:41:18): > @Kevin Rue-Albrechtuploaded a file:fun_with_fishes.png - File (PNG): fun_with_fishes.png
Kevin Rue-Albrecht (15:41:33): > dibs on packageaquaRium
Aaron Lun (15:41:35) (in thread): > Well, the overlaps don’t help.
Davide Risso (15:51:20) (in thread): > I think you can tweak the plot to avoid overlaps
Aaron Lun (15:52:15) (in thread): > So it’s just a typical density plot, then.
Davide Risso (16:21:08) (in thread): > yes!:smile:That’s why I like it!
Aaron Lun (16:21:29) (in thread): > :face_with_rolling_eyes:
Aaron Lun (16:22:10) (in thread): > I don’t have much love for violin plots either, but they are a bit nicer when there is no clear axis to terminate at.
Aaron Lun (16:23:40): > My colleague sitting next to me wants to know how you made that.
Aaron Lun (16:23:47): > Or specifically, what it means.
Aaron Lun (16:23:57): > (Students. Can’t live with them, can’t kill them.)
Kevin Rue-Albrecht (16:24:15): > I was just trying to understand the (three) different ways of scaling violins
Kevin Rue-Albrecht (16:24:42): > one is by area, another by max width, ….trying to remember the third
Aaron Lun (16:24:48): > My coleague thought it could help for stuff.
Aaron Lun (16:25:16): > But now he’s disappointed.
Aaron Lun (16:25:26): > He’s on the floor crying.
Kevin Rue-Albrecht (16:25:38): > well, it can definitely help, I took the idea from a Satija papuer figure, if I don’t confuse papers
Kevin Rue-Albrecht (16:26:07): > sorry to disappoint, it’s all ggplot2 magic
Kevin Rue-Albrecht (16:26:30): > geom_violin:scaleif “area” (default), all violins have the same area (before trimming the tails). If “count”, areas are scaled proportionally to the number of observations. If “width”, all violins have the same maximum width.
Aaron Lun (16:26:40): > Of course, it’s only useful when one violin is always contained within the other.
Aaron Lun (16:26:51): > Otherwise you end up with the same problem as for overlapping ridge plots.
Kevin Rue-Albrecht (16:27:51): > ahh sorry I should have said, for this particular figure, I’ve obviously overlaid the same data twice:geom_violin(..., scale="area") + geom_violin(..., scale="width")
Kevin Rue-Albrecht (16:28:22): > then play with the alpha of the smallest one (typically scale=area)
Aaron Lun (16:30:02): > Yes, I figured that. My comment was with respect to more general applications.
Kevin Rue-Albrecht (16:31:03): > ah.. well, then it depends on the information carried by the area, width, and number of data points in each violin
Kevin Rue-Albrecht (16:31:46): > - area was crap for 10x data, because all violins are typically shrunk to tiny area because there is usually always a group/cluster that has a distribution peak at 0
Aaron Lun (16:32:42): > lol
Kevin Rue-Albrecht (16:32:56): > - scale=width was nice in our case because it emphasises the mode of each group
Kevin Rue-Albrecht (16:33:28): > - scale=count can be helpful to convey how many cells are in each violin
Kevin Rue-Albrecht (16:33:59): > although that can also shrink smaller cluster to a useless area/size (more useful in presence of relatively balanced groups)
Kevin Rue-Albrecht (16:36:19): > PS: anyone who guesses the cell type(s) in this experiment, based on the figure, wins a free copy ofiSEE
Kevin Rue-Albrecht (16:36:28): > oh wait.
Aaron Lun (16:37:39): > It’s all MHC stuff.
Aaron Lun (16:37:44): > Every cell has those
Aaron Lun (16:38:09): > Macrophases (based on CCL2)?
Kevin Rue-Albrecht (16:38:34): > ding ding, we got a winner
Aaron Lun (16:38:43): > Fuck yeah
Kevin Rue-Albrecht (16:39:04): > to be fair, it’s either macros of DCs these days
2018-03-14
Federico Marini (03:23:41): > ouh
Federico Marini (03:24:03): > stephen hawking has passed away
Kevin Rue-Albrecht (03:24:33): > :confused:
Federico Marini (04:51:04): > plus, you guys
Federico Marini (04:51:17): > did you talk to the xkcd guy lately?
Federico Marini (04:51:19): > https://xkcd.com/1967/ - Attachment: Violin Plots > [Title text] “Strictly speaking, ‘violin’ refers to the internal structure of the data. The external portion visible in the plot is called the ‘viola.’”
Kevin Rue-Albrecht (05:26:46): > did anyone test theExperimentHubpart of the vignette recently?
Kevin Rue-Albrecht (05:27:39): > I just had a bug now
Kevin Rue-Albrecht (05:28:25): > at theirlba(exprs(sce1))step
Kevin Rue-Albrecht (05:32:33): > Problem solved, I think. Could indicate a bug in theexprs()method forSCEobjects
Kevin Rue-Albrecht (05:32:57): > > > exprs(sce1) > NULL >
Kevin Rue-Albrecht (05:33:25): > > > dim(assay(sce1, "exprs")) > [1] 23368 7706 >
Federico Marini (05:45:34) (in thread): > aaron, wait for the jellyfish plots to become a thing
Federico Marini (05:45:48) (in thread): > half violin half beeswarm
Kevin Rue-Albrecht (05:47:14): > I’ll retry after abiocValid(fix=TRUE)
Kevin Rue-Albrecht (07:21:42): > okay then, it’s just a matter of avoiding theexprs()accessor, I don’t really have time to investigate why it returns NULL on the SCE
Federico Marini (08:18:31): > probably connected to the S4Vectors mess?
Kevin Rue-Albrecht (08:18:39): > possibly
Kevin Rue-Albrecht (08:19:36): > i’ve got the fix locally, but if we can delay the submission to tomorrow, I wouldn’t mind going through the package one more time, with biocValid(fix=TRUE) up to date
Kevin Rue-Albrecht (08:20:05): > or maybe submitting tonight, if all goes well
Kevin Rue-Albrecht (14:29:39): > Anyone alive?
Kevin Rue-Albrecht (14:35:08): > Just wondering what the status is, on the two branches
Kevin Rue-Albrecht (14:35:57): > I’m on my way home where I’ll push a 3rd one with the fix and maybe some more after testing the test of the vignette code
Federico Marini (14:58:50): > on my way to health, while all others in the family fell in the flu trap again, good news
Federico Marini (14:59:24): > I got a Regular talk slot at ERUM in Budapest
Federico Marini (14:59:46): > with the title “Interactivity meets Reproducibility: the ideal way of doing RNA-seq analysis”
Federico Marini (14:59:47): > i.e.
Federico Marini (15:00:37): > if you don’t mind, I am happy to spread someiSEEjoy over there too combined with my other 2 pkgs
Kevin Rue-Albrecht (15:04:07): > Good with me. I’m curious to see how iSEE is received by the broader community
Federico Marini (15:06:08): > WORST news: is netflix down?!
Kevin Rue-Albrecht (15:07:06): > don’t think so, m’lady is watching it right now
Federico Marini (15:07:45): > she’s stealing my bandwidth:smile:
Federico Marini (15:08:13): > http://downdetector.com/status/netflix
Federico Marini (15:08:21): > ok, now seems to be ok again
Federico Marini (15:08:58): > i don’t know how reliable that site is, but it figures with my login pains. anyway, back on topic for the submission closings
Federico Marini (15:09:21): > I have tried to replace the icons with smaller buttons, but buttons behave unexpectedly
Kevin Rue-Albrecht (15:09:30): > ah. how so?
Federico Marini (15:09:34): > sometimes force the newline, sometimes don’t
Kevin Rue-Albrecht (15:09:39): > ahh. damn
Federico Marini (15:09:58): > inspecting the html itself, there are indeed some
s that slip in
Kevin Rue-Albrecht (15:10:08): > -_-
Kevin Rue-Albrecht (15:10:43): > Here, I’m going through the vignette code. I was too focused on the content, and hadn’t run the unevaluated chunks in a while
Kevin Rue-Albrecht (15:11:42): > … i’ll figure it out soon enough, but do we have an example scenario in the vignette that opens up an app with a row data plot?
Kevin Rue-Albrecht (15:13:02): > just so that users have the ‘full’ experience without having to worry about setting it up
Federico Marini (15:23:10): > not really, as far as i can recall
Kevin Rue-Albrecht (15:26:11): > btw, there is still a section in the vignette “Linking plots with brushing {#brushing}”
Kevin Rue-Albrecht (15:26:35): > does it sound weird “Linking plots with point selection” ?
Kevin Rue-Albrecht (15:26:42): > “by point selection”?
Kevin Rue-Albrecht (15:26:58): > “through point selection”? via?
Federico Marini (15:31:15): > what about Linking plots together , full stop
Kevin Rue-Albrecht (15:32:09): > Could be. > I also just thought of “Selecting data points and linking panels”
Kevin Rue-Albrecht (15:39:57): > little spoiler on a list of items that I’m preparing: > do we want the two print statements that show up when callingiSEE()?
Kevin Rue-Albrecht (15:40:50): > > > app <- iSEE(sce) > list() > character(0) >
Charlotte Soneson (15:41:15): > I was wondering that too: where are they coming from?
Kevin Rue-Albrecht (15:42:29): > all I can say for now is “not from iSEE-main”, I haven’t grep elsewhere
Charlotte Soneson (15:43:15): > seems to be in iSEE-extras
Kevin Rue-Albrecht (15:43:19): > separately, I might fix it myself to refresh my mind, but the cytof example is also behaving weird
Kevin Rue-Albrecht (15:43:29): > all the panels haveTimeon the Y axis
Kevin Rue-Albrecht (15:43:38): > instead of the markers set up in the vignette
Kevin Rue-Albrecht (15:43:56): > probably one of the featPlotArgs having been renamed
Kevin Rue-Albrecht (15:45:34): > XAxisGeneText->XAxisFeatName
Kevin Rue-Albrecht (15:45:38): > (and co.)
Kevin Rue-Albrecht (15:47:00): > (problem solved, i’ll push soon)
Charlotte Soneson (15:48:26): > Ok cool. Thecharlotte_fixbranch has just a couple of minor fixes (removing the unnecessarysizearguments, renaming one variable)
Federico Marini (15:50:25): > i think these two lines are leftovers from debugging sessions
Federico Marini (15:50:41): > for me they can disappear
Kevin Rue-Albrecht (16:31:46): > @Federico Mariniwhen I push back, can you please run the cytof example, and open the modal of plot links? > I’m getting anError: unused argument ()
Kevin Rue-Albrecht (16:32:43): > might be worth trying it now, even though you’ll have the issue about preset genes being ignored for the axes
Kevin Rue-Albrecht (16:35:03): > it’s pushed to branchkevin_edits
Kevin Rue-Albrecht (16:38:57): > dunno who’s expertise it is but the error is: > > Warning: Error in [.reactivevalues: unused argument () > Stack trace (innermost first): > 106: [ > 105: .snapshot_graph_linkedpanels >
Federico Marini (16:42:17): > i’ll check now
Kevin Rue-Albrecht (16:42:30): > cheers
Kevin Rue-Albrecht (16:47:31): > @Kevin Rue-Albrechtuploaded a file:cancer by gender example - File (PNG): cancer by gender example
Kevin Rue-Albrecht (16:48:09): > just thought that kind of view can be a nice, intuitive, visualisation of bi-categorical plot
Federico Marini (16:49:43): > uh
Federico Marini (16:49:51): > my error is different somehow
Kevin Rue-Albrecht (16:49:55): > ah?
Federico Marini (16:49:56): > unused argument (alist())
Federico Marini (16:50:13): > > Listening on[http://127.0.0.1:6456](http://127.0.0.1:6456)Warning: Error in [.reactivevalues: unused argument (alist()) > Stack trace (innermost first): > 106: [ [/Users/fede/Development/iSEE/R/codetrack.R#207] > 105: .snapshot_graph_linkedpanels [/Users/fede/Development/iSEE/R/codetrack.R#207] > 104: renderPlot [/Users/fede/Development/iSEE/R/iSEE-main.R#365] > 94: <reactive:plotObj> > 83: plotObj > 82: origRenderFunc > 81: output$outc2ff1e5bfe93d4c5 > 4: <Anonymous> > 3: do.call > 2: print.shiny.appobj > 1: <Promise> >
Federico Marini (16:51:16): > which points to the line where we look for rowstattables
Federico Marini (16:53:11): > the error still persists if I manually add the table, hmm
Federico Marini (17:12:55): > huh
Federico Marini (17:13:10): > the error is there also for the normal example?!
Kevin Rue-Albrecht (17:13:25): > you know what… I haven’t tried
Kevin Rue-Albrecht (17:13:36): > I just finished checking the vignette code
Kevin Rue-Albrecht (17:14:27): > I’m about to push that, so that it’s done, and that we can focus on the rest
Federico Marini (17:14:53): > there is also something strange
Federico Marini (17:15:01): > the docu says:
Federico Marini (17:15:02): > #’ Plot a snapshot of the links among panels > #’ > #’ Creates a graph of the current links among all currently active panels, and plots it. > #’ > #’ @param active_panels A data.frame containing information about the currently active panels. > #’ @param pObjects An environment containing .
Federico Marini (17:15:23): > but active_panels (print-ed out) is a reactive_values object
Kevin Rue-Albrecht (17:19:16): > …we should ask@author Federico Marini:wink:
Federico Marini (17:21:51): > technically correct
Federico Marini (17:22:15): > I do not remember anyway myself in switching from the data.frame to the reactive_values class
Federico Marini (17:22:37): > lemme check in the changes history
Kevin Rue-Albrecht (17:23:55): > btw, I’m leaving the code at that for the evening. See branchkevin_edits, branched offrenamer
Federico Marini (17:26:33): > back here it used to work
Federico Marini (17:26:34): > https://github.com/csoneson/iSEE/commit/610f5a64f8701f9ce262cf9e156593932bdd745e#diff-3a1df304f53cb0377884c78ee77c38cc - Attachment (GitHub): added sanitize cmds to tracking function, plus documenting it right away · csoneson/iSEE@610f5a6 > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (17:28:38): > merge 148? you mean it’s been broken for 2 weeks?
Federico Marini (17:29:33): > no it is definitely more recent
Federico Marini (17:29:39): > I think here came the change:https://github.com/csoneson/iSEE/commit/c2fc6d3f07ee695b15af043f8e7d292190599009 - Attachment (GitHub): Bug fixes after renaming to pass CHECK. · csoneson/iSEE@c2fc6d3 > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (17:35:50): > @Kevin Rue-Albrechtuploaded a file:unrelated, little motivational view of code coverage: - File (PNG): unrelated, little motivational view of code coverage:
Federico Marini (17:37:07): > ok I am a little lost - I think some of the changes in that round of renaming broke it. I need to check what the new names are - unless@Aaron Lunpops in like a real wizard of oz?:slightly_smiling_face:
Kevin Rue-Albrecht (17:37:49): > Well, I think he gave us a fair warning right after he pushed the updates; hence my checking of the vignette, better late than never
Kevin Rue-Albrecht (17:37:58): > other than that it all seems ok
Kevin Rue-Albrecht (17:38:53): > The only thing that I haven’t touched tonight was the UI, if anyone wants an easy win, my suggestions about the collapsible panels: > - [x] “Brushing parameters” -> “Selection parameters” > - [ ] “Receive select from” -> “Receive selection from” > - [x] “Brush effect” -> “Selection effect”
Federico Marini (17:50:13): > ok, fix found
Federico Marini (17:50:35): > just one instance where we were still calling the function with the reactive objects
Federico Marini (17:50:56): > kev, where is it most convenient to commit the bug fix?
Federico Marini (17:51:38): > seen the network plot, I’ll stick to kevin_edits
Federico Marini (17:53:58): > pushed now
Federico Marini (17:54:39): > sorry for the confusion generated/still in me. The mighty paracetamol can do wonders but still knocks me out a little
Kevin Rue-Albrecht (18:10:05): > Yeah, I’m not touching kevin_edits anymore tonight
Kevin Rue-Albrecht (18:10:08): > Going to sleep
Kevin Rue-Albrecht (18:11:00): > Well, it’s already admirable that you put up the time tonight, if you’re sick:face_with_thermometer:
Aaron Lun (19:08:47): > Was in a training course all day, the battle seems to be over.
Aaron Lun (19:09:43): > Yes, I switched the code tracker to take inactive_panelsas an argument, to try to reduce the scope to what the code tracker actually needed rather than exposing the entire reactive object to the function (which makes it difficult to determine what the function actually does).
Aaron Lun (19:10:21): > I thought I switched a number of them, maybe not. Lots of changes going on there.
Aaron Lun (19:10:56): > I guess everyone’s working offrenamer, then?
Aaron Lun (19:11:11): > Let’s aim for submission on Fri - is this achievable?
Aaron Lun (19:21:44) (in thread): > Destroy them. Probably debugging of brush or table links leftover from some ancient past.
Aaron Lun (19:22:13) (in thread): > scater’sexprs()looks for “logcounts”, not “exprs” anymore.
Aaron Lun (19:23:13) (in thread): > Happy with that.
Aaron Lun (19:23:43) (in thread): > Also can you add a section about how the row data plots link to the row statistics tables?
Aaron Lun (19:24:44) (in thread): > FINISH HIM.
Aaron Lun (19:25:38): > I have to read 2-3 papers tomorrow + organize an interview, so will be out of it. Again. Sorry but you guys will have to pick up the slack (lol); I’ll try to test what comes out at the end.
2018-03-15
Kevin Rue-Albrecht (03:04:40) (in thread): > Ragnarok: on
Kevin Rue-Albrecht (04:09:56) (in thread): > done
Aaron Lun (07:34:08): > Talk to me, people. How are we going?
Kevin Rue-Albrecht (07:37:04): > i think i got thekevin_editsbranch in a happy place
Kevin Rue-Albrecht (07:37:50): > could you have a look athttps://github.com/csoneson/iSEE/commit/f0be231feee6c0fb76b26128fe32c3458c185af7to see if I didn’t confuse point selection with Shiny brush anywhere?
Aaron Lun (07:40:02): > Almost. There’s one observer that is literally for the Shiny brush, so “brush” references there are accurate.
Kevin Rue-Albrecht (08:10:59): > Cool. I looked up the constant definitions but confused myself
Kevin Rue-Albrecht (08:13:28): > Sitting in a CyTOF seminar right now. Smiling at the lasso sélections:sunglasses:
Kevin Rue-Albrecht (08:26:10): > Btw, I think we should use the featExprPlotDefaults family of methods to set up the DataFrames in the vignette. > Please refer to the updated cytof vignette use case for an example
Kevin Rue-Albrecht (08:27:37): > I just realised that « Use case II: Changing the default start configuration » still defines DataFrames in a fairly manual (not robust if the column names change, as happened recently)
Kevin Rue-Albrecht (08:28:24): > Also it (the current DataFrames) doesn’t give users an idea of all columns that they could fiddle with
Kevin Rue-Albrecht (08:36:37) (in thread): > I can fix it tonight. If you feel like trying « git revert »https://git-scm.com/docs/git-revert
Charlotte Soneson (08:50:58): > I fixed Use Case II in the vignette and mergedkevin_editsintocharlotte_fix. I am running from meeting to meeting today, but I’ll try to have a final look tonight.
Kevin Rue-Albrecht (08:52:30): > Thanks ! > I’ll have to help people this afternoon. So can’t really do any programming until tonight either.
Kevin Rue-Albrecht (15:47:55): > ’sup?
Aaron Lun (15:54:07): > God, just finished my third paper of the day.
Charlotte Soneson (16:04:02): > Anyone object to me merging incharlotte_fix(which would also mergerenamer)? It seems to work as it should.
Aaron Lun (16:04:23): > I’m not going to get around to testing it today, so I’ll just trust you guys.
Kevin Rue-Albrecht (16:05:11): > well, i was happy with my changes, i trust you all anyway, but i’ll just have a look at the recent commits now simply i’m curious
Kevin Rue-Albrecht (16:07:13): > yup. nothing else to add. happy to see the merge happening. Do we have any other obvious last minute tweak on the agenda after that?
Charlotte Soneson (16:24:00): > Ok, it’s merged
Kevin Rue-Albrecht (17:14:12): > aaargh.. i just realised a markdown mistake of mine. “log[2]-expression” that didn’t turn into subscript in the vignette. Got confused withexpression()
Kevin Rue-Albrecht (17:15:29): > also, the text about the ExperimentHub is a bit out of sync with the code chunk: “immediately ready for analysis” describes a chunk where I apply PCA and t-SNE …
Kevin Rue-Albrecht (17:16:45): > eh nevermind, reading the ExperimentHub bit again, it does say “ready for post-processing and …
Kevin Rue-Albrecht (17:25:51): > eh. just sawcould not find function "geom_raster"in the heat map panel, after doing a fresh install
Kevin Rue-Albrecht (17:28:52): > alsogeom_scale_areawas still imported in theNAMESPACE:wink:
Kevin Rue-Albrecht (17:44:08): > Travis just finished… right when I spotted another typo:sob:
Kevin Rue-Albrecht (17:48:27): > mhh. looking at the time Travis spends “checking examples”, I wonder if we should consider using an even smaller SCE object in the man pages to save time. > I’d say we wait to see if we run into timeout issues on the Bioc build system
Kevin Rue-Albrecht (17:54:00): > ahhhrg hang on,@Aaron Lun, the erroneous brush -> select that I did today, was that pulled in master, then?
Kevin Rue-Albrecht (17:54:18): > I just remembered I said I’d fix it, if no one had time today
Kevin Rue-Albrecht (17:56:10): > you left your comment near the “Brush structure observers.“, while saying “These are all actually Shiny brush references.” I assume that means I should only revert things in that particular section,notthe entire commit, right?
Kevin Rue-Albrecht (17:57:58): > Although, looking back at it now, I’m still not 100% sure of which is Shiny and which is generic anymore. Can I leave that to you? Sorry that I brought confusion in there.
Federico Marini (18:23:16): > dropping in after the full duty day
Federico Marini (18:27:01): > I think we should leave a duck in the code for the pkg reviewers:smile:
Federico Marini (18:28:37): > jokes apart, good job in spotting out the couple of errors still floating around
Kevin Rue-Albrecht (18:29:50): > Hahahah
Kevin Rue-Albrecht (18:31:56): > Thanks. I’m leaving the branch open for whoever knows better than me which is shiny brush and which is point selection in commithttps://github.com/csoneson/iSEE/commit/f0be231feee6c0fb76b26128fe32c3458c185af7 - Attachment (GitHub): Brush -> Select, edit to man page · csoneson/iSEE@f0be231 > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
2018-03-16
Aaron Lun (05:13:03): > Not the entire commit.
Aaron Lun (05:13:06): > Dammit
Aaron Lun (05:13:11): > Just the Shiny brush observer.
Aaron Lun (05:13:40): > Look, if it’s dealing with the Shiny brush (brush_id), then it’s a damn brush.
Aaron Lun (05:23:35): > Okay, got my morning grumps out of the way.
Aaron Lun (05:23:59): > If it’s dealing with the input frombrushOpts, then it’s a shiny brush.
Aaron Lun (05:24:07): > Everything else is a select.
Aaron Lun (05:24:28): > This is why I had to go through them manually and replace them, rather than doing a find/replace.
Kevin Rue-Albrecht (05:27:38): > no worries, morning grumps are healthy too^^
Kevin Rue-Albrecht (05:28:12): > :sweat_smile:was a bit tired when I looked at that part of the code, and I confused myself starting to look through the code
Aaron Lun (05:33:11): > As a general rule, if it’s dealing with.brushFieldor.brushData, the Shiny brush is involved.
Aaron Lun (05:33:20): > I left those constants named as they were for that reason.
Kevin Rue-Albrecht (05:35:21): > When you wrote “This is why I had to go through them manually and replace them, rather than doing a find/replace.”, you mean before I messed them up again, right? Or do you currently have a commit locally when you did it (again) ?
Aaron Lun (05:35:54): > Don’t have any commit, so you’ll have to fix yours.
Aaron Lun (05:36:02): > I was referring to before.
Kevin Rue-Albrecht (05:36:17): > ok no worries
Kevin Rue-Albrecht (05:36:49): > just fixing a leftover of analysis from yesterday, and I’ll get to iSEE as soon as I can
Aaron Lun (05:38:02): > Just breathe, take your time, and message me about specific cases if confused.
Aaron Lun (05:38:19): > Though I will be inactive from 10;30 to ~2pm
Kevin Rue-Albrecht (05:38:39): > Yup. I want to avoid another confusing commit, too. I’ll ask before I commit, this time.
Kevin Rue-Albrecht (05:38:55): > Alright. Let’s catch up later today then.
Kevin Rue-Albrecht (09:49:32): > Just looking atiSEE-pkg.Rto lift over to another package, I noticed: > “Maintainer: John or Jane Doe ” > Will that one be an issue for the Bioc submission?
Aaron Lun (11:08:19): > Put@Charlotte Soneson
Federico Marini (15:01:38): > if we leave it like this, yes:stuck_out_tongue:
Federico Marini (15:01:42): > fine with charlotte
Kevin Rue-Albrecht (15:12:40): > alright, i’ll do those last couple of changes after dinner then
Charlotte Soneson (16:10:42): > Sure, I can do it. However, if@Kevin Rue-Albrechtor@Federico Mariniis up for it, that’s also very fine with me, especially given the amount of code provided by each of you exceeds my own contribution.
Aaron Lun (17:02:19): > Found one bug, horizontal violin plot.
Kevin Rue-Albrecht (17:19:16): > how’s it looking?
Aaron Lun (17:22:29): > Fixed, but graph maker is broken.
Kevin Rue-Albrecht (17:23:24): > ok. I’m about to start going through the ‘brush’ comment fix only now.
Aaron Lun (17:30:02): > You can tell that the code tracker didn’t really get much coverage, because I spotting an obvious bug there.
Aaron Lun (17:30:55): > Can you merge furtherfix into your PR before further fixes?
Aaron Lun (17:32:38): > NVM, I just did it.
Kevin Rue-Albrecht (17:32:41) (in thread): > Obviously all of us have contributed particular expertise in separate aspects of the package, so it’s not easy designing a single maintainer. Also, I am sure all of us are happy to keep maintaining and contributing to the package as a group. > I would personally love to look after the package and act maintainer, although I have to point out that I agreed with Steve to limiting my time for the benefit of local projects, and not get involved in significant new features
Kevin Rue-Albrecht (17:33:50): > opening up RStudio. Does that mean I should work off master again, now?
Aaron Lun (17:33:55): > MESSAGE TO ALL: please also test the row data plots, doable by adding some value torowDatabefore launch.
Kevin Rue-Albrecht (17:34:01): > oh no nvm, i got it
Kevin Rue-Albrecht (17:42:27): > btw, i don’t know how much you guys use that RStudio, but the----that I’m adding (e.g. > > # code section name ---- > > ) generate boomarks on the right of the editor panel, that I use for quick navigation
Aaron Lun (17:42:44): > I loathe Rstudio.
Aaron Lun (17:42:56): > But knock yourself out
Kevin Rue-Albrecht (17:43:17): > works with# title ####too, if you like
Kevin Rue-Albrecht (17:43:36): > (i feel like at this point, you don’t care ^^)
Aaron Lun (17:46:15): > I am exhausted right now.
Aaron Lun (17:46:18): > Haven’t had dinner.
Kevin Rue-Albrecht (17:46:35): > oww take care of yourself bud
Kevin Rue-Albrecht (17:46:58): > i personally don’t code well on an empty stomach ^^
Kevin Rue-Albrecht (17:49:54) (in thread): > I’d say this point is better discussed/agreed when everyone is synchronously online. > For the time being I’ve just brought iSEE-pkg and DESCRIPTION to a consistent state
2018-03-17
Aaron Lun (10:32:58): > Are we good?
Aaron Lun (10:54:20): > Unless someone tells me not to, I’m putting in the issue tonight.
Kevin Rue-Albrecht (11:03:59): > I’m out right now and hands too frozen to check: do we need a version bump?
Aaron Lun (11:04:21): > Well, we will when I put in the issue.
Kevin Rue-Albrecht (11:05:00): > Yep. Perfect
Kevin Rue-Albrecht (11:57:51): > Speaking of the pre-submission commit: > I’m actually thinking about changing to my GMail address, which is the one that I registered at BioC to maintain my other package
Aaron Lun (12:06:36): > Always a bit dodge.
Aaron Lun (12:06:48): > I like my email addresses to have affiliations.
Aaron Lun (12:06:51): > Looks more pro.
Kevin Rue-Albrecht (12:08:03): > i’ve always been torn with that side of the coin
Aaron Lun (12:09:16): > Anyone can sign up toaaron.lun@gmail.com
Kevin Rue-Albrecht (12:09:27): > absolutely
Aaron Lun (12:09:39): > With some effort, I could probably makekra.bioconductor@gmail.com
Kevin Rue-Albrecht (12:09:44): > well, you’re right. I’ll keep the other two the way they are (one of them I lost access to the university account anyway)
Aaron Lun (12:10:00): > Just update them all to your latest affiliation
Aaron Lun (12:14:53): > Does anyone still have that link to the server?
Aaron Lun (12:15:00): > From fed
Kevin Rue-Albrecht (12:15:06): > in my history, hang on
Kevin Rue-Albrecht (12:15:14): > imbei.uni-mainz.de:3838/iSEE/
Aaron Lun (12:15:28): > thx
Aaron Lun (12:16:47): > damn, takes too long to load.
Kevin Rue-Albrecht (12:18:06): > depends what kind of server it is; during my PhD i set up a Shiny server on a 10 year old box that hadn’t been updated in 5 years I think
Aaron Lun (14:48:43): > Just noticed that certain types of linked table changes can cause the memory to become invalid. Specifically, when you color a particular panel by feature name, change the linked table for colour, then pick to colour by coldata; the new linked table is stored in memory but no edge exists to the current panel in thetable_links.
Aaron Lun (14:49:23): > Upon deletion of the linked table, the current panel will be trying to colour by a table that doesn’t exist.
Aaron Lun (14:50:16): > Not entirely sure how to fix this, I’ll think about it tomorrow.
Aaron Lun (14:50:28): > But submission can be done tonight, it’s a very subtle bug.
Aaron Lun (14:51:26): > Just merge infurtherfixesbefore you do so.
Federico Marini (14:54:23): > thatsite might take ages to load:stuck_out_tongue:
Federico Marini (14:54:29): > there’s ashinymissing
Federico Marini (14:54:55): > http://shiny.imbei.uni-mainz.de:3838/iSEE
Federico Marini (15:06:35): > reading up the README.md, do we want to take out the hover-part?
Kevin Rue-Albrecht (15:32:17): > I’d say yes. We can always have it as an open issue to document discussion, if at all possible in our current setup
Federico Marini (15:53:23): > ok, edited that. plus the heatmap part which is now:white_check_mark:
Federico Marini (16:04:44) (in thread): > summing up in my conversation with kevin, I would also be honoured to commit in the maintaining of the pkg, probably still the best constellation is two of us. > From my side, I am relatively free in setting up my duties, and nothing else big to maintain is there
Federico Marini (16:05:06) (in thread): > the two-maintainer is probably blocked itself from R CMD check
Federico Marini (16:05:41) (in thread): > but we could list us in the pkg documentation
Charlotte Soneson (16:07:24) (in thread): > Yeah, I guess anywhere except in the Maintainer field there can be multiple people. To me it doesn’t matter so much who is listed there, I am also assuming we will all contribute to the actual maintenance (I guess we can all follow theiSEEtag on BioC-support?)
Federico Marini (16:08:22) (in thread): > paraphrasing what kevin said right now, this honour-duty thing is best suited for the sunlight: we all have credits for creating this piece of ~heart~ code
Federico Marini (16:08:46) (in thread): > exactly, yes
Federico Marini (16:09:09) (in thread): > BTW, is there a way to set up auto-notification on own tags?
Federico Marini (16:09:14) (in thread): > on the support site
Charlotte Soneson (16:10:00) (in thread): > If I remember correctly you can decide which tags you want to get notifications for, and youhaveto follow the ones where you are the maintainer
Federico Marini (16:12:03) (in thread): > ok, mine are in
Charlotte Soneson (16:12:46) (in thread): > Ehm, sorry, your what?
Federico Marini (16:12:58) (in thread): > my tags, in my profile:stuck_out_tongue:
Charlotte Soneson (16:13:04) (in thread): > Ah ok:slightly_smiling_face:
Federico Marini (16:13:12) (in thread): > sorry, too few verbosity
Kevin Rue-Albrecht (17:15:12) (in thread): > I’ve just put the tags in my profile too, which reminds me how much nicer this is compared to emails:slightly_smiling_face:
Kevin Rue-Albrecht (17:15:34) (in thread): > (to track questions, discussions, and solutions)
Federico Marini (17:49:17): > also edited the vignette, removed some leftover comments, and added the website link over there
Federico Marini (17:51:21): > same also for the README
2018-03-18
Aaron Lun (08:22:59): > @Federico MariniMinor changes required to.snapshot_graph_linkedpanels, to reflect a minor change in the table linking strategy. See recent changes in.define_table_linksfor an example. Full explanation as follows:
Aaron Lun (08:25:35): > The problem was what I discussed in my comment yesterday, where table links were being destroyed when a plot, e.g., chose to colour by column data rather than the feature name (with a linked table). This is problematic when you then delete the table, as the deletion observer uses the table links to reset the memory of the linked table in each receiving plot. Without a table link, it is possible to invalidate the memory state of the app by having a link to a table that no longer exists.
Aaron Lun (08:27:17): > Now, the table links are no longer destroyed, which ensures that memory resetting upon table deletion operates as expected. However, it also means that there are links between tables and panels even if the receiving panel is not colouring by feature name (or ordering the x-axis by feature name, for feature expression plots).
Aaron Lun (08:28:00): > This causes no problems for the observers, where re-rendering of the panel is only triggered when thecolorBy(orXAxis) is set to “Feature name”. However, it does show up in the panel link information and in the graph, which would seem “incorrect” to users as the panel does not seem to be receiving any information from the table. The former I have already fixed via changes todefine_table_links; the latter needs a corresponding fix via.snapshot_graph_linked_panels.
Kevin Rue-Albrecht (08:30:15): > wow.. well done investigating this one:no_mouth:
Aaron Lun (08:32:21): > Anyone in<!channel>with some free time can try to breakfurtherfixes.
Aaron Lun (08:32:41): > If we don’t get any bug reports by tonight,@Kevin Rue-Albrechtsubmit to BioC.
Kevin Rue-Albrecht (08:35:09): > Sure I can do that… but didn’t you mean that the.snapshot_graph_linkedpanelsfix has to happen first? Or is it minor enough to submit even before that?
Aaron Lun (09:52:51): > Yes, it is minor enough. But it would be nice to also fix that.
Aaron Lun (10:06:59): > I mean, you can also give it a shot, it is pretty easy to do. I could do it but I thought I’d share the love.
Federico Marini (10:40:04): > I won’t be able to catch up on that until later tonight - kids first, then parents committee meeting (yee-haa)
Federico Marini (10:40:37): > if someone else can/want to pick up on that, feel free to do so. And to cash in my thankfulness:slightly_smiling_face:
Kevin Rue-Albrecht (10:41:14): > I’ll have a look then. I haven’t kept a close eye on the graph-linking of panels, to be honest, but this seems like a good excuse for me to catch up on it
Kevin Rue-Albrecht (10:41:59): > I was writing docs on something else anyway, and welcome a break of recreational programming ^^
Aaron Lun (11:13:32): > Okay, I’m done for the day..
Kevin Rue-Albrecht (11:15:01): > on other stuff, or generally? I was just getting familiar with theprotected <- switch(mode,area of iSEE-main.R here
Aaron Lun (11:15:23): > Moving onto other stuff now.
Aaron Lun (11:15:31): > Make sure you pull the latestfurtherfix.
Aaron Lun (11:15:47): > I modified that exact region that you’re talking about, because the feature name observers need some distinct handling.
Kevin Rue-Albrecht (11:16:38): > I did pull the latest. I’m studying the changes as much as what the code generally does in that section
Aaron Lun (11:16:59): > Be prepared for:shocked_face_with_exploding_head:
Kevin Rue-Albrecht (11:17:34): > hehehe.. i got used to that feeling every time you refactored the code:stuck_out_tongue:
Kevin Rue-Albrecht (11:27:36): > boy.. it took me a bit of time to understand and reproduce the scenario you described yesterday, I can see the arrow that still exists from rowDataTable to the panel even after I change ColorBy from featureName-by-rowDataTable to colData
Kevin Rue-Albrecht (11:27:55): > headache is kicking in:sweat_smile:
Kevin Rue-Albrecht (11:30:46): > but yeah, as you described, I can see that the panel link information disappears. Onto the igraphpulling up sleeves
Kevin Rue-Albrecht (11:41:16): > btw, useful for debug:for ( obj in ls(pObjects) ) { cat('---',obj,'---\n'); print(get(obj, envir = pObjects)) }
Kevin Rue-Albrecht (11:41:36): > a bit verbose, but at least i know what’s going on
Kevin Rue-Albrecht (11:44:00): > hm..@Aaron Lunare you positive that the fix needs to go in.snapshot_graph_linkedpanels? > from what I see the function does what it’s supposed to do, given the input, it receives: > > --- table_links --- > $rowStatTable1 > $rowStatTable1$color > [1] "featExprPlot1" > > are consequently draw an edgerowStatTable1 -> featExprPlot1
Aaron Lun (11:44:29): > This should be the case even when you changeColorbyin featExprPlot1 from “Feature name” to “Column data”?
Aaron Lun (11:45:09): > Which would not be correct.
Kevin Rue-Albrecht (11:45:39): > hang on. i click but somehow my print statement does not seem to trigger again.
Kevin Rue-Albrecht (11:46:30): > ah no stupid me, the print only triggers when I display the graph
Kevin Rue-Albrecht (11:47:27): > yep - confirmed - even when I switch to colData, I still see: > > --- table_links --- > $rowStatTable1 > $rowStatTable1$color > [1] "featExprPlot1" >
Kevin Rue-Albrecht (11:48:21): > I’ll keep digging, but if you want to convince yourself, put the abovefor ( obj in ls(pObjects) ) { cat('---',obj,'---\n'); print(get(obj, envir = pObjects)) }as the first line of.snapshot_graph_linkedpanels
Kevin Rue-Albrecht (11:48:59): > then, every time you open the graph modal, it’ll print the contentpObjects, including thetable_linksvariable
Kevin Rue-Albrecht (11:53:09): > looking fortable_linksthrough iSEE-main.R, I have a feeling it’s something to do around line 1159# Observers for the linked color by feature name. This also updates the table_links information.
Aaron Lun (11:53:30): > What are you trying to do?
Aaron Lun (11:53:44): > The fact that the table is linked is correct.
Aaron Lun (11:53:56): > The only thing we need to change is that the link does not get shown in the plot of the graph.
Aaron Lun (11:54:11): > We can’t unlink the table, see my comments at the start of the day.
Kevin Rue-Albrecht (11:55:52): > ah. I’ll go find that comment, because the problem seemed to be that thetable_linksshouldn’t keep stating that rowDataTable ->colors-> panel, when the color by selection is changed to e.g. colData
Aaron Lun (11:56:36): > Start of the day
Aaron Lun (11:56:38): > I meant
Kevin Rue-Albrecht (11:56:53): > ok, no worries, I’ll read through that again
Kevin Rue-Albrecht (11:59:16): > riiiight > i won’t go into the details of why the link is important to invalidate the memory state, but I get the bottomline - Attachment: Attachment > The problem was what I discussed in my comment yesterday, where table links were being destroyed when a plot, e.g., chose to colour by column data rather than the feature name (with a linked table). This is problematic when you then delete the table, as the deletion observer uses the table links to reset the memory of the linked table in each receiving plot. Without a table link, it is possible to invalidate the memory state of the app by having a link to a table that no longer exists.
Aaron Lun (12:00:20): > Because the function that deletes the table searches through the table links to wipe the memory of the deleted table from all receiving panels.
Kevin Rue-Albrecht (12:01:08): > ok. anyway, no matter whattable_linksremembers, I can see thatmemoryhas the right information to draw the up-to-date edges
Aaron Lun (12:01:28): > yes
Kevin Rue-Albrecht (12:01:48): > as you said::exploding_head:
Aaron Lun (12:04:25): > Yeah, welcome to the club.
Kevin Rue-Albrecht (12:40:40): > ehm.. just playing with the app, to further understand the links.. i caused a crash when i linked the heat map to the rowDataTable at the moment of importing the features
Kevin Rue-Albrecht (12:41:03): > did anyone try that recently? I don’t think I touched any code related to that aspect
Aaron Lun (12:41:18): > what do you mean “at the moment of importing”
Kevin Rue-Albrecht (12:41:40): > oh no wait, not the table, the plot
Kevin Rue-Albrecht (12:41:59): > after setting up the link, when i click the button
Aaron Lun (12:42:23): > probably something to do with the damn observer that we didn’t get around to fixing.
Kevin Rue-Albrecht (12:42:25): > got it, it crashes when no point is selected in the plot
Aaron Lun (12:42:53): > I think that code is fully isolated in a single observer, so just have a look at it.
Kevin Rue-Albrecht (12:42:56): > btw error isError in which: argument to 'which' is not logical
Kevin Rue-Albrecht (12:44:28): > I won’t have the time to fix all that today I’m afraid, I haven’t finished dealing with the table links yet
Aaron Lun (12:44:52): > Table links are taking that long?
Aaron Lun (12:45:00): > Should be a case of just copying fromdefine_table_links.
Kevin Rue-Albrecht (12:46:05): > I didn’t look at it this way:sweat:
Kevin Rue-Albrecht (12:49:40): > as I said, I hadn’t really look at creation/deletions of links so far, I was merely accessing them through memory to doodle the ggplots
Aaron Lun (12:53:33): > Don’t bother looking at creation/deletion of links, this should not be necessary for fixing the graph plot.
Kevin Rue-Albrecht (13:11:01): > I’m going for a walk. I’m just not there today. Need to clear my head
Aaron Lun (15:37:35): > Just pull inmemoryinto thesnapshot_graph_linkedpanelsfunction, and copy the philosophy in.define_table_links.
Federico Marini (18:29:09): > @Kevin Rue-Albrecht: i’m back, I can be on it
Federico Marini (18:30:28): > @Aaron Lun, I’ll read the logic behind the other function and carry over
Kevin Rue-Albrecht (18:34:49) (in thread): > yeah, if you can handle, I’m happy to leave it to you. I just couldn’t wrap my head around it, and I was not feeling top notch today
Federico Marini (18:35:09) (in thread): > noprobs
Kevin Rue-Albrecht (18:43:27) (in thread): > also, I haven’t looked up yet where in the code to solve the heat map bug when features are imported from a row data plot without any point selected. It seems to be because awhichstatement somewhere returns an empty vector
Federico Marini (18:44:16) (in thread): > on that I cannot help much out of the blue
Federico Marini (18:44:18) (in thread): > but
Federico Marini (18:44:20) (in thread): > for my part
Federico Marini (18:44:25) (in thread): > the fix is coming
Federico Marini (18:44:46) (in thread): > I’ve reused the code from.define_table_links
Kevin Rue-Albrecht (18:45:07) (in thread): > i’ll spend a little while trying to pinpoint it now
Kevin Rue-Albrecht (18:46:34): > re: submission, let’s just wait for Fede’s fix to pass Travis and merge, that’ll give me time to look at the heat map bug when 0 features are imported from a row data plot. With those two we’ll be all set tomorrow night
Federico Marini (18:46:46): > -> indeed most of the code could be copied
Federico Marini (18:46:59): > fix pushed
Federico Marini (18:47:28): > I’m curious to see the spb in action:slightly_smiling_face:
Kevin Rue-Albrecht (18:50:27): > i must be really out of it: the what?
Kevin Rue-Albrecht (18:50:46): > as much as I’d like to imagine a ‘Shiny power brush’, I don’t think that’s it
Aaron Lun (18:52:54) (in thread): > Note that you don’t needtrans[2], this was only for.define_table_links.
Aaron Lun (18:53:17): > single package builder, I presume.
Aaron Lun (18:54:50): > I daresay that the import bug is generated on line 1396 ofiSEE-main.R.
Aaron Lun (18:55:12): > When there is no brush, you end up withwhich(NULL), basically.
Kevin Rue-Albrecht (18:56:38): > yup, I was just a few lines below,if (length(incoming) > limit)for the counterpart
Kevin Rue-Albrecht (18:57:18): > what’s the desired behaviour in this case? I was thinking anelse ifwith ashowNotificationand actually no update of the panel
Aaron Lun (18:58:30): > That seems sensible to me.
Federico Marini (19:01:04) (in thread): > should i cut out the 2nd element for each?
Kevin Rue-Albrecht (19:03:30): > any estimate how many bad things could happen if.get_selected_pointswould returnlogical()instead ofNULLfor 0 points? > I can see three calls to the function fromiSEE-main: > > R/iSEE-main.R:1185: selected <- .get_selected_points(rownames(pObjects$coordinates[[plot_name]]), dec_name, > R/iSEE-main.R:1269: selected <- .get_selected_points(rownames(gene_data), pObjects$memory$rowStatTable[id0,.selectByPlot], > R/iSEE-main.R:1385: incoming <- which(.get_selected_points(rownames(gene_data), origin, pObjects$memory, pObjects$coordinates)) >
Kevin Rue-Albrecht (19:04:00): > I’m building and testing that approach right now, before I fallback on the sensible approach I described above
Aaron Lun (19:04:19) (in thread): > Yep
Aaron Lun (19:05:08): > I suspect the others are fine, andNULLis actually a sensible output when a selection doesn’t exist (rather thanlogical(0), which is valid when a brush exists in a plot with no points).
Kevin Rue-Albrecht (19:05:51): > Interestingly, it causes a warning on startup: > > Listening on[http://127.0.0.1:1234](http://127.0.0.1:1234)Warning: Error in [[<-.data.frame: replacement has 0 rows, data has 20908 > > but then behaves as the sensible handling I described
Federico Marini (19:06:13) (in thread): > done, pushing
Kevin Rue-Albrecht (19:06:14): > Anyway, in that case I agree that returningNULLand handling the situation explicitly is better
Federico Marini (19:07:01) (in thread): > thank you guys for the soft sneaking-back-in. took a while to get that the solution was 80% already there
Aaron Lun (19:07:05): > The other uses have explicitNULLprotection.
Aaron Lun (19:13:45): > Anyway, I bought a new lightbulb. Very happy with it.
Aaron Lun (19:14:06): > A+ energy rating, one of those non-filament ones.
Aaron Lun (19:14:13): > 4 GBP well spent.
Federico Marini (19:14:59): > well done:slightly_smiling_face:
Federico Marini (19:15:19): > that expensive light bulbs over there?
Federico Marini (19:15:33): > or are they “sunday prices”?
Aaron Lun (19:15:38): > Dunno, it’s the first time I bought them.
Aaron Lun (19:15:59): > Accidentally broke my old one by hitting it when I was putting on my jumper.
Kevin Rue-Albrecht (19:16:50): > damn.. I was losing my sanity now getting why the error was still thrown after rebuilding the package: forgot to runapp <- iSEE()before running the app again -_-
Kevin Rue-Albrecht (19:17:24): > alright, problem solved: pulling-pushing
Federico Marini (19:17:46): > are we rrrrrrready to rumble?
Kevin Rue-Albrecht (19:17:53): > I was gonna ask
Kevin Rue-Albrecht (19:18:13): > i feel like I’ve said that too many times before, but here goes: I’m happy where we’re at
Federico Marini (19:18:53): > my point on this is to get the process running
Federico Marini (19:19:02): > smaller things, we can always fix as we go
Kevin Rue-Albrecht (19:19:18): > exactly
Kevin Rue-Albrecht (19:22:18): > space odyssey music please:innocent:
Kevin Rue-Albrecht (19:28:36): > Seriously, just for the mechanics right now: I’m waiting on Travis to be happy, then I merge tomaster. In the meantime, I’ve filled the GitHub issue on Bioconductor/Contributions and just wait for the merge to push the green button. Does that sound right?
Federico Marini (19:28:50): > guess so
Aaron Lun (19:28:57): > Yep. Also, four space indents, at least for all code in the same function.
Kevin Rue-Albrecht (19:30:41): > (I saw your review comment on GH Aaron, btw. Can’t believe I dropped thewhich)
Kevin Rue-Albrecht (19:30:52): > really off my game today
Kevin Rue-Albrecht (19:31:43): > I had moved it for a second, when I tested thelogicalapproach, and forgot to put it back in place
Kevin Rue-Albrecht (19:31:54): > ah.. while I’m at it: version bump no?
Aaron Lun (19:32:15): > Yes.
Kevin Rue-Albrecht (19:32:28): > I’ll check now, but I believe it has to be 0.99.1
Aaron Lun (19:32:52): > 0.99.0 was fine the last time I submitted.
Kevin Rue-Albrecht (19:33:16): > yep, i literally just read it now
Kevin Rue-Albrecht (19:33:21): > https://bioconductor.org/developers/package-submission/
Kevin Rue-Albrecht (19:35:02): > Is the format of the README ok as it is? I know a few of them are valid, but I never did a simple markdown format
Kevin Rue-Albrecht (19:35:37): > the link above suggestsInclude an inst/NEWS file
Kevin Rue-Albrecht (19:36:30): > I’ve been using more of a *Rd format in the past:https://github.com/kevinrue/TVTB/blob/master/inst/NEWS.Rd
Federico Marini (19:43:34): > they’re all fine as long as they’re read from the news function
Aaron Lun (19:44:10): > I use *.Rd as well.
Kevin Rue-Albrecht (19:44:17): > Also, anyone ran BiocCheck recently? The only one that caught my attention wasChecking for namespace import suggestions...with for instance a whole bunch ofimportMethodsFrom
Aaron Lun (19:44:18): > None of this markdown rubbish.
Kevin Rue-Albrecht (19:44:27): > ah? fight? ^^
Aaron Lun (19:44:29): > I ran it, didn’t see anything untoward.
Aaron Lun (19:44:53): > Did you rundevtools::document()recently?
Kevin Rue-Albrecht (19:44:54): > Prefer a pureNEWStext format, Aaron?
Kevin Rue-Albrecht (19:44:58): > I did
Kevin Rue-Albrecht (19:45:03): > didn’t change a thing
Aaron Lun (19:45:16): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (19:45:41): > that kind:https://github.com/Bioconductor/BiocParallel/blob/master/NEWS? - Attachment (GitHub): Bioconductor/BiocParallel > BiocParallel - Bioconductor facilities for parallel evaluation
Aaron Lun (19:45:41): > I thought we caught everything. Best to go through and see where they’re coming from.
Kevin Rue-Albrecht (19:45:46): > sure
Kevin Rue-Albrecht (19:46:52): > lol, I guess we’ll stick with Markdown
Aaron Lun (19:48:45): > :face_with_open_mouth_vomiting:
Kevin Rue-Albrecht (19:49:09): > eh.. to be honest, I just checked, and I can’t see the NEWS file listed on the iSEE index page, can you guys?
Kevin Rue-Albrecht (19:49:32): > my TVTB package that uses the *Rd format has it listed in there
Aaron Lun (19:50:08): > What index? There is no NEWS file currently.
Kevin Rue-Albrecht (19:50:33): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-03-18 23.50.20.png - File (PNG): Screenshot 2018-03-18 23.50.20.png
Aaron Lun (19:50:35): > It should live ininst
Kevin Rue-Albrecht (19:51:33): > moving it now
Kevin Rue-Albrecht (19:54:41): > still doesn’t show after either build/reload or document
Kevin Rue-Albrecht (19:55:18): > I’ve just checked online for other packages that have a NEWS.md > e.g.https://github.com/yihui/knitr - Attachment (GitHub): yihui/knitr > knitr - A general-purpose tool for dynamic report generation in R
Aaron Lun (19:55:32): > Sounds like we needNEWS.Rd.
Aaron Lun (19:55:38): > The way it should be.
Kevin Rue-Albrecht (19:55:48): > still, it seems the *Rd is the one kicking in:https://github.com/yihui/knitr/tree/master/inst - Attachment (GitHub): yihui/knitr > knitr - A general-purpose tool for dynamic report generation in R
Kevin Rue-Albrecht (19:55:56): > i’ll do that now
Kevin Rue-Albrecht (19:56:16): > as part of thewhich()fix anyway
Federico Marini (20:06:00): > my 2 cents, I have my NEWS for my packages in the main folder, as plain text
Federico Marini (20:06:21): > but I’m fine with either option
Kevin Rue-Albrecht (20:07:23): > i’m just gonna go ahead and say it: i’m too lazy to push another commit right now:sweat_smile:
Kevin Rue-Albrecht (20:08:18): > Travis is ‘building package’ right, now, and to pass the time I’m playing with the app one last time before clicking the submission button on GH/Bioc/Contributions
Kevin Rue-Albrecht (20:15:11): > alright, it’s happening
Kevin Rue-Albrecht (20:16:04): > 678: not that bad for a number:https://github.com/Bioconductor/Contributions/issues/678 - Attachment (GitHub): iSEE: Interactive SummarizedExperiment Explorer · Issue #678 · Bioconductor/Contributions > Update the following URL to point to the GitHub repository of the package you wish to submit to Bioconductor Repository: https://github.com/csoneson/iSEE Confirm the following by editing each che…
2018-03-19
Charlotte Soneson (03:06:55): > Great job!
Kevin Rue-Albrecht (03:46:23): > Is it enough to tag people@Federico Marini? Or do they have to comment/subscribe to the issue as well? I can see only three participants: bot , you and me
Federico Marini (05:09:32): > I hope so. Otherwise well, we will just post too
Federico Marini (09:29:50): > ok, the engine is on:slightly_smiling_face:
Federico Marini (09:30:03): > the SPB gave us already green light, which is a good start!
Charlotte Soneson (09:31:57): > :+1:and yes, it is enough to be mentioned in the issue, I got the notifications too.
Federico Marini (09:42:12): > excellent
Federico Marini (09:42:21): > we got the antennas up:smile:
Aaron Lun (14:58:21): > PHAWRG
Kevin Rue-Albrecht (14:58:30): > i was gonna say that too
Aaron Lun (14:58:40): > Am so tired.
Kevin Rue-Albrecht (15:07:26): > one thing that bugs me about the current behaviour of the heat map “Import feature” button: even if I clear all the genes to prepare for a ‘clean’ import (from either rowDataPlot or table), the new set of features will include the last unselected feature from the previous heat map
Aaron Lun (15:08:14): > Blame that on Shiny.
Kevin Rue-Albrecht (15:08:31): > i do
Kevin Rue-Albrecht (15:10:19): > still, I think we should have a friendlier way of handling than telling users to manually remove the first feature (which seems to be always the one carried over) in the updated selectize
Kevin Rue-Albrecht (15:11:37): > possibly overkill, but I see just enough space for one more button beside “import features” and “suggest order” that could host a button “set features”
Kevin Rue-Albrecht (15:12:52): > if that’s how we agree to handle it, I should be able to add the button and the action, reusing the code of ‘import features’
Aaron Lun (15:14:42): > Don’t understand what this “set features” button would do.
Kevin Rue-Albrecht (15:15:04): > ‘Import’ adds, ‘Set’ would… well, set
Aaron Lun (15:15:15): > Still don’t understand.
Kevin Rue-Albrecht (15:17:25): > ah. well, say you’ve selected a bunch of genes in rowDataPlot, and rather than adding them to the list of features already shown in the heat map, you want the heat map to show only that new set of features.
Kevin Rue-Albrecht (15:18:03): > that’s what the “Set features” would be for
Aaron Lun (15:18:39): > right.
Kevin Rue-Albrecht (15:18:48): > it’s basicallyreplacingthe list of features in the heat map, rather thanextendingit
Kevin Rue-Albrecht (15:25:57): > hm.. I’m a little doubtful how to implement it: seems to require duplication of iSEE-main lines 1405-1435 to create new observers. Smarter option?
Kevin Rue-Albrecht (15:27:30): > all that for the sake of dropping the linecombined <- union(pObjects$memory[[mode0]][id0, .heatMapFeatName][[1]], incoming), as onlyincomingwould be relevant then
Aaron Lun (15:35:02): > Just add a button that says “Clear selection”, and the next import will be fresh.
Kevin Rue-Albrecht (15:37:16): > ah right, i get you: the empty selectize doesn’t send an update to redraw the heat map, but its value can still be cleared
Kevin Rue-Albrecht (15:43:08): > also, just thinking it now: we probably want the buttons to beabovethe selectize, to save some scrolling
Aaron Lun (15:43:32): > Yes, that’s true.
Kevin Rue-Albrecht (15:45:03): > one more thing: is “Suggest feature order” a little verbose? Is it a safe way of saying “Cluster features”, without implying that there is ‘one true way of clustering’ ?
Aaron Lun (15:45:47): > Change it if you like.
Kevin Rue-Albrecht (15:46:17): > (I like ‘two word’ button style, and try to make a bit of spaces for the other buttons)
Kevin Rue-Albrecht (18:07:43): > have a look at branchheatmap_setplease, and let me know if that’s alright for you guys
Kevin Rue-Albrecht (18:07:53): > (over and out)
Kevin Rue-Albrecht (18:10:52): > PS: from now on, we should track new stuff in theinst/NEWS.Rdfile. > PS2: I’ve version-bumped already, as it’s a side branch. Considering we’re now watched by bioc-bot, let’s be careful to merge only when we’re addressing the reviewer comments
2018-03-20
Federico Marini (05:37:41): > <!channel>, or better,@Charlotte Soneson: did we already setup the webhook for the further changes?
Charlotte Soneson (05:38:04): > Nope (or at least I didn’t)
Federico Marini (05:40:39): > i don’t know whether we can since you are the official owner of the repo
Kevin Rue-Albrecht (05:40:41): > i didn’t do anything either. I think only Charlotte can
Charlotte Soneson (05:40:56): > Ok:slightly_smiling_face:I’ll do it now
Charlotte Soneson (05:44:43): > Done
Federico Marini (10:37:13): > AAAAAAAAAAAAAA
Federico Marini (10:37:15): > guys
Federico Marini (10:37:20): > we’re in!
Federico Marini (10:37:34): > > I've looked over this package and I'm happy to say that I have no issues. The package is extremely well done as it is! I'm going to move to accept this package now. >
Charlotte Soneson (10:37:58): > Cool!
Aaron Lun (10:39:51): > Woah, that was quick.
Federico Marini (10:45:44): > Well we should pat on our shoulders for all we did in proactive way
Federico Marini (10:46:09): > we saved them also valuable time, especially in pre-release phase:slightly_smiling_face:
Aaron Lun (10:51:25): > This is only the beginning.
Aaron Lun (10:51:55): > I propose we split into two teams. Team A (myself) will explore the downsampling problem. Everyone else should start writing the damn paper.
Kevin Rue-Albrecht (11:20:50): > I’ve been thinking about the paper a lot recently, at lest trying to figure out what could go in there. I’ll definitely have to spend a section on the ExperimentColorMap. But that also depends on which article format/journal we shoot for
Aaron Lun (11:23:24): > Well, I will start working on the downsampling problem tomorrow, so if you guys bang your heads together we should have a draft MS before the start of next week.
Kevin Rue-Albrecht (11:54:20): > PS: I’ll merge in the commits from theheatmap_setbranch intomasterwhenever we get more information form Bioc. At that point, I think i’ll just have to set an extra remote to the bioc-git server, and request write access to push the new versions. That’ll take care of version 1.0.1 and testing the write access
Charlotte Soneson (11:56:05): > Ok, writing team then: shall we continue with the google doc that Kevin started some time ago? Do you have the possibility for a Skype call to coordinate in the coming days?
Kevin Rue-Albrecht (11:57:21): > @Kevin Rue-Albrecht pinned a message to this channel.
Kevin Rue-Albrecht (11:57:52): > Just found the GoogleDocs link again. I’ve pinned it to the channel
Kevin Rue-Albrecht (11:59:57): > For Skype, Thursday afternoon of Friday still look ok this week.
Kevin Rue-Albrecht (12:00:17): > (if anyone prefers weekend, I can arrange that too)
Charlotte Soneson (12:00:59): > Thursday or Friday afternoon are fine with me (or weekend for that matter)
Aaron Lun (12:38:46): > I can do anytime. My computer is getting replaced tomorrow, so I will be twiddling my thumbs (i.e., trying to work on a Mac) for the rest of the week.
Kevin Rue-Albrecht (13:18:40): > What’s our next challenge after getting a BioC package accepted without a single issue? > Getting the manuscript accepted without revision?:sweat_smile:
Aaron Lun (13:26:40): > That is difficult, but possible.
Aaron Lun (13:26:50): > I’ve had a couple of close ones.
Kevin Rue-Albrecht (13:29:51): > Otherwise, free tickets to Toronto this summer would do, to reward our good effort:wink:
Kevin Rue-Albrecht (13:48:58): > <!channel>: I assume everyone has an ssh key setup with BioC, right? I suggest that each of you pokes Martin back athttps://github.com/Bioconductor/Contributions/issues/678to obtain write access
Aaron Lun (13:49:19): > Well, in theory only one person should have write access.
Aaron Lun (13:49:30): > There was a discussion about this somewhere.
Kevin Rue-Albrecht (13:50:19): > as far as I was aware, the discussion was about the name/email to put in DESCRIPTION
Aaron Lun (13:51:16): > Yes, and to push as well.
Kevin Rue-Albrecht (13:53:37): > well, apparently we just learned that they seem to pick the person who opened the contribution issue, as the first one set up with SSH key access to the repo, irrespective of who’s listed in the DESCRIPTION file, or who the GitHub repo belongs to, for that matter:sweat_smile:
Aaron Lun (13:54:07): > Congratulations, Kevin - or should I say, Charlotte?
Kevin Rue-Albrecht (13:54:45): > … considering that we eventually settled on Federico for the DESCRITPION file:laughing:
Kevin Rue-Albrecht (13:55:16): > (I have the commit on theheatmap_setbranch in GitHub)
Kevin Rue-Albrecht (13:55:47): > anyway, bottom line has always been that it didn’t really matter anyway: we’re all watching the tag of the support website
Kevin Rue-Albrecht (15:55:54): > btw, instructions athttps://bioconductor.org/developers/how-to/git/sync-existing-repositories/worked flawlessly for me (pull-only): > > [...] > $git merge upstream/master > Already up-to-date. > > Although as you can see there wasn’t anything to update yet
2018-03-21
Charlotte Soneson (05:24:01): > @Federico Marini, would you also be available for a skype call to discuss the paper plan Thursday or Friday afternoon?
Federico Marini (05:24:37): > Friday is not good, I am teaching a full-day course
Federico Marini (05:24:42): > but Thu is quite food
Federico Marini (05:24:44): > good
Federico Marini (05:25:07): > morning or afternoon I can be flexible
Charlotte Soneson (05:26:29): > Kevin seemed to prefer afternoon on Thu, so let’s try for that. Anyone has any particular restrictions on the time?
Federico Marini (05:28:15): > not really. But we can stick to our usual 4pm swiss/german if it suits for everyone
Charlotte Soneson (05:28:47): > Fine with me
Kevin Rue-Albrecht (05:32:52): > Hi both. Yeah, I’m booked the morning at least 10-11am (UK). So that would leave 9-9:45am or 11:15-12pm (UK) if you prefer the morning
Kevin Rue-Albrecht (05:33:11): > Afternoon looks clear to me >1pm (UK)
Kevin Rue-Albrecht (05:33:55): > The usual 4pm (3pm UK) sounds good to me
Kevin Rue-Albrecht (05:34:47): > separately, I was just thinking last night about package development from now on, considering that we have to pull fromupstream/master(BioC)
Kevin Rue-Albrecht (05:35:31): > We can’t followexactlythe procedure athttps://bioconductor.org/developers/how-to/git/sync-existing-repositories/“Make sure you are on the master branch.”git checkout master
Kevin Rue-Albrecht (05:35:39): > considering that we can’t push to our master
Kevin Rue-Albrecht (05:38:03): > or actually, we (each of us) can probably merge the bioc master to our respective local mastergit merge upstream/master masterbut then, we need to merge our local master to the working side branch that we’re developing the featuregit merge master side_branchthat way, we can open a PR fromside_branchtomasteron our GitHub, that I can then push to bioc
Kevin Rue-Albrecht (05:39:51): > makes sense? > Anyway, when the package shows up in the devel branch, I’ll try this out with theheatmap_setbranch
Aaron Lun (07:09:28): > I don’t see the problem.
Aaron Lun (07:09:52): > Once you have everything on master, you can push to the upstream.
Kevin Rue-Albrecht (07:14:46): > yep, maybe it’s easier than I imagine, I was just wondering whether our ‘protected’ master branch can complicate things as we can’t push to it, but if we only pull and merge, then we’re fine
Aaron Lun (07:25:46): > Am struggling to make a ggplot again. What’s the simplest command to use if I have x and y axes?
Aaron Lun (07:27:39): > Hold on, got it.
Aaron Lun (07:29:01): > here’s a taste: 4-5-fold speed-up: > > subsetGrid <- function(X, Y, resolution=200) { > # Avoid integer overflow when computing ids. > resolution <- min(resolution, sqrt(.Machine$integer.max)) > resolution <- as.integer(resolution) > > # X and Y MUST be numeric, i.e. use jitteredX instead. > rangeX <- range(X) > rangeY <- range(Y) > > binX <- (rangeX[2] - rangeX[1])/resolution > xid <- (X - rangeX[1])/binX > > binY <- (rangeY[2] - rangeY[1])/resolution > yid <- (Y - rangeY[1])/binY > > # Getting unique IDs, provided resolution^2 < .Machine$integer.max > # We use fromLast=TRUE as the last points get plotted on top. > id <- floor(xid) + floor(yid) * resolution > !duplicated(id, fromLast=TRUE) > } > > library(ggplot2) > N <- 1e5 > X <- runif(N) > Y <- X + runif(N)/5 > plot.data <- data.frame(X, Y) > > png("whee.png") > out <- ggplot(plot.data, aes(x=X, y=Y)) + geom_point() > system.time(print(out)) > dev.off() > > keep <- subsetGrid(X, Y) > sub.data <- plot.data[keep,] > png("whee2.png") > out <- ggplot(sub.data, aes(x=X, y=Y)) + geom_point() > system.time(print(out)) > dev.off() >
Aaron Lun (09:28:44): > Okay, my computer replacement is very imminent.
Aaron Lun (09:30:14): > IT COMES
Aaron Lun (09:31:11): > okay, 50% of the way there for downsampling. Will pick up the rest after I get my new computer.
Federico Marini (09:31:59): > have fun on your new MAC:smile:
Aaron Lun (09:32:18): > NOOOO
Aaron Lun (09:39:19): > They tried to give me a WINDOWS machine.
Aaron Lun (09:39:26): > I told them, no.
Federico Marini (09:44:15): > you can still install ubuntu via app:stuck_out_tongue:
Aaron Lun (09:57:02): > The thought makes me:face_with_open_mouth_vomiting:
Aaron Lun (10:37:17): > Downsampling is implemented on thespeedupbranch. Click on “visual parameters”, check “points”, and enable “Downsample for speed”. The resolution indicates how finely you want the downsampling to be done, with higher values => more accuracy. See also?subsetPointsByGrid, which is the function that does all the work. Can someone with a heavy dataset (>20000 points) give it a run and see how well it improves responsiveness?
Aaron Lun (13:33:05): > Talk to me people.
Kevin Rue-Albrecht (13:37:38): > 42
Kevin Rue-Albrecht (13:40:17): > Has anyone tried theheatmap_setbranch? I’m not fully convinced by the current behaviour: > At the moment, it’s behaving as Aaron described it: a “clear features” button that clears the selectize field as well as the memory, so that the next “import features” is ‘clean’ (as in, it doesn’t include any of the features prior to clicking the “clear features” button
Kevin Rue-Albrecht (13:41:54): > However, this behaviour currently creates a temporary seemingly inconsistent state, IMHO: between the “Clear Features” and the following “Import Features”, the heat map is not updated (it couldn’t, it would have to show 0 features)
Kevin Rue-Albrecht (13:43:14): > Which is why my initial implementation (commit 1c009762f5260314c8a8d622c738f5ab138c3c95) instead had a “Set features”, which basically did both at the same time: 1) clear current features, 2) import current selection of features
Kevin Rue-Albrecht (13:44:21): > which seemed more intuitive (to me), as it does not create that ‘inconsistent’ state where the selectize is empty but the heat map still shows the previous selection
Aaron Lun (13:51:43): > I think it’s fine.
Aaron Lun (13:52:10): > Clearing does its job, and we can also FORCE a replotting by incrementing the plotting counter.
Kevin Rue-Albrecht (13:55:59): > Oh right I thought about that yesterday. Do we know what the heat map function would do, if forced to updated with 0 features?
Kevin Rue-Albrecht (13:56:36): > Otherwise I wrote this function recently: > > ggplot(data = data.frame(x=0, y=0)) + > geom_point(aes_string(x = "x", y = "y"), alpha = 0) + > geom_text(x = 0, y = 0, label = "No data", size = rel(10)) + > theme_void() >
Aaron Lun (13:57:12): > It would probably shit itself
Aaron Lun (13:57:16): > So add a validate()
Kevin Rue-Albrecht (13:57:19): > Think I’m gonna stick it in the.onLoadcall of all my future packages:stuck_out_tongue:
Aaron Lun (13:58:23): > lol
Aaron Lun (14:37:33): > come on, give me some news
Kevin Rue-Albrecht (14:50:05): > haven’t had the chance to look yet, at the daily build
Kevin Rue-Albrecht (14:50:30): > here we go:http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/
Kevin Rue-Albrecht (14:51:07): > all lights are green
Kevin Rue-Albrecht (14:53:23): > about the validate that you mentioned above; we may have misunderstood each other: I don’t see the point adding neither a force nor a validate, considering that 1) we don’t want to force re-drawing a heat map right after clear, given that there are no features to show, 2) even if we did force redrawing, the following validate would block it anyway, resulting in the current behaviour (clearing without redrawing)
Kevin Rue-Albrecht (14:53:31): > so I’m not sure what your point was
Kevin Rue-Albrecht (14:54:29) (in thread): > saving this one as a thread, to get on it ASAP
Kevin Rue-Albrecht (14:54:50) (in thread): > with a little @channel poke
Aaron Lun (14:57:22): > No.
Aaron Lun (14:57:28): > Force redrawing from the clear features observer, in addition to clearing the observers.
Aaron Lun (14:58:10): > In the heatmap, there is avalidateanyway.
Aaron Lun (14:58:26): > So the clear will have an instant visual effect, where the heatmap disappears and is replaced with the validation message.
Aaron Lun (14:59:14) (in thread): > Don’t think channel pokes work in threads.
Aaron Lun (14:59:22) (in thread): > In any case, I will wait for some confirmation before merging.
Kevin Rue-Albrecht (15:00:35): > ah, right
Kevin Rue-Albrecht (15:00:54): > i’ll look at it in more details
Kevin Rue-Albrecht (15:44:47): > If anyone want to throw in some ideas, I’ve started listing some bullets points in the Google Docs. Here is the link again:https://docs.google.com/document/d/1MW3MZQzuxkUBm795o-Rmm1to7VS5lx6FRT6xvhdK8RM/edit#
Kevin Rue-Albrecht (15:44:47): > @Kevin Rue-Albrechtshared a file:iSEE_manuscript_draft - File (Google Docs): iSEE_manuscript_draft
Kevin Rue-Albrecht (15:44:57): > @Kevin Rue-Albrecht pinned ’s Google Docs iSEE_manuscript_draft to this channel.
Aaron Lun (18:20:03): > The question is whether we want it as an application note or a research article.
Kevin Rue-Albrecht (18:21:06): > TBH, I can’t really picture how to spin it as a research article
Aaron Lun (18:21:13): > 1000 words… hm.
Aaron Lun (18:21:18): > That’s like two pages and a figure.
Aaron Lun (18:22:01): > Which is doable.
Kevin Rue-Albrecht (18:22:22): > you just reminded me that I collected some notes about application note here:https://docs.google.com/document/d/1WaiqAeYYg9bv5JK-lAGr8KhU3VezUKV7OWnr70wHG0c/edit#heading=h.iaqpu3ni1f3i
Kevin Rue-Albrecht (18:22:23): > @Kevin Rue-Albrechtshared a file:iSEE to Oxford Bioinformatics - File (Google Docs): iSEE to Oxford Bioinformatics
Aaron Lun (18:22:42): > Okay, let’s crank out some text.
Kevin Rue-Albrecht (18:24:57): > @Kevin Rue-Albrecht pinned ’s Google Docs iSEE to Oxford Bioinformatics to this channel.
2018-03-22
Kevin Rue-Albrecht (07:25:37): > are we on for Skype @3pm (UK) / @4pm (Europe) ?
Aaron Lun (07:32:07): > What, today?
Federico Marini (07:34:42): > yep?
Aaron Lun (07:36:56): > Huh?
Aaron Lun (07:37:06): > I thought this was friday
Kevin Rue-Albrecht (07:37:14): > Friday doesn’t work for Fede
Aaron Lun (07:59:16): > Oky, well, I’ll probably be late. I have a meeting with the J-bone at 2;30.
Aaron Lun (07:59:26): > Also, I didn’t bring my laptop, so I’ll just text in.
Aaron Lun (08:03:06): > And listen to you guys, like a stalker on the phone.
Aaron Lun (08:03:11): > I can breathe heavily as well.
Federico Marini (08:13:08): > can’t wait for the conversation to happen:smile:
Kevin Rue-Albrecht (08:14:34): > Same here. Has anyone tried to edit the Google doc that I sent links? I realised last night that they were read-only, and made them editable (I think)
Aaron Lun (08:49:55): > Yes, I was just writing the introduction.
Kevin Rue-Albrecht (11:02:49): > just finding a spot to take the call
Federico Marini (12:29:15): > I have to confess I had to LOL a couple of time reading back the transcript of Aaron’s convo
Federico Marini (12:29:59): > especially > > Yep, something like that. > > yep > > YES. > > sigh >
Federico Marini (12:30:06): > :smile:
Federico Marini (12:40:37): > alternative to GB for a somewhat lnger format: NAR
Federico Marini (12:41:17): > with as an example the metaviz paper (wagner….corrada bravo) - which somehow follows the structure of how we put the vignette.
Federico Marini (12:42:00): > but anyway, I’m almost well convinced this GB format is kind of the best for displaying all the things we have - even the extra section for ECM would fall naturally in it
Charlotte Soneson (12:52:39): > Ok, I started a cover letter. I really need to get to the train now, so feel free to continue:https://docs.google.com/document/d/1l5FSAYzgJqlPZLImG-MJcXzJV5tAQPCPnc_l9FPa9f8/edit?usp=sharing
Aaron Lun (15:03:33): > I am… absolutely spent today, I will deal with this tomorrow.
Federico Marini (15:05:07): > I went to the CL now, and it is already well thought, good job on that
Federico Marini (15:05:47): > I am switching the url to the compact form
Federico Marini (15:07:32): > + adding somewhere we can deal seamlessly with scRNA-seq
Kevin Rue-Albrecht (15:35:35): > Cool. I can feel the weight of the day on me too right now. I’ll try to look at the letter after dinner if I can throw in a few ideas.
Aaron Lun (16:02:59): > Okay, I am actually done for the day. See the updated CL.
Aaron Lun (17:06:30): > I HAD ANOTHER VISION.
Aaron Lun (17:06:55): > @Federico MariniIs it possible for arbitrary rintrojs tours to be supplied at the command line?
Aaron Lun (17:07:26): > Then we can set up a number of apps for each of three applications - TCGA, a single-cell data set, and mass cytometry - and walk the audience over how to interact with them to get particular biological insights.
Aaron Lun (17:09:24): > I imagine that this would absolutely blow the reviewers away.
Aaron Lun (17:34:13): > Interesting package from the bowels of history:https://www.bioconductor.org/packages/release/bioc/html/interactiveDisplay.html - Attachment (Bioconductor): interactiveDisplay > The interactiveDisplay package contains the methods needed to generate interactive Shiny based display methods for Bioconductor objects.
Aaron Lun (17:34:27): > Does everything except SummarizedExperiment objects (at least to any real depth).
Federico Marini (17:34:58): > we can have as many tours as we want actually
Federico Marini (17:35:57): > basically we just would need to script different data frames
Federico Marini (17:36:43): > ideally you can have 3 buttons, each starts its tour - or we have 3 instances: 1dataset-1tour
Aaron Lun (17:40:01): > The key question is whether a user can supply the tour viaiSEE(), rather than it being caked into the package.
Aaron Lun (17:40:23): > Or specifically, whether a server can be set up to run custom tours.
Aaron Lun (17:40:38): > That, I think, would be amazing for a lot of people wanting to showcase their data.
Federico Marini (17:42:26): > well the tour ingredient is a data frame. we read in it and that is kicked off by the actionButton
Aaron Lun (17:42:52): > Is it possible to have it start upon app initialization?
Federico Marini (17:43:21): > https://github.com/csoneson/iSEE/blob/b6ca2475ec633fa62cca54b572f8de92c9ae50ad/R/iSEE-main.R#L307-L317 - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Federico Marini (17:43:28): > I would say so
Federico Marini (17:47:31): > lemme try
Federico Marini (17:50:25): > being
Federico Marini (17:50:57): > > intro_firststeps <- read.delim( > system.file("extdata", "intro_firststeps.txt",package = "iSEE"), > sep=";", stringsAsFactors = FALSE,row.names = NULL) >
Federico Marini (17:50:58): > and
Federico Marini (17:51:24): > > minitour <- intro_firststeps[c(1,10,20,30),] >
Federico Marini (17:52:00): > check branchtravel_agency
Federico Marini (17:52:08): > and launch after the example
Federico Marini (17:52:32): > iSEE(sce)andiSEE(sce,intro_firststeps = minitour)
Federico Marini (17:53:36): > there would be not that much of a validation possibility, so users could use it at own risk. but indeed they could use it, and that was what you were hoping:slightly_smiling_face:
Federico Marini (17:57:25): > I can refine & document tomorrow or in the next days. I am also quite with no fuel left today
Federico Marini (17:57:55): > but the proof of principle was so close to get done right away:rocket:
Federico Marini (17:59:36): > we need also a battle name, and with 4 it comes to my mind the 4 horsemen
Federico Marini (17:59:48): > and the 3 musketeers + d’artagnan
Federico Marini (17:59:57): > of course I am open to better fits:smile:
Federico Marini (18:00:18): > ok, I’m tiredandfreewheeling
Federico Marini (18:00:33): > I (would) need a drink
Kevin Rue-Albrecht (18:01:01): > i thought a while back that I should have had a more French name like Serge or Stephane.. our initials could haveactuallymade FACS
Kevin Rue-Albrecht (18:02:00) (in thread): > well.. we got KFC and Aaron.. does that count?
Kevin Rue-Albrecht (18:02:45) (in thread): > or KFCA on the sound of YMCA..
Kevin Rue-Albrecht (18:03:05) (in thread): > (now I’m free wheeling:sweat_smile:)
Federico Marini (18:06:40): > and the current status quo makes it almost blasphemic:stuck_out_tongue:
Federico Marini (18:07:37) (in thread): > I would be Fried, my dear Kentucky
Aaron Lun (18:15:44): > lol
Federico Marini (18:18:25): > anyway. Aaron, is it as you pictured it?
Aaron Lun (18:19:09): > can’t open it on my home computer, so it’ll have to wait till I get back to work.
Aaron Lun (18:19:27): > But I assume it does as it says on the tin
Aaron Lun (18:19:36): > Supply a dataframe toiSEE()and you’re away.
Federico Marini (18:20:49): > esatto:slightly_smiling_face:
Kevin Rue-Albrecht (19:03:51): > I made a couple of small tweaks here and there. But damn that CL looked great already
Aaron Lun (19:25:50): > Dammit, this stupid rash is back.
2018-03-23
Aaron Lun (06:00:49): > Is everyone happy with the CL? I added a bit on the stories.
Kevin Rue-Albrecht (06:01:16): > will look at it again. waiting for aparSapplyto complete -_-
Kevin Rue-Albrecht (06:41:26): > Only for the sake of shortening it a bit: isn’t that phrase a bit superfluous; “The process of data exploration almost always involves a visual component whereby plots are created for key metrics of interest (e.g., gene expression, experimental metadata, results of dimensionality reduction) to facilitate rapid understanding of various aspects of the data.”
Kevin Rue-Albrecht (06:55:48): > Alright. From the discussion on the document, I think all we need are a few references to existing tools to accompany the statement: “” > and then sign the letter and send. That could happens today, except if we any reason to give ourselves the weekend to refine anything (e.g., mergingspeedup?).
Kevin Rue-Albrecht (06:56:47): > how’s this one for a ref?http://bioconductor.org/packages/release/bioc/vignettes/pRolocGUI/inst/doc/pRolocGUI.html
Kevin Rue-Albrecht (07:00:24) (in thread): > and@Aaron Lun’s suggestion, which I guess is a big closer to our generic approachhttps://www.bioconductor.org/packages/release/bioc/vignettes/interactiveDisplay/inst/doc/interactiveDisplay.pdf
Kevin Rue-Albrecht (07:01:24) (in thread): > http://www.bioconductor.org/packages/3.7/bioc/vignettes/epivizr/inst/doc/IntroToEpivizr.html
Kevin Rue-Albrecht (07:01:36) (in thread): > https://academic.oup.com/bioinformatics/article/33/7/1081/2838384
Kevin Rue-Albrecht (07:01:44) (in thread): > https://rdrr.io/bioc/shinyMethyl/ - Attachment (rdrr.io): shinyMethyl: Interactive visualization for Illumina methylation arrays version 1.14.0 from Bioconductor > Interactive tool for visualizing Illumina methylation array data. Both the 450k and EPIC array are supported.
Aaron Lun (07:05:24) (in thread): > Not quite what I wanted, unfortunately; I wanted the intro to start immediately on app load.
Aaron Lun (07:05:33) (in thread): > Currently people still have to click the button.
Aaron Lun (07:06:13) (in thread): > I’ve tried to get it to start immediately, but it’s hard to get the timing right; the UI elements aren’t set up at the start, so the intro ends up pointing to nothing.
Aaron Lun (07:14:40) (in thread): > Now fixed.
Aaron Lun (07:30:42): > Just so we’re clear on all this: the author order will be KRA, FM, CS and ATLL. Everyone will be joint first author; I will be corresponding. This is the understanding that I had; are we all on the same page?
Kevin Rue-Albrecht (07:33:13): > I’m happy with this.
Charlotte Soneson (07:36:56): > Fine with me
Aaron Lun (09:36:54): > Not sure I’d use the word “unified” to describe our package. What exactly are we unifying?
Kevin Rue-Albrecht (09:38:33): > oh right, i picked up on that too, I’m sure what is meant there either. Maybe “one UI to rule all omics” kinda unified?
Aaron Lun (09:39:37): > I changed it to “general cross-platform”
Charlotte Soneson (09:39:51): > I guess what I meant was that we can visualize any type of numeric data with the same type of UI, not a specific “MA-plot function” and another specific “volcano plot function”
Charlotte Soneson (09:40:06): > and so on…
Aaron Lun (09:43:50): > That meaning is probably a bit too subtle without further explanation.
Aaron Lun (09:49:46) (in thread): > Are any of these in GB?
Aaron Lun (09:50:55) (in thread): > Googled Genome Biology shiny…
Aaron Lun (09:50:58) (in thread): > Got this:https://doi.org/10.1186/s13059-015-0862-3
Aaron Lun (09:51:06) (in thread): > Dunno… first author seems a bit dodgy…
Kevin Rue-Albrecht (09:51:54) (in thread): > pRolocGUI (rather, pRoloc): Bioinformatics:http://computationalproteomicsunit.github.io/pRolocGUI/
Aaron Lun (09:52:15) (in thread): > Yeah, I know.
Kevin Rue-Albrecht (09:52:43) (in thread): > Ah. I was about to go through each of them. Did you already?
Aaron Lun (10:04:54): > Okay, everyone have a read over the CL before I paste it into UCam’s offical letter format.
Charlotte Soneson (10:14:13): > Looks good to me, just added one small comment
Aaron Lun (10:16:15): > BTW, if you do want to change anything, make some noise, because I’ll be editing my internal doc from now on.
Aaron Lun (10:16:28): > I’m not going to send it around to everyone, mostly because it’s got my signature on it.
Kevin Rue-Albrecht (10:17:19): > Sure. I’m just sending an urgent email and then I’ll have a final look
Aaron Lun (10:18:52): > @Charlotte SonesonLUkas’s dataset is over a million cells, right?
Charlotte Soneson (10:19:20): > I am not sure. I think he said that the original was 800,000 or something like that. I’ll ask.
Charlotte Soneson (10:20:53): > Is that the one you meant for the mass cytometry case study?
Aaron Lun (10:22:50): > yeah
Aaron Lun (10:23:27) (in thread): > I’m not sure we need to add them… the cover letter is already getting a bit big.
Aaron Lun (10:23:37) (in thread): > If they were in GB, it’d be a different story.
Kevin Rue-Albrecht (10:25:28) (in thread): > Thought the letter was getting a bit long too. Just felt awkward to make the statement without reference. Probably not a big deal for an initial enquiry though. If anything it might trigger a discussion with the editors on the subject
Aaron Lun (10:25:39) (in thread): > I mean, if we cite all theseBioinformaticspapers, the question would be why we don’t just publish in Bioinformatics.
Aaron Lun (10:26:47): > Do we need the Bioconductor web link?
Kevin Rue-Albrecht (10:27:47): > mhh.. it kinda states that the packageisavailable already
Kevin Rue-Albrecht (10:28:10): > in a CL, we can’t really have it as a hyperlink on theiSEEword, can we?
Aaron Lun (10:28:43): > I’m sure we could, but the demo page is far more relevant.
Kevin Rue-Albrecht (10:29:17): > “whether … would be suitable for publication” -> “is suitable …” ?
Aaron Lun (10:29:32): > Probably fine as it is.
Aaron Lun (10:30:06): > Anyway, I’m getting rid of the Bioc hyperlink. If there’s any single link that we want them to use, it’s the one to the server.
Aaron Lun (10:30:55): > … and adding the GB citation for Bioconductor.
Aaron Lun (10:32:02): > Yes, that looks better.
Aaron Lun (10:34:22): > How many TCGA samples are there?
Kevin Rue-Albrecht (10:34:53): > oh i know i know i know professor! 67.. something hang on
Kevin Rue-Albrecht (10:36:02): > 7706 samples
Kevin Rue-Albrecht (10:36:09): > (23,368 features)
Aaron Lun (10:38:06): > Okay, I think it looks pretty tight now.
Kevin Rue-Albrecht (10:39:24): > “high-throughput biological assays” -> “genome-wide” or “large scale” ?
Aaron Lun (10:39:42): > No, because mass cytometry is not genome wide
Aaron Lun (10:39:51): > And large scale … is pretty much the same as high-throughput.
Aaron Lun (10:41:26): > Okay, I’m wandering off from my computer, will be back in a bit.
Kevin Rue-Albrecht (10:42:10): > ok. just going through the letter on Google Drive, but at this point it looks so good anyway, that all I can offer are those nagging details that I call suggestions
Kevin Rue-Albrecht (10:47:04): > “and feature assay (e.g., expression) values”: > -> “and assay[ed] features values (e.g. expression)” ? > it reads a bit odd to have the “e.g.” in the middle
Kevin Rue-Albrecht (10:48:16): > I thought of “quantitated assay features”, but I think we previously discussed that qualitative values should be possible as well (right?)
Aaron Lun (10:48:42): > yeah, okay, change that
Kevin Rue-Albrecht (10:49:45): > I like “observations”
Aaron Lun (10:50:08): > Just changed it to “experimental observations”
Aaron Lun (10:50:18): > Might as well cut to the chase
Aaron Lun (10:50:39): > hm.
Aaron Lun (10:50:41): > actually.
Kevin Rue-Albrecht (10:50:53): > I liked feature-level too, but I also just realised that it’s already in the first half of the phrase
Kevin Rue-Albrecht (10:52:38): > looking good now:slightly_smiling_face:
Kevin Rue-Albrecht (10:52:51): > (the letter, too)
Charlotte Soneson (10:56:59) (in thread): > There are 172,791 cells in total in the CyTOF data set we subsampled from for the vignette
Charlotte Soneson (10:57:05) (in thread): > But of course there are bigger ones
Federico Marini (11:48:43) (in thread): > same, fine for me
Federico Marini (11:51:47) (in thread): > :wink:
Federico Marini (12:01:20): > if we want to cite something from GB itself, we should do it with SCANPY
Federico Marini (12:01:31): > it would have a similar format as ours too
Federico Marini (12:01:45): > and would highlight the focus GB has onbig data
Aaron Lun (12:30:30): > Not sure how that would fit in. We’d have to have at least another section talking about big data but that’s not really a focus here. Also, we’re not Python, so we can’t claim to have “seamless integration with existing tools like SCANPY”, which is what I would normally say.
Aaron Lun (12:36:58): > Well, unless anyone else has anything to say, I’m sending it off.
Aaron Lun (12:37:27): > … just going to run it through John first.
Kevin Rue-Albrecht (12:45:24): > isn’t “[multi-panel layout that is]fully[customizable]” a bit overstated/vague in terms of what “fully” is meant ?
Kevin Rue-Albrecht (12:45:48): > -> “in an intuitive, customizable, multi-panel layout”
Kevin Rue-Albrecht (12:46:37): > apart from that, I’d say we’re good to go
Aaron Lun (12:46:45): > You know, you’re lucky I couldn’t find John.
Kevin Rue-Albrecht (12:48:46): > Well, again, I wouldn’t have died of shame if you sent the letter as is. I just read it again when you just poked us
Aaron Lun (12:54:22): > Anyway, that is changed.
Kevin Rue-Albrecht (13:06:33): > cool. I look forward to the editors’ reply now:slightly_smiling_face:
Aaron Lun (13:13:14): > Still need to find that scot
Federico Marini (13:16:02) (in thread): > I would have just referred to that as “Your journal published recently [scanpy], another tool which offers a number of capabilities to explore and plot…”
Federico Marini (13:16:40) (in thread): > more than the integration, I’d stress the fact we are a strong R/Bioc counterpart
Federico Marini (13:17:28): > I would suggest in sending to someone specific and not just the editorial team
Federico Marini (13:18:04): > good fits in term of interest could be Sarah Teichmann and the couple of cam/oxford-based ones
Aaron Lun (13:18:49): > Do we really know someone well enough to do that?
Federico Marini (13:19:03): > I do not, sadly:disappointed:
Kevin Rue-Albrecht (13:46:19): > I can’t say I do. > I’ll just point out that Chris Ponting is on a consortium with John and Steve, and that I’ve heard Fiona Powrie (head of the Kennedy institute where I work) is on project(s) with Sarah Teichmann
Charlotte Soneson (13:48:38): > I’ve interacted in various ways with some of them, but I guess it is a matter of definition how well you “know” someone. Alicia Oshlack may be interested in this kind of things?
Aaron Lun (14:23:00): > I think she doesn’t likes me particularly much.
Aaron Lun (14:26:21): > And I have a feeling that the editorial team makes the desk rejection decisions, not the board.
Aaron Lun (14:26:53): > Looking back at my last GB paper, it was handled by Barbara (chief editor).
Aaron Lun (14:27:24): > So I say we just submit to the editorial address and let the chips fall where they may.
Aaron Lun (14:27:45): > Unless any of you know people on the editorialteam?
Kevin Rue-Albrecht (14:39:04): > Nope. I do not:sweat_smile:
Charlotte Soneson (15:25:11): > Yeah, me neither. We have interacted with Andrew Cosgrove for our previous papers.
Aaron Lun (16:47:12): > No more comments? I’m going to send it off.
Aaron Lun (16:47:45): > Can everyone test out the downsampling over the WE, and if it’s okay, merge it in and get@Federico Marinito update the version on the server?
Aaron Lun (16:48:00): > I don’t think the eds would go that far, but best to cover our bases.
Aaron Lun (16:53:21): > It is done.
Aaron Lun (16:53:39): > In the meantime, we should start prepping a MS, in the unlikely case that we get a “yes”.
Kevin Rue-Albrecht (18:18:30): > Agreed
Kevin Rue-Albrecht (19:20:44): > just testing now (although I’m gonna go to sleep soon), thespeeduplooks amazing (i.e. lighting fast). I’m just surprised I got aelement_blankmissing import error, but I’ll look at that with a fresh mind tomorrow
2018-03-24
Kevin Rue-Albrecht (07:01:35) (in thread): > I need to go out for a few hours, but I want to check it out again this afternoon. > One question that popped to mind this morning, is whether the downsampling behaves only visually (expected), or whether the ‘hidden’ downsampled points are lost from subsequent point selections (from the downsampled plot to a child)
Kevin Rue-Albrecht (07:09:52): > also ggplot2unitimport is missing after fixing the above one. Anyway, as said I’ll look at this again later today. > I really do like the speedup. However, see thread for a question
Kevin Rue-Albrecht (07:11:16) (in thread): > in other words: do point selections also capture ‘hidden’ data points? (while excluding those ‘restricted’ by a parent brush)
Kevin Rue-Albrecht (07:32:35) (in thread): > nevermind, I looked at the code tracker, and downsampling is only done for the visual panel. Coordinates of hidden points are stored in all.coordinates as expected
Kevin Rue-Albrecht (07:32:43) (in thread): > :thumbsup:
Aaron Lun (10:17:22) (in thread): > Yes… though see my comments in.plot_wrapper.
Kevin Rue-Albrecht (10:53:18) (in thread): > cool. I see it now.
Kevin Rue-Albrecht (10:54:51): > Due to the missing imports, i’ve mergedspeeduptoheatmap_set. I think we can merge the two in version 1.0.1. The only thing that I’m missing is how to trigger the replot of the heat map when “Clear selection” is clicked
Kevin Rue-Albrecht (11:03:53): > hang on. think I found it
Aaron Lun (11:07:37): > should be a case of just incremetning the counter.
Kevin Rue-Albrecht (11:08:57): > yup.. i spotted the counter for the heat map legend, the counter for the point based plots, and I was just putting 2 and 2 together
Kevin Rue-Albrecht (11:11:38): > I was going to point out that I’ve noticed a slow-down of the heat map, but I’m actually playing with the TCGA data set right now
Kevin Rue-Albrecht (11:12:22): > don’t know if we should explore down-sampling for the heat map as well
Aaron Lun (11:12:57): > Not sure how to do that.
Kevin Rue-Albrecht (11:13:07): > PS: it’s working: I get the validation message instead of the heat map when clearing the features
Aaron Lun (11:13:07): > As the bars do not cover other bars
Aaron Lun (11:13:12): > good
Kevin Rue-Albrecht (11:25:24): > mh.. just brainstorming here: > My initial reaction would have been to drop samples that are very similar to others, but that would change the relative size of clusters in the heat map, so no. > An alternative would be to aggregate/average nearby samples ‘evenly’: even merging sets of 2-3 neighbor samples would reduce the ‘sample’ resolution of the heat map by that same factor
Kevin Rue-Albrecht (11:26:39): > or something more brutal like kind of image compression, averaging nearby tiles of the heatmap. Not sure how the resulting heat map would look, though
Aaron Lun (11:28:06): > Yes, I think we would have to average the colours.
Aaron Lun (11:29:00): > Not as easy as just averaging RGB values, apparently.
Kevin Rue-Albrecht (11:29:37): > Anyway, I’ve added stuff to theNEWSfile on branchheatmap_set, which itself includes the updates ofspeedupmerged. > As far as I’m concerned, we could just added the speedup info into that NEWS file, and merge tomaster
Aaron Lun (11:34:42): > Very interesting post:https://www.youtube.com/watch?v=LKnqECcg6Gw - Attachment (YouTube): Computer Color is Broken
Kevin Rue-Albrecht (11:40:26): > I learned more in these few minutes than in the last three weeks of seminars:sweat_smile:
Kevin Rue-Albrecht (11:51:01): > Btw, I’m just adding the downsampling to theNEWS.Rd. > Looking at the other commits, I wonder whether we want to document things that are not necessarily visible at the user level, e.g. “Centralized plotting commands into a single wrapper function.”
Aaron Lun (11:52:59): > No.
Aaron Lun (11:53:12): > No point confusing people.
Kevin Rue-Albrecht (11:53:20): > Thought so. Too confusing (exactly)
Kevin Rue-Albrecht (12:08:44): > Alright, i’ve sent the PR in. Up to you@Aaron Lunif you want to mergespeedupseparately, or add anything toheatmap_set(that mergesspeeduptoo). I don’t have anything to add today, as far as I can see.
Kevin Rue-Albrecht (12:09:57): > Then when it’s all merged and I have the green light, I’ll push our updatedmasterto bioc-git too.
Aaron Lun (12:11:17): > Sounds good.
Kevin Rue-Albrecht (13:16:36): > <!channel>I’m all set for a push to bioc. Any last minute hand raised?
Kevin Rue-Albrecht (13:17:41): > (it can wait til tonight if people still want to play with the updated app a bit)
Charlotte Soneson (13:21:21): > No, not really. I was just trying out the subsampling with the full (170,000 cells) CyTOF data set, and it seems to work. Only one thing: when I click in the “points” box in the Visual parameters (without anything else changing), the plot updates, but maybe this is necessary.
Kevin Rue-Albrecht (13:36:47): > is it necessary? doesn’t sound like it should be (from a user perspective). I’ve tried it now, and I notice that the other checkboxes have the same effect as well (‘Color’ and ‘Other’)
Aaron Lun (14:10:18): > This is not necessary and shouldn’t happen.
Aaron Lun (14:12:22): > You can delete therObjects[[plot_name]] <- .increment_counter(isolate(rObjects[[plot_name]]))line on 1127
Kevin Rue-Albrecht (14:15:10): > Done. I’m building and testing now
Kevin Rue-Albrecht (14:17:30): > Looks fine to me. Pushing in a minute (PR etc.), and giving more time for everyone to play with the new version.
Aaron Lun (14:19:49): > Also, your BioC-devel link on the READMe is a bit… meh
Kevin Rue-Albrecht (14:20:24): > Yeah, not the biggest fan of the blue link: suggestion?
Aaron Lun (14:20:27): > I don’t think we need the “Bioc-devel” text
Aaron Lun (14:20:39): > Just have it in a separate sentence in the intro
Kevin Rue-Albrecht (14:21:18): > Ok. The text was because the badge itself doesn’t say that it’s bioc
Aaron Lun (14:21:30): > Don’t think people will even notice
Kevin Rue-Albrecht (14:21:34): > I didn’t care so much for making it a link]
Kevin Rue-Albrecht (14:21:53): > nah, it’s more for us to have a sort of dashboard of build statuses
Kevin Rue-Albrecht (14:22:02): > https://github.com/kevinrue/TVTB/blob/master/README.md - Attachment (GitHub): kevinrue/TVTB > TVTB - The VCF Tool Box
Aaron Lun (14:22:17): > That looks okay, actually
Aaron Lun (14:23:09): > Also, can someone mergemasterintotravel_agency?
Aaron Lun (14:23:35): > I think customizable tours is going to be another killer feature
Aaron Lun (14:23:42): > But we need to showcase it really well.
Kevin Rue-Albrecht (14:24:12): > We should poke@Federico Marinito see what else needs to go in there, but I guess I can mergemasterin there already
Kevin Rue-Albrecht (14:25:30): > alright then, I’ll move the README in table format then
Aaron Lun (14:27:31): > It needs documentation for the newtourargument. I’m not sure it needs any cov, though.
Aaron Lun (14:27:41): > Because the new things are mostly server-side.
Kevin Rue-Albrecht (14:31:35): > about the README: I think I’ll drop the footnotes for iSEE
Aaron Lun (14:35:41): > We also don’t need SingleCellExperiment anymore in the name.
Kevin Rue-Albrecht (14:36:44): > I can see it in the README. Anywhere else you spotted it?
Aaron Lun (14:36:55): > In the intro tour, I think.
Kevin Rue-Albrecht (14:37:44): > found it
Kevin Rue-Albrecht (14:38:11): > argh.. there are more than one in there: “in exploring SummarizedExperiment or SingleCellExperiment objects”
Kevin Rue-Albrecht (14:38:21): > “objects derived from SummarizedExperiment” ?
Kevin Rue-Albrecht (14:38:47): > I feel misleading to leave only “exploring SummarizedExperiment objects”
Aaron Lun (14:39:32): > It’s fine.
Aaron Lun (14:39:37): > SCE is an SE object.
Kevin Rue-Albrecht (14:40:00): > Right.isis always right
Aaron Lun (15:13:17): > YEAH, MERGE IT IN
Aaron Lun (17:02:43): > I was planning to start writing up tomorrow, but I have another paper on my lap, so I will push it back to Tuesday when I’m at the EBI (and hence on my Mac, and away from a real computer that I can do work on).
Kevin Rue-Albrecht (18:06:10): > Alright. We good. I’ve merged againmasterintotravel_agency. That’ll be good for 0.99.2
2018-03-25
Federico Marini (14:41:46): > From the animal farm “stealing” some other guy’s wifi…
Federico Marini (14:42:06): > Just let me know when I need to update the server
Federico Marini (14:42:43): > Sorry I could not do that much in polishing thetravel_agencyuntil now
Federico Marini (14:44:07): > I fetch & pull everything up to the current state
Federico Marini (14:46:25): > @Charlotte Soneson, would you mind adding a couple of tags to the repo itself in github?
Federico Marini (14:46:59): > we can pick from the BiocViews or customize them a little
Federico Marini (14:52:43): > @Aaron Lun, the current version of the tour becomes “undefined” when I launch it
Federico Marini (14:53:08): > as an example I took the default one, and pass it as a parameter
Federico Marini (15:00:45): > - or am I using it not the way it is supposed?
Aaron Lun (15:16:32): > How exactly are you doing this?
Aaron Lun (15:16:49): > And what do you mean by undefined?
Federico Marini (15:23:31): > building on the example, > > mytour <- read.delim(system.file("extdata", "intro_firststeps.txt",package = "iSEE")) > iSEE(sce,tour = mytour) >
Federico Marini (15:24:40): > @Federico Mariniuploaded a file:Screen Shot 2018-03-25 at 21.23.58.png - File (PNG): Screen Shot 2018-03-25 at 21.23.58.png
Kevin Rue-Albrecht (17:12:18): > unrelated: i got notified of a retweet with a comment that started with “Interesting @Bioconductor […]“, but when i opened it up the tweet was deleted:cry:
2018-03-26
Aaron Lun (04:52:01): > @Federico Marinisep=“;” would probably help.
Federico Marini (05:08:57): > OMG. sorry for the false alarm, holiday modeon
Federico Marini (05:10:11): > I’ll finish up documenting later today
Federico Marini (05:10:25): > then we can merge it in master, bump up, and update the server
Kevin Rue-Albrecht (05:13:53): > speaking of which, 0.99.1 should show up today, as I pushed it to bioc-devel Saturday night
Kevin Rue-Albrecht (05:15:36): > If igit clonehttps://git.bioconductor.org/packages/iSEE, I can see the version bump. So it’s up to their build and propagation mechanism now
Charlotte Soneson (05:17:42) (in thread): > I added a bunch of topics, according to the BiocViews in our DESCRIPTION file
Aaron Lun (05:26:07): > Just looking at MORPHEUS again. Who the hell came up with this?
Kevin Rue-Albrecht (05:27:46): > 2 contributors:https://github.com/cmap/morpheus.js/graphs/contributors - Attachment (GitHub): cmap/morpheus.js > morpheus.js - JavaScript matrix visualization and analysis
Kevin Rue-Albrecht (05:28:06): > or, looking at commits: 1.00001 contributor(s)
Aaron Lun (05:28:31): > Lol.
Aaron Lun (05:28:55): > I find the BROAD interface positively useless.
Kevin Rue-Albrecht (05:29:00): > (not sure I put enough 0s)
Aaron Lun (05:30:55): > Trying to find a citation for Moprheus; there’s at least 3 of them for diferent tools
Aaron Lun (08:01:12): > What are some other weaknesses of existing tools?
Kevin Rue-Albrecht (08:12:16): > I haven’t used that many interactive ones, tbh
Aaron Lun (08:12:51): > clicking. pfft.
Aaron Lun (08:12:58): > Only losers use a mouse.
Kevin Rue-Albrecht (08:14:12): > actually, that clicking part reminds me of ‘transparency’ (or rather the opposite), that I felt when using Ingenuity Pathways Analysis (IPA), and I’ve heard of Metacore
Kevin Rue-Albrecht (08:15:24): > we cannot emphasise enough the code tracker that offers full transparency
Kevin Rue-Albrecht (08:20:49): > https://www.ncbi.nlm.nih.gov/pmc/articles/PMC310693/(1998… unfair?:innocent:) - Attachment (PubMed Central (PMC)): BioViews: Java-Based Tools for Genomic Data Visualization > Visualization tools for bioinformatics ideally should provide universal access to the most current data in an interactive and intuitive graphical user interface. Since the introduction of Java, a language designed for distributed programming over the …
Kevin Rue-Albrecht (08:22:07): > … the hell is that?https://www.fiosgenomics.com/data-visualization-and-data-analysis/ - Attachment (Fios Genomics): Data Visualization and Biology - Fios Genomics > Why data visualization is so important for researchers? Using heat maps, volcano plots, bar charts is possible to turn data into meaningful information.
Kevin Rue-Albrecht (08:24:10) (in thread): > https://www.fiosgenomics.com/wp-content/uploads/2017/03/heatmap.png - Attachment: Attachment
Kevin Rue-Albrecht (08:24:50) (in thread): > They actually refer to “several algorithms”:https://www.nature.com/articles/nmeth.2540“Altmetric: 1 Citations: 1” - Attachment (Nature Methods): Bioinformatics: Visualizing high-dimensional data > Bioinformatics: Visualizing high-dimensional data
Kevin Rue-Albrecht (08:26:27) (in thread): > Actually, maybe we can mention viSNE:https://www.ncbi.nlm.nih.gov/pubmed/23685480 - Attachment (ncbi.nlm.nih.gov): viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia. - PubMed - NCBI > Nat Biotechnol. 2013 Jun;31(6):545-52. doi: 10.1038/nbt.2594. Epub 2013 May 19. Research Support, N.I.H., Extramural; Research Support, Non-U.S. Gov’t; Research Support, U.S. Gov’t, Non-P.H.S.
Kevin Rue-Albrecht (08:27:00) (in thread): > and/or learn something from them
Kevin Rue-Albrecht (08:33:07): > Found the cyt tool that interactively displays “viSNE” plots. First instruction to launch cyt: “Launch Matlab”
Aaron Lun (08:33:16): > lol
Kevin Rue-Albrecht (08:33:34): > To generate and visualize viSNE maps, download the associated matlab based tool, CYT, here (https://www.c2b2.columbia.edu/danapeerlab/html/cyt-download.html).
Kevin Rue-Albrecht (08:33:42): > oh no wait
Kevin Rue-Albrecht (08:33:57): > These instructions (https://www.c2b2.columbia.edu/danapeerlab/html/CYT/cyTutorial.ppt) explain how to run viSNE from within CYT.
Kevin Rue-Albrecht (08:34:12): > User manual = PPT presentation
Kevin Rue-Albrecht (08:34:41): > (and that’s a 2013 paper, not the 1998 that I mentioned earlier)
Kevin Rue-Albrecht (08:42:01): > What’s the story withepivizr?http://www.bioconductor.org/packages/3.7/bioc/vignettes/epivizr/inst/doc/IntroToEpivizr.html
Aaron Lun (08:42:24): > ¯*(ツ)*/¯
Aaron Lun (09:57:03): > And that is an application note written.
Aaron Lun (09:57:09): > Just need a figure and we’re done.
Aaron Lun (10:03:44): > Could probably use paperpile for refs.
Kevin Rue-Albrecht (10:04:10): > feel free; I don’t have experience with paperpile
Aaron Lun (10:04:56): > Depends on whether we want to bother with thtat.
Aaron Lun (10:05:01): > Because it’s all going to Latex anyway.
Charlotte Soneson (10:05:02): > I guess that depends on the format that we want to export it to (should work for Word, don’t know for LaTeX). Have we heard back from Genome Biology?
Aaron Lun (10:05:04): > No.
Kevin Rue-Albrecht (10:05:10): > In the absence of EndNote, I’ve always fallen back to PMID as text, that are converted to proper references at the last minute to avoid conflicts between reference managers
Aaron Lun (10:05:27): > GB will take the week to reply I think.
Aaron Lun (10:05:47): > And I will eat this nespresso capsule if they say yes.
Kevin Rue-Albrecht (10:06:06): > !!?:joy:
Aaron Lun (10:07:32): > OKAY NEW COMPUTER IS HERE I”M GOING OFFLINE
Kevin Rue-Albrecht (11:16:31): > FYI: version 0.99.1 is available:http://bioconductor.org/packages/devel/bioc/html/iSEE.html
Aaron Lun (11:19:43): > I’m back baby
Aaron Lun (11:19:51): > running on 16.04… yeah
hcorrada (14:08:35): > @hcorrada has joined the channel
Aaron Lun (14:37:26): > Talk to me, people. Who’s still here?
Kevin Rue-Albrecht (14:37:56): > Just got home. Will pick up the conversation later though
Kevin Rue-Albrecht (14:41:52): > Welcome@hcorrada. I actually just broughtepivizrin the conversation earlier. Feedback welcome, with respect to iSEE and interactive visualisation in general:slightly_smiling_face:
Jayaram Kancherla (14:55:00): > @Jayaram Kancherla has joined the channel
Aaron Lun (14:55:26): > Man, this new syntax highlighting on vim is great.
hcorrada (15:13:04): > Hi there, just started to look at iSEE as we were thinking about how to tackle single-cell things on epiviz(r).@Jayaram Kancherlais the lead developer and will be looking at this. Nice to meet you!
Aaron Lun (15:15:56): > I think there’s some nice complementarity here, because we don’t deal with genomic regions at all. In fact, I think one interesting opportunity for interplay would be through ourannotFunargument iniSEE(), which could conceivably call anepivizRapp running on a separate port (or something) to examine a genomic region that a user has selected iniSEE().
Aaron Lun (15:16:53): > We don’t do much with genomic coordinates other than displaying their literal values.
hcorrada (15:20:27): > Great! We’ve been working on making epivizr visualizations modular so we have flexibility to either start a separate app, or integrate coordinate-based data into an existing visualization
Aaron Lun (16:13:41): > Deprecatedscran::exploreData,scran::selectorPlotandscater::scater_gui, all of which now point toiSEE::iSEE.
Aaron Lun (16:13:49): > Not that I think anyone uses the first three anyway.
Aaron Lun (16:14:44): > Just noticed that BiocCheck is bitching aboutR (>= 3.6). WTF.
Kevin Rue-Albrecht (16:15:32): > Well.. even R-devel is still only 3.5 . Why require 3.6?
Aaron Lun (16:16:54): > Dunno. Anyway, I’m going home.
Federico Marini (16:47:38): > briefly from the farmlands:slightly_smiling_face:thetravel_agencybranch is now documented. Are we all set for bumping version on Bioc?
Kevin Rue-Albrecht (16:48:18): > You tell me:slightly_smiling_face:
Kevin Rue-Albrecht (16:49:57): > wanna add stuff in theNEWS?
Federico Marini (17:01:48): > I read up on the naked Rd format first:smile:
Federico Marini (17:02:37): > to be sure: do we just have the tour to add right?
Kevin Rue-Albrecht (17:06:52): > As far as I know. Wecouldmark the releases on GitHub to compare against.. but that would be just as overkill as it sounds ^^
Federico Marini (17:09:19): > maybe this we could do with the release changes
Kevin Rue-Albrecht (17:09:48): > that’s your diff to 0.99.1:https://github.com/csoneson/iSEE/compare/2626f21…travel_agency
Federico Marini (17:10:02): > once the rel is out, we can also think of a small extra maintainance to provide on top a conda package
Kevin Rue-Albrecht (17:11:17): > Note that for the diff, the way I did it is to fetch theLast Commit: 2626f21from the build report (http://bioconductor.org/checkResults/3.7/bioc-LATEST/iSEE/)
Kevin Rue-Albrecht (17:12:24): > then you just initiate a PR on GH between the commit of the new candidate release and that last commit (don’t go through with the PR)
2018-03-27
Aaron Lun (09:30:58): > comon people. Talk to me.
Kevin Rue-Albrecht (09:31:02): > Hi@Federico Marini. As far as I can see, it’s all ready for merge tomaster. I’ll leave it open til tonight if anyone has anything to throw in this new version (e.g. typos). Then I’ll send it upstream
Aaron Lun (09:33:57): > Looks pretty good to me.
Kevin Rue-Albrecht (09:35:38): > I’m head down in a pipeline (i.e. haven’t looked at the udpated code), but if you want to merge it in master, I can push upstream
Kevin Rue-Albrecht (09:36:48): > I can never remember when the Bioc build starts, did we miss it for today?
Aaron Lun (09:37:30): > Probably
Aaron Lun (09:39:26): > I haven’t heard back from GB, but I’m going to assume they’’ll knock us back, in which case I have some MS-related tasks for everyone. I will start working on the figure; everyone can pick a dataset they like and know well, and write up a little tour viaiSEE.
Kevin Rue-Albrecht (09:39:50): > Probably for a next release, but we could already write a custom tour for themodeGating, just occurred to me
Aaron Lun (10:20:15): > Anyway, just decide on a dataset in which you want to give a tour.
Aaron Lun (10:24:47): > OMG unit testingscranis so fucking boring
Kevin Rue-Albrecht (10:26:52): > can unit tests come with a progress bar? ^^
Aaron Lun (10:31:16): > The problem being that it’s going backwards.
Kevin Rue-Albrecht (13:03:47): > Dodgy paper we got for journal club tomorrow, something about “Bias, robustness and scalability in single-cell differential expression analysis”:face_with_raised_eyebrow:
Charlotte Soneson (13:05:58) (in thread): > Hmm…let me know what it’s about:wink:(or more seriously, I’d be happy to hear if you have any comments, although there’s not much I can do about the paper at this stage…)
Kevin Rue-Albrecht (13:08:25) (in thread): > Funny.. I was going to ask if you had any pointer to impress my colleagues by my expert understanding of the methods, but I should probably read it properly first ^^
Charlotte Soneson (13:11:20) (in thread): > I’d say most of the conclusions are included in Figure 5. And there’s a lot of variability between data sets (as can be seen from Figure 3). Personally I like Figure 2, showing the biases of the different methods.
Kevin Rue-Albrecht (13:23:35) (in thread): > Thanks!:grin:
Kevin Rue-Albrecht (14:36:37): > Alright, I’ll merge tomasternow. The individual tours will go in 0.99.3
Kevin Rue-Albrecht (14:52:47): > @Federico Marini: justactuallytesting the custom tour now. Does it feel clunky to have the user type the wholeread.delim(..., sep=";", stringsAsFactors = FALSE,row.names = NULL)? > What’s more important: > 1) that they respect the file input format > 2) that they provide a data.frame with two columns, ‘element’ and ‘intro’? > In the former case, we could ask them the file path, and we’d take care of the parsing. In the latter, that could give more freedom on the file format, as long as they supply an adequate data.frame
Aaron Lun (15:33:16): > The latter, definitely.
Aaron Lun (15:37:47): > All you need is~love~adata.frame, AFAIK
Kevin Rue-Albrecht (15:40:39): > Never used introjs before, that’s why I asked. Documentation isn’t super clear either?introjs->https://github.com/usablica/intro.js/wiki/Documentation->https://introjs.com/docs/intro/options/->https://github.com/usablica/intro.js/blob/master/example/programmatic/index.htmlwhich does show a list of ‘intro’ and ‘element’ keys… but boy do you need to dig (and guess a bit) - Attachment (GitHub): usablica/intro.js > intro.js - A better way for new feature introduction and step-by-step users guide for your website and project. - Attachment (introjs.com): Intro.js - Options > Intro.js - Better introductions for websites and features with a step-by-step guide for your projects.
Federico Marini (15:52:44): > about the introjs functionality: yes, sometimes it is frustrating to dig into the DOM components
Federico Marini (15:52:58): > and sometimes it just does not behave in the way one would expect
Federico Marini (15:54:07): > re: your question on the tour parameter: i’d stick to the df as long as the requirements are respected
Kevin Rue-Albrecht (15:57:05): > it seems definitely simpler that way. I was mainly curious whether the format of the current tour followed any specific introjs requirement. But since it’s just a data.frame in the end, we’re all good
Federico Marini (16:27:28): > some more freedomandbetter documentation would not be bad
Federico Marini (16:28:25): > some other libraries with similar functionality might have that, but none is that nicely in R - enough for me to use it seamlessly
Aaron Lun (18:42:23): > Most user-made tours should be simple enough; one step per plot describing what the plot is actually saying.
Aaron Lun (18:42:55): > Though possible, it would probably be beyond most users to highlight specific UI elements.
2018-03-28
Aaron Lun (05:08:50): > What’s everyone’s email?
Kevin Rue-Albrecht (05:09:19): > kevin.rue-albrecht@kennedy.ox.ac.uk
Charlotte Soneson (05:15:23): > charlotte.soneson@uzh.ch
Aaron Lun (05:18:59): > @Federico Marinigive me an email
Kevin Rue-Albrecht (05:19:44): > I’m pretty sure hisidealemail would be fine:http://bioconductor.org/packages/release/bioc/vignettes/ideal/inst/doc/ideal-usersguide.html(marinif@uni-mainz.de)
Kevin Rue-Albrecht (05:21:07): > Is there a rush? Did we get a reply from GB?
Aaron Lun (05:21:36): > Yes.
Aaron Lun (05:21:51): > And also, I told you so.
Kevin Rue-Albrecht (05:23:14): > (?!) That’s the last news that I remembered - Attachment: Attachment > I haven’t heard back from GB, but I’m going to assume they’’ll knock us back, in which case I have some MS-related tasks for everyone. I will start working on the figure; everyone can pick a dataset they like and know well, and write up a little tour via iSEE.
Aaron Lun (05:23:58): > Yeah.
Aaron Lun (05:30:19): > I mean, it’s not surprising.
Aaron Lun (05:30:39): > Fortunately we have an application note almost ready to go.
Kevin Rue-Albrecht (05:31:33): > Yep. And it was good to know before we invested more time and effort in a GB format
Kevin Rue-Albrecht (05:32:06): > Just looking at impact factors: > * GB: 11.908 (Google, 2016) > * BMC Genomics: 3.729 (Google, 2016) > * BMC Bioinformatics: 2.448 (Google, 2016) > * Oxford Bioinformatics: 7.307 (on their webpage, today)
Aaron Lun (05:46:26): > sighjust got an email on the “Berkeley matter”
Aaron Lun (06:37:08): > Anyway, we should think about how to proceed with a Bioinformatics application note (BAN for short).
Kevin Rue-Albrecht (06:37:58): > Everyone with experience being rejected from there raise the hand:hand:
Aaron Lun (06:38:07): > :hand:
Kevin Rue-Albrecht (06:38:17): > Anyone got through once? I’d love to know what the trick is
Aaron Lun (06:38:26): > What, for a BAN?
Kevin Rue-Albrecht (06:38:35): > yup
Aaron Lun (06:41:04): > No idea.
Aaron Lun (06:43:19): > The last time I did it (forcsaw) I got knocked back because I didn’t compare to existing methods. How was I meant to do that in two pages?
Kevin Rue-Albrecht (06:50:39): > Yeah. At least they only asked me that in BMC Bioinformatics, where I had space
Kevin Rue-Albrecht (06:52:47): > Just found that BAN can include “Additional Supplementary data can be published online-only by the journal”https://academic.oup.com/bioinformatics/pages/instructions_for_authors
Kevin Rue-Albrecht (06:54:55): > Sorry for the verbose copy-paste, but this seems relevant:Applications NotesAbstracts for Applications Notes are much shorter than those for an Original Paper. They are structured with four headings: Summary, Availability and Implementation, Contact and Supplementary Information. > - Summary: This section should summarize the purpose/novel features of the program in one or two sentences. > - Availability and Implementation: See above for advice and examples for this section. > - Contact: Full E-mail address to be supplied, preferably an institution address. > - Supplementary information: Links to additional figures/data available on a web site, or reference to online-only Supplementary data available at the journal’s web site.
Charlotte Soneson (07:06:05): > I went through twice, both with papers essentially related to visualization tools. I can’t remember any requests for comparisons.
Federico Marini (07:31:49): > :hand:for BAN
Federico Marini (07:32:16): > with mounting rage on my side
Federico Marini (07:33:17): > first no news for 5 months. after being “sent out” at least according their tracker, than no reply nothing. I contact the editor janet kelso directly, she checks in a couple of days, finds out nothing has been done.
Federico Marini (07:34:37): > gets sent out again - probably rushing the reviewers too. reviews come back, one of them was saying “cannot install the tool, pasting the log” -> the log says “missing unix libraries”. The other one compared pcaExplorer to a shiny app for exploring the iris dataset -> no novelty for this
Federico Marini (07:34:54): > the editor does not disagree with the sloppy reviews
Federico Marini (07:35:08): > and my fist hits repeatedly the table
Kevin Rue-Albrecht (07:38:24): > damn you beat me: I had 4 months without reply after sending it out
Kevin Rue-Albrecht (07:39:35): > one encouraging review with some work to do (my supervisor guessed it was a postdoc review) > one single-line review: “I don’t think that’s novel enough” (guess it was a PI)
Aaron Lun (09:05:51): > Okay, well, let’s see if it gets in as a BAN. I’m thinking of aiming it for the “Data and text mining” section.
Kevin Rue-Albrecht (09:09:23): > True. I initially thought of the “Gene expression” section (that encompasses much more than RNA, btw), but from their own description the “Data and text mining” sounds like a better fit (https://academic.oup.com/bioinformatics/pages/scope_guidelines)
Aaron Lun (09:13:00): > Especially as the gene expressionMUST compare to existing methods
Aaron Lun (11:01:16): > I can slap the rest of the MS together. Everyone else should put together tours for their fav datasets.
Aaron Lun (11:04:48): > We can host it on the CI’s shiny server, or elsewhere.
Kevin Rue-Albrecht (15:22:31): > @Kevin Rue-Albrechtuploaded a file:that historic moment… the calm before the storm ;) - File (PNG): that historic moment… the calm before the storm ;)
Aaron Lun (15:24:58): > yeah… racking ’em up.
Charlotte Soneson (15:28:34): > I’m putting together a tour for the 170,000 cell CyTOF data set, will try to finish it tomorrow or Friday.
Aaron Lun (15:32:13): > I suggest everyone make their intended further edits to the MS now, as I will convert it to the BAN format tomorrow; see how much space I have to work with regarding figuring.
Kevin Rue-Albrecht (15:42:28): > @Federico MariniI just spotted the first ‘ERROR’ in the bioc nightly build:http://bioconductor.org/checkResults/3.7/bioc-LATEST/iSEE/It’s therentrez::entrez_summarythat (randomly) runs into a 502
Aaron Lun (15:44:17): > I didn’t realize entrez_summary did a web call.
Kevin Rue-Albrecht (15:44:32): > Should we\notrunthis example, given that it depends on a working internet connection? I guess we can check out therentrezbuild report if it fails in iSEE
Aaron Lun (15:46:12): > Yes, but make sure you only\dontrunthe generated function, not the function that generates the function.
Kevin Rue-Albrecht (15:51:52): > Gotcha.
Federico Marini (16:01:15): > You got that already. Yes,rentrezdoes use the connection to NCBI dbs
2018-03-29
Kevin Rue-Albrecht (03:07:03): > A couple of things: > - I was looking whether the original publication associated with the TCGA data set could help write an accompanying tour, but from a quick look, it’s not really obvious how (https://academic.oup.com/bioinformatics/article/31/22/3666/240143) > - how do we want to integrate the tours to the app? bearing in mind that a tour depends on a layout (i.e., element being available). Are we wrapping this indemo()to repeat the preprocessing of the data each time (time consuming for the TCGA!) ? or do we want to make some user-visibleload_tour()that parses one of the tour descriptions to return the data.frame ready foriSEE(…, tour=)?
Aaron Lun (04:12:40): > Huh?
Aaron Lun (04:12:53): > Why would we integrate them?
Aaron Lun (04:13:11): > The idea is to have a shiny server running the tours on demand.
Aaron Lun (04:21:00): > This is just for the paper.
Kevin Rue-Albrecht (04:59:22): > Ahh sorry . Got you
Aaron Lun (09:29:27): > Talk to me, people.
Kevin Rue-Albrecht (09:31:37): > head split between two projects at once here, with two interviews and a seminar just over with:sweat:
Aaron Lun (09:32:40): > Looking at the BANs in the current issue. It can be said that they are of limited scientific quality.
Kevin Rue-Albrecht (09:39:01): > I haven’t read many BANs recently, tbh. Maybe the format has lost popularity, in general?
Aaron Lun (09:42:10): > Well, Bioinf has fallen from grace in general.
Aaron Lun (09:42:30): > I mean, their most cited was ages ago; clustalw, edgeR, samtools.
Aaron Lun (10:31:38): > @Charlotte SonesonDoes lukas actually use an SE in his workflow?
Aaron Lun (10:31:43): > Currently looking at the code.
Charlotte Soneson (10:33:34): > Which code are you looking at? Lukas’s package (which is about to be submitted to Bioc) will use a SummarizedExperiment-based representation. For Gosia’s workflow, that may not be the case.
Aaron Lun (10:34:33): > Right.
Aaron Lun (10:34:37): > Was looking at the workflow.
Aaron Lun (10:34:46): > Wanted to avoid a long stretch of Lun et al’s.
Aaron Lun (10:34:56): > What’s Lukas’ paper? The clustering one?
Aaron Lun (10:35:26): > Note: self-citations become much more embarrassing for author-year citation styles
Charlotte Soneson (10:35:45): > No, the package he is working on (the one he presented at the EuroBioc meeting).
Aaron Lun (10:37:31): > I guess this doesn’t have a paper yet, then.
Charlotte Soneson (10:38:37): > No, not yet. Hopefully not too far away
Aaron Lun (10:40:05): > Fuck. Haven’t even added a figure and we’re already over. Maybe we don’t need a figure? Just refer people to the guided tours?
Aaron Lun (10:48:48): > Okay, sent out invites to a repo
Aaron Lun (10:49:01): > Just enter the directory and typemake, itshouldwork.
Kevin Rue-Albrecht (10:51:21): > probably on my laptop, haven’t bothered installingwgeton the desktop workstation yet:upside_down_face:
Aaron Lun (10:51:48): > well, apparnetly OUP doesn’t like batch downloads; need to pretend to be a browser, hold on.
Kevin Rue-Albrecht (10:58:41): > honestly, don’t worry about it. I just tried quickly out of curiosity, but I won’t be able to do anything until this evening anyway
Aaron Lun (11:01:03): > Well, the makefile wouldn’t have worked even if you had wget. Now it does.
Kevin Rue-Albrecht (11:01:22): > ah. right.
Aaron Lun (11:38:12): > realized i overused the work framework
Kevin Rue-Albrecht (11:40:48): > Btw, I’ve got family visiting Oxford for the weekend, so I won’t be much online.
Aaron Lun (11:56:08): > God, the bib is almost as big as the text
Kevin Rue-Albrecht (11:56:27): > sounds like my PhD thesis
Aaron Lun (12:05:16): > Well, it’s done.
Aaron Lun (12:11:10): > Now to put together a figure
Aaron Lun (13:30:19): > @Aaron Lunuploaded a file:Figure 1Band commented: (A) will be a screenshot, yet to do. - File (PDF): Figure 1B
Aaron Lun (13:36:02): > Ugh, I’m sick of making figures. I’ll deal with the screenshot tomorrow.
2018-03-30
Kevin Rue-Albrecht (06:53:47): > FWIW, i’ve started writing notes for a tour of the TCGA data set
Kevin Rue-Albrecht (06:54:16): > however, i’m gonna go offline for the weekend now, going to pick up family in London this afternoon
Charlotte Soneson (07:02:14): > @Charlotte Sonesonuploaded a file:cytof_tour_code.Rand commented: I made a little tour for the CyTOF data set. - File (R): cytof_tour_code.R
Aaron Lun (07:07:40): > Cool, I might look at that later today if I make good progress with the rest of the figure.
Aaron Lun (07:08:13): > Feel freee to commit the tour to the repo; make atours/cytofdirectory and commit relevant (non-data) files there.
Aaron Lun (07:08:32): > Probably need a script describing how the data were obtained, though.
Aaron Lun (08:06:23): > ggplot team: font size does not affect the title or the legend, but it probably should.
Aaron Lun (10:34:44): > God, this BAN format is a real pain in the ass when dealing with figures that need to span the two columns.
Aaron Lun (11:11:49): > Okay, you would have all got a link to the folder. I’ve attached Fig1.pdf to allow you to recompile the tex. Note, however, that the screenshot is hihgly dependent on the screen size, so don’t take it again (and then edit the SVG for the mismatching indices) unless we’ve talked about it.
Aaron Lun (12:03:48): > Where the fuck is everyone?
Charlotte Soneson (12:48:13): > well, since it’s a holiday - maybe not working?
Kevin Rue-Albrecht (13:19:20): > Yup. Just in the bus back to Oxford bringing the family back. I’ve just red through the BAN. Good work and nice figure! > Two typos: > 1) Figure legend « Panel types are available visualize » : misses « to » visualise
Kevin Rue-Albrecht (13:21:02): > 2) actually not a typo: « transmitting plot will highlight » I would add « or restrict »
Kevin Rue-Albrecht (13:23:05): > When I get a chance I’ll add the reference to the TCGA paper. I think I pasted it earlier in the channel if someone has an urge to add it before I do
Aaron Lun (14:38:59): > OMG - one day into the long weekend - and one toilet is already clogged by someone.
Aaron Lun (14:39:19): > At this rate, I’ll run out of toilets before the cleaners get back to work.
Aaron Lun (14:39:43): > God, I’ll have to use the toilets that the wet lab people use.
Aaron Lun (14:39:45): > Ugh.
2018-04-01
Aaron Lun (14:58:44): > Anyone home?
Aaron Lun (14:58:57): > Rejigging all my Bioc workflows to drag them to BioC-devel
Aaron Lun (14:59:27): > Racking up those commits.
Aaron Lun (14:59:36): > Also, the dog threw up on my bed.
Aaron Lun (15:01:35): > But I was like, well, you’re not the only one.
Aaron Lun (15:01:46): > On the plus side, I meant that I washed my bedsheets for the first time this year
Federico Marini (15:03:17): > Kids have been puking too, if this makes you feel less lonely:disappointed:
Federico Marini (15:03:41): > and on my side, can’t count the frequency of fresh sheets
Federico Marini (15:03:43): > :smile:
Aaron Lun (15:04:11): > Yeah, that’ll teach us.
Aaron Lun (15:04:15): > Never work with small animals and kids
Federico Marini (15:04:28): > I’ll be driving home tomorrow but i’ll try to get a good look at the BAN
Federico Marini (15:04:41): > I assume there the main issue is the damn space
Federico Marini (15:05:23): > which sometimes forces to cut short the refs and be somewhat un-fairest
Federico Marini (15:06:13): > side note, I could not update yet the server. I needed a VPN tunnel which was not working
Aaron Lun (15:07:25): > No probs, no rush anyway.
Aaron Lun (15:07:49): > I need to prepare to go to Boston week after next, so will be out of it for a while.
Federico Marini (15:09:36): > Okie dokie:wink:
2018-04-03
Charlotte Soneson (04:02:04): > I wanted to fix this, but my laptop decided on Saturday that it didn’t want to start anymore, so I’m currently stuck with an old laptop that with the current OS can’t handle the new versions of R…hopefully it will be fixed soonhttps://community-bioc.slack.com/archives/C8BJLSP8T/p1522411583000273 - Attachment: Attachment > ggplot team: font size does not affect the title or the legend, but it probably should.
Charlotte Soneson (04:02:45): > @Aaron Luncould you send the Dropbox invitation also to my gmail (charlottesoneson@gmail.com). I think I also sent an access request on Friday. Thanks
Aaron Lun (04:20:17): > Done
Aaron Lun (04:20:26): > I thought I did so on Friday, but perhaps it didn’t go through.
Charlotte Soneson (04:21:08): > Thanks. I didn’t receive anything on Friday but maybe I missed it.
Kevin Rue-Albrecht (04:52:29): > Hi all, > Sorry that I dropped out for the weekend. Not sure whether it takes more energy to handle parents or kids for a weekend (I don’t have experience with the latter yet, but I’m sure@Federico Marinican share some wisdom on that)
Kevin Rue-Albrecht (05:11:06): > I tried out the tour from@Charlotte Sonesonlast night andreallyliked it. > In addition, it also emphasised how I’m not yet happy even with the first steps of the TCGA tour that I had started preparing last week. Mostly because I hadn’t decided yet on an aspect of the data set to focus on, which means that I had only started describing writing fairly generic facts about the data set and the (default) iSEE interface. However, The paper associated with the GEO record (https://www.ncbi.nlm.nih.gov/pubmed/26209429) mentions some results from breast-tumor samples and lung cancer samples that I should be able to highlight in a tour, like Charlotte did. - Attachment (ncbi.nlm.nih.gov): Alternative preprocessing of RNA-Sequencing data in The Cancer Genome Atlas leads to improved analysis results. - PubMed - NCBI > Bioinformatics. 2015 Nov 15;31(22):3666-72. doi: 10.1093/bioinformatics/btv377. Epub 2015 Jul 24. Research Support, N.I.H., Extramural
Charlotte Soneson (05:15:35): > :slightly_smiling_face:I guess one thing I wasn’t sure of was how much to make it a tour of the data set (which it would be in a real case) and how much a tour of the capabilities of iSEE. In the end it landed somewhere in between…
Aaron Lun (05:16:19): > Feel free to commit your tours and app.R scripts to the GH repo, e.g., likehttps://github.com/LTLA/iSEE2018/blob/master/manuscript/pics/setup.R.
Aaron Lun (05:16:36): > I would make a separatetours/directory and proceed with that.
Aaron Lun (05:17:06): > The idea would be that people could reproduce the tours on their own machines (hopefully the data are public).
Charlotte Soneson (05:20:27): > Yeah, I will do that as soon as I’m back on R-devel and can try it out again…for the data, Lukas just submitted anHDCytoDataExperimentHub package to Bioconductor that contains the CyTOF data that I’m using. This also contains the description of how the data was processed. My idea is to use that also for the vignette, to avoid the Dropbox link.
Aaron Lun (05:21:45): > Okay.
Federico Marini (14:31:11): > finally onto the manuscript
Federico Marini (14:31:24): > couple of comments after first deep look:
Federico Marini (14:31:38): > should we put the screenshot with a (now fake) 1.0 version?
Aaron Lun (14:31:54): > The difference being… the title?
Federico Marini (14:32:09): > overkill mode on, but “yes”
Aaron Lun (14:32:14): > I would like a function argument that allows users to specify the title themselves.
Aaron Lun (14:32:31): > For example, to describe the dataset or paper PMID
Federico Marini (14:32:32): > ok, even better
Federico Marini (14:32:51): > which branch is currently most recent?
Aaron Lun (14:33:06): > Not sure.
Federico Marini (14:33:28): > i saw now furtherfix is pretty much on par
Aaron Lun (14:33:36): > yep
Federico Marini (14:33:40): > andstill there
Aaron Lun (14:33:42): > probably should prune out some of these guys
Federico Marini (14:33:52): > so it can be a quick fix to do there?
Aaron Lun (14:34:03): > Whoops, just deleted it.
Federico Marini (14:34:07): > ehehe
Federico Marini (14:34:42): > I can also removeiconized. It seems we are all pretty much ok with having the text of buttons instead of buttons themselves
Federico Marini (14:35:02): > plus, it had a weird-y behavior
Federico Marini (14:35:27): > pruned
Aaron Lun (14:43:12): > ok
Federico Marini (15:07:11): > General “trick” to exploit later - when we might need every single line: > it could be possible to shorten up the refs, and have first … et al instead of the full list, if we copy-paste-and-edit the content of the bbl file
Aaron Lun (15:07:19): > Yes.
Aaron Lun (15:07:28): > I CBF’d at that point, but yes, we should do that.
Aaron Lun (15:07:42): > make sure you do it all in one commit so we can easily revert it later
Federico Marini (15:07:53): > I know it is not the fairest thing for the second & third & so on, but the BAN format is a pain in the ass
Federico Marini (15:08:08): > I miss the acronym for CBF
Federico Marini (15:08:35): > and i guess it is not the confederacion brasileira de futbol
Aaron Lun (15:08:42): > no
Federico Marini (15:08:51): > or whatever it should sound in correct portuguese
Federico Marini (15:10:24): > ok, urban dictionary to the rescue
Unknown User (15:29:00): > @Federico Marini commented on @Charlotte Soneson’s file https://community-bioc.slack.com/files/U34P8RS3B/F9YT7M7TP/cytof_tour_code.r: Excellent work @Charlotte Soneson - File (R): cytof_tour_code.R
Federico Marini (15:30:00): > One thought: when the tour is provided extra, and the users clicks out, the tour is kinda gone. Should we add a button to the ui to replaythattour passed as param?
Federico Marini (15:30:41): > instead of nasty full reloads
Aaron Lun (15:30:45): > Overwrite the default tour button with the provided tour, if the tour is provided.
Federico Marini (15:31:17): > or have the extra button just below if tour!=NULL?
Charlotte Soneson (15:31:19): > Good point, something like that would definitely be useful
Federico Marini (15:32:00): > so that the general one is still there, just in case. but this mode would be for showcasing, so I get the point of removing the default
Aaron Lun (15:32:41): > An extra button would be more painful, we’d have to set up a separate observer
Federico Marini (15:33:45): > I was thinking of a simplerenderUIsomewhere
Federico Marini (15:34:04): > which spits out the button only if an extra tour is there
Aaron Lun (15:44:44): > I’d favour just replacing the default tour, especially as there is no guarantee that the default tour works with arbitrary instances. For example, what happens when you don’t have a reducedDim plot?
Federico Marini (15:45:26): > points to somewhere which is not there
Federico Marini (15:45:37): > not so optimal
Aaron Lun (15:49:05): > Alright, I’ve had enough. GOing home
Federico Marini (16:05:36): > branchcustomizedis up for comments, if you want to take a look
Federico Marini (16:08:27): > … as well a round of comments on the manuscript -fede_edits
Kevin Rue-Albrecht (16:40:57): > question: > if the demo apps come with some preprocessing, like TCGA that is a lot nicer after a fair wait of t-SNE computations… do those computations have to be run for every instance of the app that are accessed, or will hosted apps only open up a new instance from precomputed data?
Charlotte Soneson (16:42:47): > I’d say precomputed…
Kevin Rue-Albrecht (16:44:35): > That’s be ideal. I think I was more uncertain about how those demo apps were going to be used, and how they would behave in each case: 1) hosted on a webserver with 1+ users accessing them, 2) run locally by users who would like to have a first guided experience through a data set.
Kevin Rue-Albrecht (16:45:26): > The latter is not a problem, as user can run separately the code to precompute the data themselves from the one line that launches the app
Kevin Rue-Albrecht (16:46:16): > not sure about situation 1) though. I seemed to understand that it was a possibility though
2018-04-04
Kevin Rue-Albrecht (05:36:13): > @Kevin Rue-Albrechtuploaded a file:test_EH1.Rand commented: Issue with predefined point selection between different plot types for demo? - File (R): test_EH1.R
Kevin Rue-Albrecht (05:37:07): > Hi all, I’ve just been trying to set up the demo app for the TCGA data set, and ran into the issue: > > Warning: Error in rep: invalid 'times' argument > Stack trace (innermost first): > 110: shiny::brushedPoints > 109: eval [<text>#1] > 108: eval > 107: .extract_plotting_data [/Users/kevin/git/iSEE/R/plotting.R#463] > 106: .plot_wrapper [/Users/kevin/git/iSEE/R/plotting.R#329] > 105: FUN0 [/Users/kevin/git/iSEE/R/plotting.R#69] > 104: renderPlot [/Users/kevin/git/iSEE/R/iSEE-main.R#1199] > 94: <reactive:plotObj> > 83: plotObj > 82: origRenderFunc > 81: output$redDimPlot1 > 4: <Anonymous> > 3: do.call > 2: print.shiny.appobj > 1: <Promise> > Warning: Ignoring unknown aesthetics: height, width >
Kevin Rue-Albrecht (05:38:04): > Weirdly, all I have to do is update the receiver plot (redDim) to get passed the warning and display the plot (e.g. change the point selection mode from ‘transparent’ to ‘color’)
Kevin Rue-Albrecht (05:41:00): > I have a feeling that the redDim plot is trying to access the selected data points before they are ‘brushedBy’. Possibly something to do with the arbitrary order in which we process panels (redDim -> colData -> featExprs), which could cause problem setting predefined selections in the latter two
Kevin Rue-Albrecht (13:33:00): > I just tracked it down to thiskeep_rows <- rep(TRUE, nrow(df))statement inshiny::brushedPoints. > Basically, I think that redDimPlot1 tries to brush on colData1 coordinate that are stillNULL(rep(1, NULL)causes theinvalid 'times' argument) above
Kevin Rue-Albrecht (13:33:41): > I’m trying to figure whether avalidatestatement, and if so, where, could block this until everything is initialised
Kevin Rue-Albrecht (13:34:42): > But as I pointed out earlier, it comes from the fact that we arbitrarily process plots in the following order:for (mode in c("redDimPlot", "featExprPlot", "colDataPlot", "rowDataPlot")) {
Aaron Lun (13:37:58): > … yes, that will cause problems.
Aaron Lun (13:38:22): > Even worse, it depends on who completes their rendering first.
Aaron Lun (13:38:44): > Even worse on servers where things can execute in parallel.
Aaron Lun (13:39:05): > Fortunately, we did spend a lot of time separating the generation of plotting data from the plotting itself.
Kevin Rue-Albrecht (13:39:08): > rendering as well? I thought theforloop would just cause problem between panel types
Aaron Lun (13:39:39): > WEll, if panel 1 received information from panel 2, it would never get that info during initialization.
Kevin Rue-Albrecht (13:40:14): > oh god:confused:
Aaron Lun (13:40:39): > The solution is to, at the start of the app, identify the dependencies (in the same way that we do for the code tracker)
Aaron Lun (13:40:52): > and then run through them to build upall_coordinates.
Kevin Rue-Albrecht (13:41:53): > I’m not even sure avalidatewould fix the situation, as it would probably just block execution the first time ifall_coordinatesis not available, without triggering a new computation when it is available, as none of the UI inputs would have changed
Kevin Rue-Albrecht (13:46:22): > preliminary look: > so something about using.get_reporting_orderon startup?
Aaron Lun (13:47:45): > We need some modifications to ensure that itonlyreports the order for linked plots.
Aaron Lun (13:49:41): > That is, remove unlinked nodes.
Aaron Lun (13:49:54): > Hold on… will deal with this in a bit.
Kevin Rue-Albrecht (14:07:12): > no worries, I still have to work out a better outline for the TCGA tour. I really liked Charlotte’s illustration of published results, and I found a couple of example genes mentioned in the companion paper to the TCGA data set
Aaron Lun (14:07:36): > hold on, still working on something else.
Aaron Lun (14:07:46): > bioc support, to be specific
Aaron Lun (14:11:31): > Is the TCGA example fast? If not, while I get to this, can you whip up something withallen?
Kevin Rue-Albrecht (14:12:30): > The TCGA is not particularly fast, especially to generate the reduced dimension data each time in a new R session
Kevin Rue-Albrecht (14:13:10): > I should have a quick MWE with allen fairly quickly
Kevin Rue-Albrecht (14:18:36): > @Kevin Rue-Albrechtshared a file:MWE allen - File (Canvas): MWE allen
Aaron Lun (15:17:53): > Fix now onfurtherfix. Can I ask you to write a few unit tests for the new functionestablish_eval_order? Also forget_reporting_order.
Aaron Lun (15:18:13): > Though read the internal document forestablishto see exactly what it does.
Kevin Rue-Albrecht (15:30:30): > I’ll look at it now. I need it anyway for the TCGA demo
Kevin Rue-Albrecht (16:51:40): > I got there. Damn it is a pain to set up tests. Need to understand what’s going on in the first place:stuck_out_tongue:
Kevin Rue-Albrecht (16:58:36): > i hate to ask those questions, but given that it’s not my code I’d rather ask: isN <- nrow(active_panels)a leftover in.get_reporting_order? it’s not used
Kevin Rue-Albrecht (17:26:27): > Separately: can anyone else: > - select a genes in rowStatsTable, > - transmit to a rowDataPlot > - tell us if the rowDataPlot also shows: > “Receiving selection from Row data plot 1 > Transmitting selection to Row data plot 1”? > (in addition to the rowDataPlot not applying the selection)
Aaron Lun (17:45:23) (in thread): > It’s not necessary, but leave it as it is, I have a commit on my other machine to deal with it that I didn’t push yet (as I was waiting for tests of the original function).
Aaron Lun (17:46:13): > By colour, or what?
Kevin Rue-Albrecht (17:49:47): > eh.. by ‘default’ I’ll say, I didn’t try another yet, so transparency
Kevin Rue-Albrecht (17:52:42): > (I was back on a TCGA demo where I don’t have rowData stuff at the moment, so I’m booting up an allen again now)
Kevin Rue-Albrecht (17:53:45): > waiiiiiiit
Kevin Rue-Albrecht (17:53:49): > hang on i’m just stupid
Kevin Rue-Albrecht (17:54:04): > I can’t read the difference between rowDataPlot and rowStatTable
Kevin Rue-Albrecht (17:54:52): > of course it was a self brush. duh. I thought I was selecting a gene from the table to highlight in the plot. sorry
Kevin Rue-Albrecht (17:55:39): > time to call it a day, it seems
Aaron Lun (17:59:03): > I thought that might be the case
Aaron Lun (18:31:08): > Anyway, I’m going to the HCA~hackathon~jamboree next week, and I’m giving a little talk. I’d like to find an excuse to shoe-horniSEEinto it, but the girl who manages our shiny server has gone overseas for the time being. Are there any other willing hosters? It will only be for a week, and it’s just the PBMC 4K dataset from 10X genomics; nothing too fancy.
Aaron Lun (18:32:31): > Also, I think ourNEWSfile is way too specific about annotating changes for minor version numbers. I think we should just have one\sectionfor each middle version bump. I think only the last entry gets compiled into the NEWS digest every release anyway.
2018-04-05
Kevin Rue-Albrecht (03:04:31): > I didn’t realise it, but indeed, only the last NEWS seems to be picked up for the Bioc release info:https://www.bioconductor.org/news/bioc_3_6_release/
Kevin Rue-Albrecht (03:05:49): > I’ll merge all the minor version NEWS into a single middle version one then
Kevin Rue-Albrecht (03:07:12): > I’m sorry, I don’t have a Shiny server at hand. It’s in the cards… that are still in the deck let’s say.
Federico Marini (03:09:56): > Message me@Aaron Lun
Federico Marini (03:10:06): > I can set up a small instance for that data
Federico Marini (03:13:02) (in thread): > I think they enter only the diff from release to release
Kevin Rue-Albrecht (03:20:30) (in thread): > argh again, i got it wrong: fede’s right, see package ‘BASiCS’ athttps://www.bioconductor.org/news/bioc_3_6_release/
Aaron Lun (08:07:58): > Mergecustomizedif there’s nothing else holding it up.
Aaron Lun (08:08:20): > Also delete the “SingleCell” from the default title.
Aaron Lun (08:08:35): > and put SummarizedExperiment as a single word.
Federico Marini (08:09:18): > it was just there for the team to check - I’ll edit, push, and merge
Aaron Lun (08:09:27): > :+1:
Federico Marini (08:15:31): > ok, waiting for CI checks - anyone can click on merge if you don’t mind
Federico Marini (08:15:51): > I’ll have a meeting to attend and probably the checks are not completed by then
Aaron Lun (08:16:12): > Ho ho ho. Who will remember to merge first -furtherfixorcustomized? The race is on.
Federico Marini (08:16:50): > Need4speed biocEdition
Aaron Lun (08:38:30): > victory is mine
Federico Marini (09:32:41): > totes legendary WINS
Federico Marini (09:32:43): > :stuck_out_tongue:
Aaron Lun (09:32:45): > See app materials onhttps://www.dropbox.com/sh/ohw3ikibo1ymku4/AADT6idzq7K0RgAGkE18e-mKa?dl=0 - Attachment (Dropbox): EmptyDrops2017 > Shared with Dropbox
Aaron Lun (09:33:02): > It’s indemonstration/, and relies on the latestmaster.
Aaron Lun (09:33:13): > Pretty neat, if I do say so myself.
Federico Marini (09:42:47): > http://shiny.imbei.uni-mainz.de:3838/iSEE_PBMC4k/
Federico Marini (09:42:51): > there you go
Federico Marini (09:43:09): > you might need to restart the tour - or wait, since the plots take a while-y
Aaron Lun (09:47:15): > Thanks, that’s great.
Federico Marini (09:47:36): > I join you on the idea that it is neat. Genome Biology will experience FOMO:smile:
Kevin Rue-Albrecht (09:47:41): > do you still need to set the selection in dimRed plot? I can’t see it
Kevin Rue-Albrecht (09:48:13): > I’m a lazy user who complains about having to brush it myself:stuck_out_tongue:
Federico Marini (09:48:22): > no i dont see it either
Aaron Lun (09:48:24): > No, I CBFd on defining a brush
Aaron Lun (09:48:47): > And this will be a live demo, so I thought I’d just do it.
Federico Marini (09:49:06): > yep, it is nice to see all update at once
Federico Marini (09:49:10): > just tried it now
Federico Marini (09:49:33): > and catched blindly all cluster 3 without knowing:smile:
Federico Marini (09:51:55): > I can leave this instance on, too
Federico Marini (09:52:18): > so feel free to use the link to propagate the love:slightly_smiling_face:
Aaron Lun (09:52:38): > :+1:
Federico Marini (09:58:43): > side thing: I am planning to come up also with a bioconda recipe for iSEE
Federico Marini (09:59:06): > I already did this for ideal too, and it is good + not so much extra effort
Aaron Lun (10:03:45): > okay. But is this a separate repo?
Federico Marini (10:04:14): > it ends up as a simple recipe here
Federico Marini (10:04:28): > https://github.com/bioconda/bioconda-recipes - Attachment (GitHub): bioconda/bioconda-recipes > bioconda-recipes - Conda recipes for the bioconda channel.
Federico Marini (10:05:03): > plus, the package here:https://bioconda.github.io/
Aaron Lun (10:05:23): > good. I didn’t want to clutteriSEEitself.
Federico Marini (10:05:31): > and it becomes installable withconda install bioconductor-iseeor similia
Federico Marini (10:05:44): > not happening, it is totally separated
Aaron Lun (11:37:01): > Fuck, I hate navigating the TCGA dataset.
Aaron Lun (11:37:12): > Am I missing something here?
Kevin Rue-Albrecht (11:37:13): > ah. why?
Aaron Lun (11:37:31): > Do you usehttps://portal.gdc.cancer.gov/?
Kevin Rue-Albrecht (11:38:05): > ahhhh.. not in iSEE you mean
Kevin Rue-Albrecht (11:38:26): > it’s so nice to navigate in iSEE:innocent:
Aaron Lun (11:38:44): > How do you get matched normals?
Aaron Lun (11:38:48): > I can’t find the damn normals
Kevin Rue-Albrecht (11:39:12): > I don’t know, but in this case, yes I totally agree with you that the portal in a pain
Kevin Rue-Albrecht (11:39:35): > it’s like a teaser about everything available, without really knowing how to get what you want
Federico Marini (11:42:49): > maybe you can exploit some of the recent blog posts from sean davis?https://seandavi.github.io/
Kevin Rue-Albrecht (11:42:56): > It’s the same thing about the ExperimentHub, actually. The EH1 data set does not include the matched controls that are described in the companion paper. I saw other versions of the TCGA data set in the ExperimentHub, I think one of them that has even more samples (9000+ I believe) but haven’t yet checked which contains what.
Kevin Rue-Albrecht (11:43:18): > So for the time being I stick with our current one
Federico Marini (11:43:27): > I also recall it is painful as hell to get matched normal tissue, not to mention the clinical covariates
Federico Marini (11:44:35): > I remember I tried in the before EH1 era to build that object myself
Federico Marini (11:44:47): > if I find the snippet I can send it over
Aaron Lun (11:54:08): > Note that on rare occasions, the tour starts before the observers are finished loading. I’m not quite sure why this is, I guess that the Shiny flushing system might have gotten a bit confused. Seems to be sporadic.
Federico Marini (11:54:46): > yeah i have seen that too
Federico Marini (11:55:00): > the observers with the objects might be there, the plots not yet
Aaron Lun (11:55:50): > No, that’s normal and fine.
Aaron Lun (11:56:04): > Normally the tour starts very soon after all the panels are loaded but the plots are not there yet.
Aaron Lun (11:56:21): > That’s okay; the reactive environment has been flushed, and we’re just waiting for the plots to re-render.
Aaron Lun (11:56:54): > What I just observed is that the tour starts after the left-side orgasnization panel is loadded, but befor ethe main panel is loaded. That is not correct. However, I haven’t been able to reproduce it since.
Federico Marini (11:58:18): > Uh. keep in mind the server is quite not the most powerful machine.
Federico Marini (11:58:27): > i don’t know if that is going to play a role
Aaron Lun (11:58:45): > Good, I was hoping for an excuse.
Aaron Lun (11:58:52): > If someone questions me about the time delay.
Federico Marini (11:59:22): > please do use that
Federico Marini (11:59:30): > the server is a low-profile machine
Federico Marini (12:00:14): > I got it for free and with exposure to public domain and I am king over there - but the price to pay is the not so hi performance
Federico Marini (12:00:34): > if you see how quick it could be on your flaming new lappy, you can show the difference
Aaron Lun (12:02:00): > lol
Aaron Lun (12:02:17): > Is it under constant attack?
Aaron Lun (12:02:26): > People trying to trojan in
Federico Marini (12:02:34): > not that I am aware
Federico Marini (12:03:01): > or if you pass me the joke, not that i’mAlware
Federico Marini (12:03:02): > ok
Federico Marini (12:03:13): > feierabend signs are clearly showing out
Aaron Lun (12:06:37): > Anyway, the premature tour problem seems to occur when the server’s been inactive for a while. It does the wrong thing the first time it restarts - and then it’s right when you try opening the link again.
Federico Marini (12:07:31): > ok, not optimal, yet still acceptable
Kevin Rue-Albrecht (12:08:16): > can a shorter timeout solve that? or actually make it worse?
Aaron Lun (12:09:53): > Not sure what’s the problem; I don’t know how shiny servers work.
Aaron Lun (12:10:01): > Is the R session always running?
Aaron Lun (12:10:14): > Doesapp.Rget re-run everytime someone opens a link?
Aaron Lun (12:10:20): > Or does the app just get reloaded?
Aaron Lun (12:10:38): > was I meant to save the app to a variable? I don’t think I did that.
Kevin Rue-Albrecht (12:12:22): > I’ve always been confused by the settings onshinyapps.iothat differentiate between timeout, sessions, instances, and workers, which I assume controls how the app behaves at the user and server levels
Kevin Rue-Albrecht (12:12:29): > never got to the bottom of that
Federico Marini (12:36:37): > that is correct how you gave it to me
Federico Marini (12:37:00): > R sessions are started upon startup i.e. visiting the page
Federico Marini (12:37:06): > that at least AFAIK
Federico Marini (12:37:40): > timeout is relevant especially when it takes a while to launch the app
Federico Marini (12:37:50): > something like many deps and so
Aaron Lun (12:38:48): > So I wonder what actually happens when I visit for the first time. Does it runapp.Rfrom scratch?
Federico Marini (12:40:07): > maybe there is some kind of “cache” mechanism?
Aaron Lun (12:42:47): > fuck this TCGA crap is bullshit
Aaron Lun (12:42:53): > who the hell came up with this idiot scheme
Kevin Rue-Albrecht (13:13:51): > Why are you looking at the TCGA btw?
Aaron Lun (13:15:22): > Collab asked
Kevin Rue-Albrecht (13:17:24): > Ah ok. Do let me know if you have a suggestion for the EH1 demo. > At the moment, I’ve settled on showing the higher expression or ERBB2 in BRCA. The companion paper mentions it relative to controls, but the other cancer types act as good enough controls for this purpose
Aaron Lun (14:02:21): > Ah,TCGAbiolinksis the one that actually works.
Aaron Lun (14:02:30): > God, that took most of the afternoon.
Federico Marini (14:17:22): > they even have a fancy app
Aaron Lun (14:20:08): > I thought the GUI icon on their vignette was giving me the middle finger
Aaron Lun (14:24:10): > Can we bold or italics in the tour text?
Kevin Rue-Albrecht (14:26:04): > seems like HTML tags could do the trick:https://stackoverflow.com/questions/20534558/using-intro-js-data-intro - Attachment (stackoverflow.com): Using Intro.js - data-intro > I am currently using Intro.js for my website and the code works is here
Welcome back
I wanted to put this audio playback button wi…
Kevin Rue-Albrecht (14:26:15): > “This is a bold tooltip.”
Aaron Lun (14:28:09): > Yep, looks like that worked.
Aaron Lun (14:29:43): > @Federico MariniI modified the start-up message so that people don’t get confused when the tour occasionally doesn’t work. The newapp.Ris on Dropbox, can you update it on your end?
Federico Marini (14:42:21) (in thread): > should be up and running
Federico Marini (14:42:49) (in thread): > please let me know if something is unexpected
Aaron Lun (15:07:44) (in thread): > :+1:
Aaron Lun (15:08:30): > I notice that we still have a lot of importFrom statements iniSEE-pkg.R. I’d like these to be moved to the relevant locations (mostlyiSEE-main.R), excepting things on which weDepends.
Aaron Lun (15:30:01) (in thread): > Just fixed the vignette choice infurtherfix; after merge, can you update the version on your server?
Aaron Lun (15:30:26) (in thread): > It’s pretty neat now; it’ll go to the release or devel version according to the version of the app.
Federico Marini (15:38:14) (in thread): > oh cool
Federico Marini (15:38:28) (in thread): > will do once merged
Federico Marini (15:38:42) (in thread): > when are you going to be presenting tmrw?
Federico Marini (15:44:25) (in thread): > FYI, merged in, updated server
Kevin Rue-Albrecht (16:32:03): > @Federico Marini: any chance we can add an app-wide<p align="justify">for the tours?
Kevin Rue-Albrecht (16:34:33): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-04-05 21.33.19.pngand commented: although it doesn’t always give the best results. But still, it somewhat helps spotting new lines - File (PNG): Screenshot 2018-04-05 21.33.19.png
Kevin Rue-Albrecht (16:35:46): > ah and last, i’m wondering whether you found the way to increase the tooltip size yet, it feels a bit too narrow sometimes
Aaron Lun (16:38:49) (in thread): > It’s sometime on Monday - Wednesday, in Boston
Aaron Lun (16:39:25): > I just use , it’s easier to see the newlines that way.
Kevin Rue-Albrecht (16:40:10): > I did that too initially. I just moved to
, that adds an even cuter little line spacing between paragraphs
Kevin Rue-Albrecht (16:40:59): > I think I’m going to mix both: no ‘justify’, but
Kevin Rue-Albrecht (16:41:14): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-04-05 21.41.05.png - File (PNG): Screenshot 2018-04-05 21.41.05.png
Kevin Rue-Albrecht (17:00:19): > Using theintro_firststeps.txtfile as a template, I’ve just come across#featExprPlot1_BrushEffect;, and I don’t have the appropriate demo running right now: shouldn’tBrushEffectbeSelectEffectin the current app? I’m 99% sure of it, but I’d rather check.
Federico Marini (17:24:52): > possibly. I did not touch the tour recently, so it could be one of the latest iterations was not covered
Federico Marini (17:25:24): > and yes, I would favor some discrete bolding here and there
Federico Marini (17:25:46): > I’ll revisit the tour tomorrow or in the next days
Kevin Rue-Albrecht (17:30:13): > I’m testing the updated SelectEffect now
Kevin Rue-Albrecht (17:30:50): > got a bit stuck looking for the name of some dropdowns that I want to anchor stuff to
Kevin Rue-Albrecht (17:34:43): > mhh.. I’m probably not doing the actions that the tour expects, because I see some weird behaviour. But the tooltip does jump to beneath the other plot, which is where the selection summary should appear. Still, I better focus on my TCGA tour and leave that up to you on a separate branch
Kevin Rue-Albrecht (17:34:46): > ’night
Federico Marini (17:38:25) (in thread): > I suggest you use a browser and devel tools such as the inspector
Aaron Lun (18:56:31): > Worth putting in bold the actions that the tour expects users to do?
2018-04-06
Federico Marini (03:12:05): > that was my thought too. So that they don’t miss or get lost the next steps if they did not do what we expected them to
Kevin Rue-Albrecht (03:17:33) (in thread): > Right. Thanks!
Federico Marini (03:47:20): > relevant for us all maybe:http://loom.linnarssonlab.org/dataset/cells/Mousebrain.org.level6/L6_Oligodendrocytes.loom/NrBEoXQGmAGHgEYq2kqi3IExagZjwBYI0R4jkA7AVwBs6UMnt5FZ4lWnUZFtk8VGgrZmbFmw59uQrAJ6k~BABw913AKwyhKeYL15EAdnyIAnOt3d8Yrrt5cDw6BSVhICALSwAdAOwVc1gANmxtP2MVTURNMM1oSigiKDF4cxEEQSM8XGVCZRIyOFSoWgYmA252ThS5GHxFV0VK9yTGtNaIIA
Federico Marini (03:47:42): > nice but IMHO way less usable than our stuff
Federico Marini (03:47:44): > :slightly_smiling_face:
Kevin Rue-Albrecht (04:11:16): > honestly, i wouldn’t have given it even a ‘nice’. Funny looking, yes, but I can’t find a plot that speaks to me
Federico Marini (04:12:27): > ok replace nice with ‘complex’:stuck_out_tongue:
Kevin Rue-Albrecht (04:31:03): > hehehe it’s like me and math-heavy papers: i call them ‘nice’ when do something that I have no clue how they do it (even though I need to use them in my pipelines)
Aaron Lun (04:32:29): > lol
Aaron Lun (04:32:37): > Yeah, nothing to worry about I think.
Aaron Lun (04:35:58): > No dynamic zoom, brushing, etc. And the metadata is gibberish
Kevin Rue-Albrecht (06:19:21): > what’s the strategy again, to get the Fig1.pdf, to compile the manuscript?
Aaron Lun (06:20:16): > should be on dropbox?
Federico Marini (06:38:52): > following up with another one…https://isb-cgc.shinyapps.io/shiny-iatlas/
Federico Marini (06:39:04): > this is more related, yet not generalizable and so on
Kevin Rue-Albrecht (06:45:57) (in thread): > damn - forgot about Dropbox: I was all about the git repo
Kevin Rue-Albrecht (06:51:05): > I do like their Analysis Modules. Where does the demo data come from? And more importantly, can custom data be loaded in there?
Aaron Lun (06:51:17): > It seems pretty bespoke to me.
Kevin Rue-Albrecht (06:51:41): > Just found “The initial version of CRI iAtlas is based on an analysis performed by The Cancer Genome Atlas (TCGA) Research Network on the TCGA data set comprising over 10,000 tumor samples and 33 tumor types (Thorsson et al. Immunity, 2018).”
Charlotte Soneson (07:32:01): > Ok, I’m back on a working computer and I’ll take a look at theggplotissues.
Charlotte Soneson (07:35:44): > I just have to check again, although I know we have talked about this before: I remember it was a problem exporting theparam_choiceswithout closing the app, but is there any way we could print out the current state like we do for the code tracker. When I have set up the tours, it just becomes a bit painful to have to specify the selection/brushing rectangles manually to start in a specific state (even if the information is available in the code tracker)…
Aaron Lun (07:51:48): > what do you mean? You would need to copy-and-paste the deparse’d structures anyway into theapp.R.
Charlotte Soneson (07:53:06): > Yes, you would need to copy it, but I was wondering if one could have all the information in one place (to generate the inputDataFrames for the*Argsarguments)
Charlotte Soneson (07:53:48): > So like the code tracker, but to generate those DataFrames
Aaron Lun (07:55:26): > Hm.
Aaron Lun (07:55:32): > Possible.
Aaron Lun (08:01:20): > You could add a button that opens a modal, which would then trigger a giant dump of all activeDataFrames.
Aaron Lun (08:01:33): > This would be huge, though.
Charlotte Soneson (08:02:22): > It would be one per plot type, right?
Aaron Lun (08:02:56): > You’d want to split up the brush and lasso definitions per individual plot, otherwise it becomes illegible.
Aaron Lun (08:03:28): > Most of them should be NULL, though.
Charlotte Soneson (08:04:41): > Ok, but does it have to be legible…:face_with_rolling_eyes:if you could just take the code and paste it to create whatever you need to feed intofeatExprArgsetc
Aaron Lun (08:05:57): > Well, you’d need to know when to start copy-and-pasting. Thedeparsed brush is pure gibberish, lots of open and closing brackets.
Charlotte Soneson (08:06:57): > Sure. But if we could do it like the code tracker (i.e., actually give the code that should generate the objects) you could just copy an entire section.
Aaron Lun (08:10:10): > I was thinking about what happens if you only want the code to reproduce of only one of many e.g. reddimplots, if multiple plots contain a brush.
Aaron Lun (08:10:19): > Well, it’s not too hard to make it look pretty.
Aaron Lun (08:10:40): > If you can set up a UI element, I’ll do the backend.
Charlotte Soneson (08:11:04): > Ok!
Charlotte Soneson (09:08:27): > The modal is in theargsmodalbranch, together with some other small fixes
Aaron Lun (10:34:03): > Oh, if anyone is giving scRNA-seq courses, this may help:https://github.com/MarioniLab/scTeachingMaterials - Attachment (GitHub): MarioniLab/scTeachingMaterials > scTeachingMaterials - Teaching materials for single-cell RNA-seq data analysis courses.
Aaron Lun (10:34:13): > Finally got around to doing it, after being nagged about it for so long.
Kevin Rue-Albrecht (10:36:30): > thanks for sharing! i don’t know where i’d fit any teaching, but it can definitely come in handy
Federico Marini (10:43:01): > Cool!
Aaron Lun (11:34:02): > @Charlotte SonesonIt is done. Can you do me a favour and add unit tests for.report_memory()and.deparse_for_viewing()? The former needs special testing for list column fields (e.g., to make sure you get sensible stuff with brushes). The latter also needs testing to ensure that nothing funny happens whendeparse()is called inside a function - it should be okay, but a few rigorous runs shouldn’t hurt. Something as simple as creating an object, deparsing it, and evaluating the commands to ensure you get the same object.
Charlotte Soneson (11:35:15): > Sure, I’ll do that.
Aaron Lun (11:40:00): > Also, have you tried accessing the docker containers for the course?
Charlotte Soneson (11:40:26): > Haven’t had time yet, thought I’d do it tomorrow. You?
Aaron Lun (11:40:54): > Trying to do so now.
Aaron Lun (11:40:56): > Ich bien confused
Charlotte Soneson (11:41:31): > Doesn’t work?
Aaron Lun (11:41:58): > Well, I don’t know what I’m meant to be doing.
Aaron Lun (11:42:12): > Don’t understand where the docker images are running.
Aaron Lun (11:43:49): > Wait, okay, I figured it out.
Aaron Lun (11:44:10): > You have to log into teamviewer, and then you can access the Rstudio docker images via firefox.
Charlotte Soneson (11:44:26): > Ok, good.
Aaron Lun (11:46:02): > fuck, teamviewer gets so confused tho.
Aaron Lun (11:46:16): > once you alt-tab, it thinks the enter key is the windows key
Charlotte Soneson (11:46:34): > Wha? Why would it do that?
Aaron Lun (11:46:44): > dunno
2018-04-07
Kevin Rue-Albrecht (06:59:26): > Hi all, I’ve got a poster deadline this weekend, so I won’t be editing the manuscript more than the TCGA reference that I’ve just pushed. Feel free to merge or continue onkevin_edits, which branched fromfede_editsand therefore is the latest version
Kevin Rue-Albrecht (07:03:47): > One more thing about the tours: I saw mention of making instructions to users inbold. On the TCGA tour, I had started to bold key info (e.g. number of samples, genes, UI panel names). Should I: 1) put it back to normal font? 2) use italics instead? 3) should instructions to users have an extra<instruction>markup on top of the bold?
Kevin Rue-Albrecht (07:05:40): > we could used the wiki again to document guidelines for that (https://github.com/csoneson/iSEE/wiki/Documentation-guidelines)
Aaron Lun (07:14:49): > Think you’re overcomplicating things.
Kevin Rue-Albrecht (07:25:28): > With the wiki, probably. I just wanted to keep track of our tour guidelines
Aaron Lun (07:26:41): > Well, we can have a look at it once it’s done and see what looks best.
Aaron Lun (07:27:05): > Doesn’t take that much effort to change, and I don’t have a good idea of what’s aesthetically appealing right now.
Kevin Rue-Albrecht (07:28:33): > Actually, that’s a good idea too. We can explore different options and pick our favorite (or mix and match)
Aaron Lun (07:32:50) (in thread): > Just started a GH issue so we don’t forget.
Aaron Lun (07:37:53): > I posted a number of GH issues, at least some of which are ggplot related. I’ll be out of it all next week, so see if we can clear a few of them before the release. I think they are fairly easy to tackle.
Federico Marini (08:28:32): > For the tours, I would suggest to use bold only for > - main features > - initial warning/important sentences > - actions that the user needs to take to proceed correctly in the tour > If we overuse it, then it loses lots of its value
Federico Marini (08:30:35): > But the most important part I think is the clarity of the step text
2018-04-08
Kevin Rue-Albrecht (14:10:32): > Hi all. Having weird problems here, trying to finish up the tour of TCGA data. Suddenly, having#colDataPlot1;...crashes the tour: it doesn’t start up ‘onFlush’ nor with the tour button. If I remove that line, the tour goes well. It doesn’t do it for the other plot types (redDimPlot1, featExprsPlot1). Did that happen to anyone?
Kevin Rue-Albrecht (14:12:06): > aaaargh… nevermind. I mindlessly added a;symbol in the text, which messed up parsing the file
Kevin Rue-Albrecht (18:41:01): > we all gotta thanks my fiancée for this one (pointed me this one knowing I’m onto some lincRNA work): > ‘iSeeRNA: identification of long intergenic non-coding RNA transcripts from transcriptome sequencing data’ (2013)https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3582448/ - Attachment (PubMed Central (PMC)): iSeeRNA: identification of long intergenic non-coding RNA transcripts from transcriptome sequencing data > Long intergenic non-coding RNAs (lincRNAs) are emerging as a novel class of non-coding RNAs and potent gene regulators. High-throughput RNA-sequencing combined with de novo assembly promises quantity discovery of novel transcripts. However, the identification …
2018-04-09
Federico Marini (03:49:46): > strange it did not even pop up once when we googled up iSEE + rna-seq or similar
Federico Marini (03:50:03): > what if we do a tour on the broad’s firehose data
Federico Marini (03:50:30): > and we call the functioniSEE_fireandcharge some rights to Ed Sheeran?
Kevin Rue-Albrecht (05:17:48): > You mean this one?https://www.ncbi.nlm.nih.gov/pubmed/18007671 > > Gene silencing by double-stranded RNA. > Fire AZ. >
Federico Marini (08:26:43): > <!channel>when do we want to make feature freeze for the upcoming release?
Federico Marini (08:27:11): > a.k.a. what of the open PR/issues do we want to address now?
Charlotte Soneson (08:28:13): > I’ll try to get to making some unit tests in the coming days (#172), but that’s hardly a new feature…
Federico Marini (08:33:36): > I was just checking the release schedule
Federico Marini (08:34:25): > we still have some time
Federico Marini (08:34:37): > but just to estimate the remaining effort
Kevin Rue-Albrecht (08:41:18): > Reading (#171) again, I think I’ll have to play with the app to fully understand what the problem is. It doesn’t sound like a lot of work, but you never know.
Kevin Rue-Albrecht (08:41:44): > My own question is whether we want to include all the tours, for the next release.
Kevin Rue-Albrecht (08:44:00): > I’ve pushed my first offer for the TCGA data set. It runs locally, so I welcome feedback if anyone finds the time to testtour_TCGA. > See?tourTCGAI just need to make Travis happy by fixing a couple of CI-specific issues.
Federico Marini (08:46:24): > I saw it but did not check it yet
Aaron Lun (08:46:38): > Holy fuck you’re engaged?
Aaron Lun (08:47:14): > Fuck you guys, I’m still young
Charlotte Soneson (08:48:24): > Do we want the tours in theiSEErepo, or in the one for the paper?
Federico Marini (08:48:47): > I’d say for the paper
Federico Marini (08:49:00): > if also in the pkg, nothing against it
Charlotte Soneson (08:49:01): > Yeah, that’s how I understood Aaron’s previous comments too
Kevin Rue-Albrecht (08:50:01): > hahah yes I am. Not at the kids step yet, still time for that ^^
Kevin Rue-Albrecht (08:51:25): > eh ok then.. so you mean I shouldn’t merge my PR for the TCGA tour?
Federico Marini (08:54:09): > (funny note: the text in bold showed up quite sooner on one run)
Kevin Rue-Albrecht (08:54:34): > in my tour?
Federico Marini (08:56:35): > yes, now I was trying it
Aaron Lun (08:57:08): > Tours should be in theiSEE2018repo
Aaron Lun (08:57:19): > We only need one tour for the package itself
Kevin Rue-Albrecht (09:01:39): > Right. So I’ll move the tour code over there, and out of the package repo. I can’t unit test it anyway given that it relies on downloading from theExperimentHuband caching the data
Aaron Lun (09:03:01): > I presume this was on aiSEEbranch, so you can just delete it.
Kevin Rue-Albrecht (09:04:06): > Absolutely. It’s all on a side branch. No worries.
Federico Marini (09:05:44): > IMHO you can keep it in the form as it is, with the function and so
Federico Marini (09:05:57): > just on the other repo
Aaron Lun (09:06:23): > Currently at the HCA jamboree, is the app good to go?
Kevin Rue-Albrecht (09:07:02): > ok. 1) definitely, i’ll move it to the iSEE2018 repo. 2) i’ll leave thetcga_tourbranch for a day or two, in case we change our minds
Kevin Rue-Albrecht (09:08:05): > @Aaron LunI think the app still needs a couple if GH issues fixed (unit tests and the discrete zoom thing), but we already have the version bump (0.99.3) on our GH repo.
Kevin Rue-Albrecht (09:08:19): > so whenever I get the green light, I can push upstream
Aaron Lun (09:09:49): > Best leave it as it is for the time being, I don’t want it to break outright during the preso.
Kevin Rue-Albrecht (09:10:16): > agreed
Kevin Rue-Albrecht (09:11:05): > just crosses my mind now: having the tours steps as part of the package makes it easy for us to parse the steps, in a known location and format (extdata/…)
Kevin Rue-Albrecht (09:11:44): > but I can also adapt a bit my preprocessing script to take a file location supplied by our expert reviewers
Aaron Lun (09:13:55): > I woudln’t worry about that.
Kevin Rue-Albrecht (16:09:50): > Alright. I’ve > 1) merged an overdue fix on help pages that include internet-dependent examples, > 2) pushed an updated branch that offers packaged functions to run the TCGA tour (not merged yet), > 3) opened a branch on the manuscript repo that extracted the functions from (2) into a script that reviewers can run, provided they edit the path to the file of tour steps to their local path
Kevin Rue-Albrecht (16:10:29): > I’ll leave you guys to think what’s best for (2) and (3)
Federico Marini (16:16:28): > for 3)
Federico Marini (16:16:49): > you can encode the path of the tour to be linked as the github repo address?
Federico Marini (16:17:33): > or alternatively we can instruct people to clone the repo, and then we give just the relative path wrt the repo home
Kevin Rue-Albrecht (17:56:19): > I like both. Plus they’re not mutually exclusive. Considering how small the ‘steps’ file is, it probably would be fine to have the iSEE2018 script given with acurlcommand pointed to the package repo.Reallyimpatient people could download the text file locally and change the path ^^
2018-04-10
Charlotte Soneson (02:53:20): > TCGA tour looks nice!
Charlotte Soneson (02:54:09): > I uploaded the CyTOF tour to theiSEE2018repo as well. The data is still a dropbox link, as soon as Lukas’s CyTOF data package is accepted in Bioconductor I’ll use that instead.
Kevin Rue-Albrecht (03:00:21): > Thanks!
Kevin Rue-Albrecht (03:02:20): > I have to give credit to your CyTOF tour for inspiration, and I’m still not using all the panel types that I could, but I thought I’d start somewhere and edit from suggestions if any.
Charlotte Soneson (03:03:59): > Yeah, I don’t think all the tours have to use all the panel types - it’s probably better to show a range of different combinations to show the capabilities
Kevin Rue-Albrecht (04:52:28): > When I get the chance, I’ll see if I can pull up a small panel of genes from a rough DE, to show as a heat map. Might be more efficient in a smaller data set likeallen, though. I don’t want the tour to require too much/lengthy pre-processing either.
Aaron Lun (06:40:54): > I feel like absolute shit.
Aaron Lun (06:41:11): > Jet lag + headache + sore throat + stomachache
Aaron Lun (06:41:20): > ugh. This was why I didn’t want to go.
Federico Marini (07:41:57): > but at least the demo did work?:slightly_smiling_face:
Aaron Lun (07:42:55): > Thank you for your concern Fed:face_vomiting:
Aaron Lun (07:43:04): > I haven’t done it yet, doing it this afternoon.
Federico Marini (07:43:33): > Oh damn. Then gute Besserung first!
Federico Marini (07:44:45): > go with the pun that you can let the data speak if your voice is gone?:tumbler_glass:(for the glass half full)
Aaron Lun (07:49:13): > lol
2018-04-11
Kevin Rue-Albrecht (05:54:22): > How are things? Feeling better? How did it go?
Aaron Lun (08:46:01): > Presenting today, because I was feeling like shit yesterday
Kevin Rue-Albrecht (08:53:18): > I hope you have a better time today then:slightly_smiling_face:
Federico Marini (10:54:11): > I was showing the manuscript draft to my mentor today and handed him a copy to read
Federico Marini (10:54:35): > Tiny thing: do we need to specify somewhere the “joint1stauthorship” somewhere?
Aaron Lun (10:54:45): > Probably yes
Kevin Rue-Albrecht (10:55:04): > Right, I haven’t seen it yet.
Federico Marini (10:55:08): > at best in a way that takes the least space:stuck_out_tongue:
Kevin Rue-Albrecht (10:56:52): > Speaking of which, I had the vague hope of looking into ways to get closer to the 2-page format… until I realised that the references take up the equivalent of one of our pages of text:sweat_smile:
Kevin Rue-Albrecht (10:57:52): > I don’t see any way to make that miracle happen
Kevin Rue-Albrecht (10:58:44): > Anyway. I’ll disappear for our weekly hour of journal club now:sweat:
Federico Marini (11:40:07): > mousebrain.orgseems to use these loom files I reported in a previous link
Federico Marini (11:40:26): > and today i ran intohttp://dropviz.org/
Aaron Lun (11:41:59): > Well, talk went well.
Federico Marini (11:45:26): > Sehr gut:slightly_smiling_face:
Aaron Lun (19:37:31): > Lol watching a show on American news discussing ball tampering in cricket.
Aaron Lun (19:37:35): > Didn’t know that they cared.
2018-04-12
Kevin Rue-Albrecht (04:08:37): > I would too, when the alternative is Trump’s Twitter feed:yum:
Federico Marini (16:16:56): > huh
Federico Marini (16:17:04): > someone already cited us
Federico Marini (16:17:05): > https://www.biorxiv.org/content/biorxiv/early/2018/04/11/298653.full.pdf
Federico Marini (16:17:12): > or better, “cited”
Kevin Rue-Albrecht (16:17:25): > ?! looking in 2 min
Federico Marini (16:17:30): > they used some fantasy in the names of the authors
Kevin Rue-Albrecht (16:28:03): > lol for the authors
Kevin Rue-Albrecht (16:28:58): > Shame on me: having lived in Ireland for 4 years, I was about to push a pun about the Aran Islands.. but it doesn’t match their typo
Charlotte Soneson (16:43:37): > :thinking_face:
Kevin Rue-Albrecht (16:45:14): > btw, I’m looking at our manuscript right now, and for the life of me I can’t figure what the hell is LaTeX for ‘these authors contributed equally’. I haven’t even worked out half the macros used to define authors yet
Charlotte Soneson (16:46:22): > Can we just add another affiliation?:slightly_smiling_face:
Kevin Rue-Albrecht (16:47:44): > I was tempted. It’s just when I saw the\correspmacro that it made me wonder how many other gems are hidden and how to find them
Kevin Rue-Albrecht (16:48:55): > grep -R corresp ./*tells me that./bioinfo.clsis a good place to start
Kevin Rue-Albrecht (16:50:53): > my god.. it just makes me realise how noob i am with Latex:sweat_smile:
Kevin Rue-Albrecht (17:29:06): > oh my, i got there in the end
Kevin Rue-Albrecht (17:29:21): > Pushing in a minute.
Kevin Rue-Albrecht (17:47:05): > Off to sleep. As per latest commit, i’ve dropped atheirfrom the acknowledgements to fit all the funding section on page 2
2018-04-13
Federico Marini (12:01:21): > @Aaron Lundid you get any significant suggestion on new features to implement at the presentation/jamboree or whatever that was?
Aaron Lun (12:01:29): > Nope
Federico Marini (12:02:10): > I was only wondering if it would be possible to support some type of gene-set-like parameter. I explain myself better, it woudl be for the heatmap
Federico Marini (12:02:47): > and instead of going by hand or importing from table, it is also possible to upload gene list, also maybe in a format similar to the gmt for GSEA
Federico Marini (12:03:02): > and these would then be available from some dropdown-like menu
Aaron Lun (12:03:11): > as long as it’s not specific to GO.
Aaron Lun (12:03:13): > or whatever
Federico Marini (12:03:22): > yes
Federico Marini (12:03:39): > the format would just be something that the gseabase package could deal with
Federico Marini (12:03:53): > if I recall correctly, readGMT or similar stuff
Federico Marini (12:04:16): > but basically it would also be “the signature our group found in a different paper”
Federico Marini (12:04:40): > … smoothly projected in the current data
Kevin Rue-Albrecht (12:04:47): > GSEABase has all that, not only for GO
Kevin Rue-Albrecht (12:05:24): > 5-7 types of gene sets as I recall, with the infrastructure to make custom ones I believe.
Federico Marini (12:05:37): > I would think of something able to recognize if in the gmt format one or more lists are there, and then these would get parsed
Kevin Rue-Albrecht (12:06:13): > I would have thought that GMT can hold any number of gene sets (one per line, right?)
Federico Marini (12:06:21): > I am just throwing the idea in the wild knowing the weekend is coming and knowing I could forget that in the kinder-hurricane
Federico Marini (12:06:37): > probably, kev
Federico Marini (12:07:15): > would need to check, but it does make sense, as this is how the MsigDB are publicly available
Kevin Rue-Albrecht (12:08:21): > Definitely something worth considering. But first I was vaguely hoping to close “Zooming for discrete variables #171” if I get the chance this weekend.
Kevin Rue-Albrecht (12:09:21): > I’d like if we could push upstream by Sunday, so that we would solve all the notable issues left before the release
Federico Marini (12:13:15): > Yes, these list-thingy is not the top priority but more of an enhancement for v 1.2.0:stuck_out_tongue:
Federico Marini (12:14:01): > FYI, the schedule for ERUM is out, I got the slot before the river cruise tour:smile:
Federico Marini (12:14:03): > http://2018.erum.io/#schedule - Attachment (2018.erum.io): eRum 2018 @ Budapest > European R Users Meeting 2018 in Budapest, Hungary
Aaron Lun (13:05:42): > I don’t know any of these people.
Federico Marini (19:25:06): > tbh me neither
Federico Marini (19:25:13): > or at least few
Federico Marini (19:25:52): > maybe i know their works without connecting it to them
2018-04-15
Kevin Rue-Albrecht (07:50:33): > How’s everyone?
Aaron Lun (07:54:00): > still a bit:face_vomiting:
Aaron Lun (07:54:05): > coming out of my nose mostly
Kevin Rue-Albrecht (07:54:16): > I’ve been looking into the zooming of discrete variables, with only partial success last night. But I think I took a wrong overall approach: re-jittering the point with width set to the minimal distance between the zoom coordinates and the closest discrete coordinate (e.g. for a discrete X variable, zoom fromx=0.9-2.3would rejitter all levels withwidth=min(abs(0.9-1), abs(2.3-2))
Aaron Lun (07:54:29): > Shouldn’t be any need for rejittering
Aaron Lun (07:54:41): > I thought settingexpand=TRUEwould have been enough…
Kevin Rue-Albrecht (07:54:41): > exactly
Kevin Rue-Albrecht (07:54:51): > ah. didn’t think ofexpand
Aaron Lun (07:55:03): > Another option is to kick out the coordinates to the nearest 0.5 on the x/y-axes.
Kevin Rue-Albrecht (07:55:09): > I’ll have to check out this option again. Can’t remember its function
Aaron Lun (07:55:19): > God PCBI is a real pain in the ass
Aaron Lun (07:55:22): > re formatting
Kevin Rue-Albrecht (07:55:29): > your last one wasexactlywhat was on my mind
Aaron Lun (07:55:36): > They want EPS… which is stupid
Kevin Rue-Albrecht (07:55:41): > (the nearest 0.5)
Aaron Lun (07:55:42): > But the thing that really takes the cake
Aaron Lun (07:55:54): > is the fact that they want me to split up the supp info
Aaron Lun (07:55:57): > into separate files.
Aaron Lun (07:55:59): > That’s just ridiculous.
Kevin Rue-Albrecht (08:19:38): > I’ve also had this feeling recently, that manuscript are like LateX.. the pain is not the content, it’s rather the formatting ^^
Kevin Rue-Albrecht (08:20:50): > anyway.. independent advice: add/remove 0.5afterapplyingfloor/ceilingfor the desired effect:sweat_smile:
Kevin Rue-Albrecht (08:26:54): > anyway. i got it for the violins. moving on to the squares
Kevin Rue-Albrecht (08:43:17): > There. I’ve tested (horizontal)violins and square plots. Branchcartes
Aaron Lun (09:17:26): > Ah. Soexpand=TRUEdidn’t work?
Aaron Lun (09:20:00): > Anyway, it looks like it works well enough.
Aaron Lun (09:21:25): > I noticed that you’re branching offsquareplots, so we might as well deal with that before merging.
Kevin Rue-Albrecht (09:22:35): > I didn’t tryexpand, mainly because it would ‘adds a small expansion factor to the limits’, which would not solve the problem. Picture a scenario where a user brushes very near the tick of the discrete level (i.e., 0.999). Expand would add a bit to the left, but it would still be unbalanced w.r.t the right side
Aaron Lun (09:22:46): > Hm..
Aaron Lun (09:22:47): > okay
Kevin Rue-Albrecht (09:23:38): > here, the fix is simple, and it avoids the need to re-jitter within the zoomed area to display all the points
Aaron Lun (09:24:00): > yep, fine.
Kevin Rue-Albrecht (09:24:14): > what’s the status onsquare_plotsthen?
Aaron Lun (09:24:40): > It works; try making a square plot with only one factor on the x or y axis
Aaron Lun (09:25:02): > but the tracked code isreallymessy
Kevin Rue-Albrecht (09:25:53): > I tried and saw a decent plot. I just couldn’t picture all that the new code was doing for that, so my question is rather whether the status is ready for merge or if it needs anything else
Aaron Lun (09:26:28): > The idea is to basically have a bar plot when there is only one factor
Aaron Lun (09:26:45): > i.e., frequencies prop to areas prop to height
Kevin Rue-Albrecht (09:28:31): > oh right, a horizontal bar plot, basically
Aaron Lun (09:28:42): > or a vertical one, depending.
Aaron Lun (09:28:49): > mirrored around the center.
Kevin Rue-Albrecht (09:29:30): > ahhh got you. When one level on the X axis. Not just when ‘X=None’ is selected.
Aaron Lun (09:29:44): > yep
Aaron Lun (09:29:47): > that’s right
Kevin Rue-Albrecht (09:30:42): > mhh.. does that sound like a new plot type then? as in ‘scatter’, ‘violin’ ‘square’… ‘bar’ ?
Aaron Lun (09:32:29): > well, it’s a generalization of the square plot.
Aaron Lun (09:32:42): > So it will naturally turn into a mirrored bar plot when the time is right.
Kevin Rue-Albrecht (09:33:50): > bit more than I anticipated, tbh. I wonder whether I should re-branch from master and cherry-pick my ‘discrete zoom’ commits into a separate PR then
Aaron Lun (09:34:15): > No, not necessary.
Aaron Lun (09:34:31): > This will be done in a moment.
Kevin Rue-Albrecht (09:36:08): > Ok. Still, I got confused when you left the issue discussion at “no need to do any shifting. Oh, good.“. Thought the problem was solved.
Aaron Lun (09:37:15): > Only in my head.
Kevin Rue-Albrecht (09:39:07): > ok… so I can leave that one to you and your head then? I got a few things of my own to finish up in that case.
Aaron Lun (10:12:23): > Done. Have a play and merge when happy.
Kevin Rue-Albrecht (10:12:47): > Cool, will look at it while compiling a local package:slightly_smiling_face:
Kevin Rue-Albrecht (10:16:18): > pushed yet? I don’t see anything more recent than my last commit
Aaron Lun (10:19:52): > should be there
Aaron Lun (10:20:32): > hm…
Aaron Lun (10:20:48): > yep, it’s there
Kevin Rue-Albrecht (10:21:33): > :musical_score:I can see clearly now
Kevin Rue-Albrecht (10:37:21): > @Kevin Rue-Albrechtuploaded a file:looks fine to me - File (PNG): looks fine to me
Kevin Rue-Albrecht (10:39:14): > also, for those who haven’t booted up my earlier fix of today, I think the simplest fix was the most intuitive. Here is the full view (no zoom)
Kevin Rue-Albrecht (10:39:25): > @Kevin Rue-Albrechtuploaded a file:full - File (PNG): full
Kevin Rue-Albrecht (10:40:29): > @Kevin Rue-Albrechtuploaded a file:and after zoom (bottom right 4)and commented: (irrespective of the selection area) - File (PNG): and after zoom (bottom right 4)
Kevin Rue-Albrecht (10:44:02): > Anyway, I’ll open a PR oncartes. If no one raises an issue about it, I’ll merge by this evening. Do you want to add anything to the NEWS@Aaron Lun?
Aaron Lun (10:51:08): > No
Kevin Rue-Albrecht (10:59:05): > I’ll remove branchfurtherfixok? it was merged over a week ago
Aaron Lun (10:59:44): > okay.
Aaron Lun (10:59:55): > I’m going home.
Kevin Rue-Albrecht (11:00:30): > Ok. With that deleted, and cartes/squareplots about to merge, there’s onlyargsmodalleft in the field
Aaron Lun (11:01:11): > Feel free to do the unit tests yourself for #172.
Aaron Lun (11:01:29): > On an unrelated note@Charlotte Sonesondo you want to grab a drink while you’re in cambridge
Kevin Rue-Albrecht (11:01:32): > I was going to ask@Charlotte Sonesonabout this, I thought she was looking into it
Charlotte Soneson (11:03:53): > Sorry, I was, but this week has been completely crazy. Today I had to pack for three weeks travel and prepare the apartment for some guests, but I was hoping to get around to it soon:grimacing:. Aaron: sure! I’ll be in Hinxton but there should be a way to get to Cambridge:slightly_smiling_face:I’ll leave on Thursday so I will unfortunately not be there for your lecture.
Kevin Rue-Albrecht (11:06:08): > Cool. As fun as unit tests are, this square plot fix is a much time as I could afford this week end. Got a sci. advisory board to prepare for this week, and a lab meeting the next
Aaron Lun (11:30:58): > I will brush up the EmptyDrops tour, and we’ll have our single-cell data analysis.
Aaron Lun (11:31:18): > Let’s think of submitting this week, if the text is okay.
Kevin Rue-Albrecht (12:46:12): > I’m happy with the current manuscript. I don’t think we can shorten it to the 2-page format, and I think it is not worth the effort thinking about it more until we know whether they’re at least interested in considering it.
Kevin Rue-Albrecht (12:52:33): > On a separate note, we have 2 weeks until the devel branch freezes for the next release. I’d like to push 0.99.3 upstream by next weekend, and simply monitor the nightly report for the last week. > Apart from upping the unit test coverage, we can play with the app over the week to spot any residual issue.
2018-04-16
Aaron Lun (09:32:00): > @Charlotte Soneson@Kevin Rue-AlbrechtRename your files so that they have consistent names. i.e., cytof_tour.txt, cytof_app.R, tcga_tour.txt, tcga_app.R.
Aaron Lun (09:32:34): > I just added the PBMC tour. Doesn’t use an EHub, though, so it’s a bit of a pain (but not too hard) to reconstruct the SCE.
Kevin Rue-Albrecht (09:33:34): > Yep. I did that for the R script, didn’t bother yet for the tour steps. Will do this evening
Aaron Lun (09:34:09): > Suggest suffixing with_app.Rand_tour.txt, as this is pretty clear.
Aaron Lun (09:34:22): > tour_code.R is a bit ambiguous
Kevin Rue-Albrecht (09:40:42): > ..and suffixing pairs the two files when listed alphabetically. I’m all for it.
Charlotte Soneson (09:40:58): > Yeah, good point. Changed mine
Kevin Rue-Albrecht (16:35:01): > Anyone up for a ggplot feedback?
Charlotte Soneson (16:37:02): > Sure
Kevin Rue-Albrecht (16:37:10): > @Kevin Rue-Albrechtuploaded a file:current stateand commented: At the moment, our plots look like this, withaxis.text.xjustified to the right, and mid-height - File (PNG): current state
Kevin Rue-Albrecht (16:37:50): > I’m proposing to the center horizontally, and justify to the top
Charlotte Soneson (16:38:11): > Agreed! And actually I thought I did that…have you merged theargsmodalbranch?
Kevin Rue-Albrecht (16:38:11): > @Kevin Rue-Albrechtuploaded a file:like that - File (PNG): like that
Charlotte Soneson (16:39:03): > here:https://github.com/csoneson/iSEE/commit/b0883dc7de13fa75d8c7f37211daa3b7b46594fb - Attachment (GitHub): Fix justification of rotated axis labels · csoneson/iSEE@b0883dc > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (16:39:05): > I think that branch is still open. I haven’t worked on it, so I didn’t push for a PR yet
Kevin Rue-Albrecht (16:39:17): > ah:sweat_smile:
Charlotte Soneson (16:39:25): > It’s still open…I’m working on the unit tests now:slightly_smiling_face:
Kevin Rue-Albrecht (16:39:47): > awesome. thanks for both the fix and the unit tests then!
Kevin Rue-Albrecht (16:40:59): > Onceargsmodalis in, I guess we’ll be ready to push upstream then.
Charlotte Soneson (16:41:26): > Yes, probably
Charlotte Soneson (17:46:34): > Ok, I added some unit tests for.report_memory()and fixed some things that came up while running them…
Charlotte Soneson (17:47:10): > Inargsmodalbranch if nayone wants to have a look before merging
Aaron Lun (18:26:40): > When are you in cambridge@Charlotte Soneson?
Aaron Lun (18:36:22): > Tests look good, but still missing some tests for.deparse_for_viewing(), which is a fancy version ofdeparsethat makes pretty indented code.
Charlotte Soneson (18:36:42) (in thread): > I’m here until Thursday afternoon. Gabry was thinking of organizing a dinner in Cambridge on wed evening after the course, she said she’d contact you as well. That might be the most practical (unless you happen to be in Hinxton for other reasons before Friday)?
Aaron Lun (18:37:49) (in thread): > I’m at Hinxton tomorrow… god forbid…
Charlotte Soneson (18:37:51): > You wanted to test the printing itself? I figured that the code it was generating was ok, since it is called from the other fcn.
Aaron Lun (18:38:16): > Yeah, I guess that’s true.
Aaron Lun (18:38:30): > Yeah, maybe it’s fine.
Charlotte Soneson (18:38:35) (in thread): > Ah ok. Well, then tomorrow is an option too.
Aaron Lun (18:39:47) (in thread): > Well, if Gabry organizes a dinner, that would also work well for me.
Aaron Lun (18:39:56) (in thread): > Let’s see what she ends up doing.
Aaron Lun (18:40:13): > Okay, well, merge in the PR.
Charlotte Soneson (18:40:28) (in thread): > Yep. I’ll check with her tomorrow.
Charlotte Soneson (18:41:31): > Will do tomorrow morning, computer is off for today:slightly_smiling_face:
2018-04-17
Charlotte Soneson (01:54:17): > Ok, PR merged
Aaron Lun (08:33:17): > Any luck with Gabs + dinner? Or should we pub tonite
Charlotte Soneson (10:14:03) (in thread): > You should have (or soon will) received an email - dinner for tomorrow is still on according to Gabry
Aaron Lun (10:46:22) (in thread): > okay good. Didn’t get an email yet, did she send it out.
Charlotte Soneson (11:44:45) (in thread): > Don’t know, she just said she would send you a message soon. She’s not here today so I can’t ask in person.
Aaron Lun (11:45:24) (in thread): > hm.
Charlotte Soneson (11:48:04) (in thread): > This is literally what she wrote 2 hours ago: > > We are indeed > > Will drop him an email shortly > > G >
Aaron Lun (11:50:12) (in thread): > okay
2018-04-18
Aaron Lun (04:59:37): > If someone has some free time, could they add a README for iSEE2018? It should also point towards the live instances of the shiny apps.
Kevin Rue-Albrecht (05:05:38): > I’ll be offline most of today and tomorrow: SAB (with John and Mike, among others)
Aaron Lun (05:07:16): > Make sure you call him “Viper”
Aaron Lun (05:07:31): > Just say, “good morning, viper”
Kevin Rue-Albrecht (05:07:49): > that’ll be an ice breaker:joy:
Kevin Rue-Albrecht (07:24:12): > btw, shall we delete the3.4branch? Last night, I tried to update it on my fiancée’s (Anna, in short) laptop, and it crashed because of a missingsizeFactorNamesthat was introduced only recently (https://github.com/drisso/SingleCellExperiment/commit/9274bec30e411dbe7a8aa50e5ffdf68ace4c49bd) - Attachment (GitHub): Rebuilt NAMESPACE with new Roxygen tags. · drisso/SingleCellExperiment@9274bec > Clone of the Bioconductor repository for the SingleCellExperiment package, see https://bioconductor.org/packages/devel/bioc/html/SingleCellExperiment.html for the official development version.
Federico Marini (07:26:01): > I would be pro-deletion. In a couple of weeks it would become anyway even weaker. And I think it is our right (plus, duty) to work with the latest stand of pkgs, especially in the sc- field
Kevin Rue-Albrecht (07:27:59): > That’s my thinking too. I like being nice to people, but we should encourage them to work with the latest version indeed.
Kevin Rue-Albrecht (07:28:42): > Plus R 3.5 is even more up to date than R-devel ^1(http://r.research.att.com)
Charlotte Soneson (07:28:49): > Yeah, I agree too.
Kevin Rue-Albrecht (07:29:56): > alright. done
Federico Marini (07:31:43): > probably an overkill, but we could trigger the build on the RC version so hat we know we’e fine fine fine?
Kevin Rue-Albrecht (07:32:12): > i think I know what you mean, but what does RC actually stand for?
Federico Marini (07:32:17): > release candidate
Federico Marini (07:32:26): > at leasthere
Kevin Rue-Albrecht (07:32:45): > alright, so pushing upstream you mean?
Federico Marini (07:33:08): > only github-travis could suffice, I think
Kevin Rue-Albrecht (07:33:24): > owww, gotcha then
Federico Marini (07:33:27): > of course the whole Bioc infrastructure would be even better
Federico Marini (07:33:51): > checking the list of things we wanted on top.. we’Re good to go for 0.99.3 anyway
Federico Marini (07:34:24): > unless any urgent bugfix or wish comes in?
Kevin Rue-Albrecht (07:39:17): > In terms of features, I also believe that we’re there. Haven’t spotted any obvious bug recently, and Charlotted fixed the vertical X tick labels, so I’ve got nothing to add
Kevin Rue-Albrecht (16:41:58): > I’m heading back from the conference dinner just now. How many votes for pushing 0.99.3 upstream tonight?
Kevin Rue-Albrecht (16:43:32): > I think it wouldn’t hurt. Worst case it’ll flag things to fix up in the next few days before the release freeze
Kevin Rue-Albrecht (16:44:02): > I don’t see any benefit waiting
Kevin Rue-Albrecht (16:45:04): > <!channel>: see above. Sorry for the late poke
Charlotte Soneson (16:46:28): > Sounds good. Aaron says yes too.
Kevin Rue-Albrecht (16:47:30): > Cool. Tell Aaron viper and John say hi
Federico Marini (16:49:37): > pro pushing
Kevin Rue-Albrecht (16:51:08): > Unrelated: seeing a faceted plot today reminded me we haven’t considered that yet.
Aaron Lun (19:25:15): > Do we need that?
Aaron Lun (19:25:30): > The panels are the facets.
2018-04-19
Kevin Rue-Albrecht (02:53:03): > I wouldn’t say weneedthe feature. My other worry is about space in the UI, even though the panels can be stretched in height and width.
Kevin Rue-Albrecht (02:55:00): > With that in mind, faceting achieves something that multiple panels can’t: combining interactions of the factors in the current X, Y, and color aesthetics with up to another two factors in a single view
Kevin Rue-Albrecht (02:56:40): > e.g. rather than coloring multiple clusters on a single t-SNE, one can split cells back by cluster and condition, in row and column facets, for instance, making each facet more readable
Kevin Rue-Albrecht (02:57:48): > I’ll try to make and share an example plot, on the bus to day 2 of the meeting
Kevin Rue-Albrecht (03:12:41): > although I remember now that we did try some faceting, early on, and dropped it because point selection was restricted to one facet at a time (which isn’t necessarily bad, depending on use cases)
Kevin Rue-Albrecht (03:20:36): > @Kevin Rue-Albrechtuploaded a file:faceted plot, example - File (PNG): faceted plot, example
Kevin Rue-Albrecht (03:22:20): > @Kevin Rue-Albrechtuploaded a file:slightly reduced point size, for readability - File (PNG): slightly reduced point size, for readability
Kevin Rue-Albrecht (03:23:58): > Two things about this: the factors of the faceting are not shown (here,tumor_status ~ gender
Kevin Rue-Albrecht (03:24:55): > Two (that I’ll check now): I can’t remember if the legend points are that small in the current app. Their color is a pain to read here because of the small point size.
Kevin Rue-Albrecht (03:39:26): > Another hurdle would be zooming. Which would have to be restricted to a single facet, and probably be told which facet to zoom in
Aaron Lun (04:21:16): > This only works if point selection is still active. Where are the points being selected from here?
Kevin Rue-Albrecht (05:26:35): > Tea break. If you’re talking about point selection for zoom, the points (more specifically the shiny brush area) would be selected from a single facet. Then we’d have to tell coord_cartesian which facet (I think they’re called domains in terms of brush data structure) and which coordinates in the domain.
Kevin Rue-Albrecht (05:27:53): > But honestly, this all sounds a massive amount of work. Not sure it’s worth it anytime soon
Aaron Lun (05:28:04): > Sounds pretty painful to me.
Aaron Lun (05:28:13): > People would have to remember which panel is used for brushing
Aaron Lun (05:28:41): > if we set it to the top-left one, it would be weird
Aaron Lun (05:28:58): > because you’d be brushing on an empty plot, and hoping that it shows up elsewhere.
Aaron Lun (05:29:04): > Maybe it’s best as a receiver plot
Kevin Rue-Albrecht (05:40:15): > I was thinking about receiver plot as well, as a simplest option. Could be a start. Anyway, good thing to keep in mind, doesn’t need to be a priority for now
Kevin Rue-Albrecht (05:40:49): > I kinda like the kind of implicit subsetting that facets do
Kevin Rue-Albrecht (05:41:05): > that would be my main interest for them
Aaron Lun (05:42:16): > So only one of the facets is brushable? Or all of them?
Kevin Rue-Albrecht (05:43:23): > AFAIK all of them are brushable, but a single brush is restricted to a single facet
Kevin Rue-Albrecht (05:43:39): > (i.e. the next brush can be on another facet)
Aaron Lun (05:44:08): > oh, that’s fine.
Aaron Lun (05:44:11): > That’s easy, then.
Aaron Lun (05:44:26): > Just add a “facetable” button to all of the plots.
Aaron Lun (05:45:38): > + what to facet by, of course.
Aaron Lun (06:36:50): > @Charlotte Sonesonlooking at the HCA schedule. Suggest one of us attend “Tools and best practices for modern web-based scientific data visualization” on Wednesday. I will probably have to go to “Consuming data from and building on top of the DCP” instead.
Charlotte Soneson (06:37:19): > Yes, I was thinking of doing the visualization one anyway, so I’m happy to go there
Aaron Lun (06:37:25): > good
Federico Marini (08:25:36): > –> on the line of other works in this direction:https://www.biorxiv.org/content/early/2018/04/18/302554.1 - Attachment (bioRxiv): STREAM: Single-cell Trajectories Reconstruction, Exploration And Mapping of omics data > Single-cell transcriptomic assays have enabled the de novo reconstruction of lineage differentiation trajectories, along with the characterization of cellular heterogeneity and state transitions. Several methods have been developed for reconstructing developmental trajectories from single-cell transcriptomic data, but efforts on analyzing single-cell epigenomic data and on trajectory visualization remain limited. Here we present STREAM, an interactive pipeline capable of disentangling and visualizing complex branching trajectories from both single-cell transcriptomic and epigenomic data.
Aaron Lun (08:28:24): > Very derived results
Aaron Lun (08:28:35): > as in the visualization and analysis are very tightly coupled
Aaron Lun (08:28:39): > not particularly flexible
Federico Marini (08:37:48): > all true
Federico Marini (08:38:17): > I just find it even more fitting that while everyone is busy doing their own viz-tool, we have a nice hammer for many nails:stuck_out_tongue:
Kevin Rue-Albrecht (08:50:58): > Btw I pushed upstream as agreed last night. Look out for 0.99.3 in the next 48h
Kevin Rue-Albrecht (08:52:17) (in thread): > A Swiss Army knife I’m sure some of us would say:stuck_out_tongue_winking_eye:
Federico Marini (08:54:19) (in thread): > @Aaron Lunout of curiosity, did you try to “wrap in iSEE instances” the two separate apps you sent us back then?
Aaron Lun (08:54:41) (in thread): > huh
Federico Marini (08:55:06) (in thread): > I was just thinking of that as an “internal proof” for us to showcase the flexibility:slightly_smiling_face:
Aaron Lun (08:55:36) (in thread): > ??? Wrap in what?
Aaron Lun (08:55:46) (in thread): > ich bien do not understand
Federico Marini (08:56:12) (in thread): > iSEE_organogenesisif you want
Aaron Lun (08:56:37) (in thread): > Oh, the apps that were not mine.
Aaron Lun (08:56:59) (in thread): > One of my other colleagues is currently submitting, I can ask if she would be interested in making an interactive app for the reviewers.
Federico Marini (08:56:59) (in thread): > yep I recall a spanish name as an author
Federico Marini (08:57:05) (in thread): > ok
Federico Marini (08:57:56) (in thread): > that would be quite cool. I was asking because a colleague of mine spoke out loud what I thought, i.e. “you can use this app to accompany any publication”
Aaron Lun (13:34:26): > Looking at HCA visualization tools:
Aaron Lun (13:34:26): > https://xenabrowser.net/heatmap/
Aaron Lun (13:34:46): > a bit meh, but okay.
Kevin Rue-Albrecht (13:38:29) (in thread): > Just wondering how exactly the app would be distributed with the publication. A your script is pretty simple. The limiting factor is rather the size of the dataset, an RDS file would probably be too large in most cases. Are we talking about a use case for the ExperimentHub?
Aaron Lun (13:38:59) (in thread): > I would set it up on an institute’s shiny server
Aaron Lun (13:39:11) (in thread): > Just allow people to refer to it from online resources
Kevin Rue-Albrecht (13:41:18) (in thread): > Right. So leaving the responsibility of making persistent apps up to the institute
Aaron Lun (13:41:31) (in thread): > or journal, should one be so forward-minded.
Kevin Rue-Albrecht (13:42:55) (in thread): > Yeah. UntiliSEE.shiny.bioconductor.orgsees the day
Kevin Rue-Albrecht (13:43:08) (in thread): > :stuck_out_tongue:
Kevin Rue-Albrecht (13:43:37) (in thread): > Kind of a catchy name, if you ask me:rolling_on_the_floor_laughing:
2018-04-20
Federico Marini (06:16:20) (in thread): > later catch-up, but… yeah, fascinating option:slightly_smiling_face:
Federico Marini (11:14:09): > It is a little clunky, but it is not that bad. It seems nicely generalizable
Federico Marini (11:15:05): > can we ask some money here retroactively?
Federico Marini (11:15:06): > https://www.chanzuckerberg.com/human-cell-atlas/comp-tools - Attachment (The Chan Zuckerberg Initiative): Human Cell Atlas - The Chan Zuckerberg Initiative > Engineering tools to map the basic units of life. The goal of the Human Cell Atlas project is to create a shared, open reference atlas of all cells in the healthy human body as a resource for studies of health and disease. This global endeavor will generate a large variety of molecular and imaging data …
Federico Marini (11:15:37): > oh well, johnisin there
Federico Marini (11:18:00) (in thread): > http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/
Federico Marini (11:18:03) (in thread): > no probs
Kevin Rue-Albrecht (11:20:45) (in thread): > It’s good having it out for the tours; I naively ran the tour on the bioc-devel 0.99.2 to realise that, the night that I pushed upstream
Aaron Lun (11:50:19): > I will try to submit this weekend. IfBioinformaticssays we need to get it under two pages… I don’t see how it can be feasibly done. So I would suggest that we try F1000Research instead.
Aedin Culhane (12:43:05): > @Kevin Rue-AlbrechtRe:uploaded this image: slightly reduced point size, for readability maybe use different shapes in addition to colors?
Kevin Rue-Albrecht (12:50:08): > Actually, yes, we could reinforce color with shape easily. I think we just left the shape aesthetic out of the equation for now, as it doesn’t help much with many data points
2018-04-21
Aaron Lun (05:47:55): > @Kevin Rue-Albrecht@Federico MariniI was talking to@Charlotte Sonesonover dinner and we were wondering whether it would just be better to submit to F1000Research in the first instance, given our collective bad experiences with BANs. I know a couple of people on the editorial board, and they may be open to hosting a Shiny server themselves. And we wouldn’t have strict page limits either.
Kevin Rue-Albrecht (07:02:08): > I have no experience whatsoever submitting F1000 but i know you do, so I trust you on that one. Plus, the online format of F1000 seems a better fit for the interactive features of iSEE. I don’t see any other place that would evenconsiderhosting a demo
Kevin Rue-Albrecht (07:09:48): > while i’m online, a core staff at the WTCHG, who is very enthusiast about iSEE brought up again the point that custom panel names would be nice. > Personally, I’m not convinced about that: between the current title stating the panel type, and the plot title stating the covariate plotted, i don’t think we want to help people turning panel names into figure legends or what not, do we?
Aaron Lun (07:11:11): > Looking at my F1000 papers, according to Google there’s 42, 22, 5 and 6 citations, the average of which already beats Bioinformatics’ IF. Albeit google is very generous with citations.
Aaron Lun (07:11:29): > Re panel names: no, I can’t see the benefit in doing so.
Aaron Lun (07:11:41): > Probably ought to add some words to the vignette about that.
Kevin Rue-Albrecht (07:20:35): > Nice point about the citation. I’ve never fully bought into the hype of impact factors anyway. Places like F1000 would give us all the visibility that we need/want anyway. Then, it’s up to users to do the rest (i.e., cite).
Kevin Rue-Albrecht (07:21:26): > Isn’t it the thing that IFs tend to be driven by the few highly cited papers; with the other lot being published in then high IF journal but not cited that much?
Kevin Rue-Albrecht (07:24:30): > Anyway. Re panel names, it just crossed my mind the mess that would ensue from the interactivity: who would bother updating a custom panel title every time an input is changed?
Aaron Lun (08:04:35): > Yes. That was the problem that I always run into.
Aaron Lun (08:49:24): > In any case, I sent an email to Thomas Ingraham (F1000Res editor) asking if they would be able to host the Shiny apps on their end. If that’s possible, I’m definitely going to recommend submitting there - imagine being able to open the apps inside the article! Insane.
Kevin Rue-Albrecht (09:51:25): > Absolutely! Interactive plot are one thing that already exist, but embedding apps into an article would be just awesome
Kevin Rue-Albrecht (09:52:14): > Basically, VR into published data sets
Kevin Rue-Albrecht (09:54:07): > I wonder if in the end it could just be a matter of embedding an ‘iframe’ to a shiny server into online articles
Aaron Lun (10:23:35): > Turns out Thomas doesn’t work there anymore, so I sent it to Vicky Hellon instead.
Aaron Lun (12:42:48): > Added a README,@Kevin Rue-Albrechtand@Charlotte Sonesonfill in instructions to run your tours.
Federico Marini (14:18:09): > Catching up late, but still…
Federico Marini (14:18:51): > I also think it could be a mess - worth doing? - to cut snippets here and there if we really have to stick to the 2pages
Federico Marini (14:19:42): > btw thomas ingraham would have been my contact of preference too:stuck_out_tongue:
Aaron Lun (14:23:03): > IMO all the fat is already gone - further cuts would start hitting the bone.
Federico Marini (14:25:59): > same fear for me. we risk having chopped the meaning just for the sake of saving couple of words
Federico Marini (14:26:17): > we still have the extra line for the associate editor thingy
Federico Marini (14:26:32): > and we “could” ask for a grace due to the many affiliations
Federico Marini (14:27:01): > plus, we could still do the trick of having the citations limited to first, et al
Aaron Lun (14:28:39): > Well, let’s see what Vicky says. If F1000 is willing to host the apps on their end, that seals the deal for me.
Federico Marini (14:28:58): > that would be reeeeally cool
Federico Marini (14:29:16): > given the fact many are intending to support plotly here and there
Federico Marini (14:29:20): > it could be possible
Federico Marini (14:36:53): > anyway, long story short. I’m up for anything that benefits our work
Federico Marini (14:38:16): > impact factor per se would only mean a couple of these points in the bloody LOM points system (for those unfamiliar: LeistungOrientierteMittelvergabe, i.e. at the uni, people get more money for the institute the more if points they accumulate)
Kevin Rue-Albrecht (15:08:04): > quick note@Charlotte Soneson: i found a typo (missing,) in the tour script:https://github.com/LTLA/iSEE2018/blob/master/tours/cytof_app.R#L2
Kevin Rue-Albrecht (15:08:17): > i’ll run the rest now to check
Kevin Rue-Albrecht (15:11:19): > there’s one more thing. Hold on, I’ll send in a PR when I’m done
Aaron Lun (17:51:06): > @Federico MariniThat LOM system seems absolutely stupid
Aaron Lun (17:52:16): > It’s hard enough writing grants and all. Now they score you on IFs? That’s insane.
Aaron Lun (17:53:50): > @Kevin Rue-AlbrechtWhen you describe your tour, put it above the PBMC one.
Kevin Rue-Albrecht (17:54:29): > ok, i was just doing thatafterthe PBMC, i’ll move it up then
Kevin Rue-Albrecht (17:54:58): > not that I have much to write. the script works as is. Only need to setwd to the tours folder.
Aaron Lun (17:55:39): > I wonder what the set-up would be if F1000Research has their own server. Would ExperimentHub work properly there?
Kevin Rue-Albrecht (17:55:58): > hm .. that depends on them I guess
Aaron Lun (17:56:06): > As in, where would the data get saved locally?
Kevin Rue-Albrecht (17:58:45): > Well, in default settings, it would indeed. I suppose one could set the local download to/tmp, but that would not be particularly useful
Kevin Rue-Albrecht (17:59:27): > My doubt is whether the local cache would be the same for all instances of the app
Kevin Rue-Albrecht (18:00:15): > i mean the cache is user-specific, and all instances should be run by the same user, I assume
2018-04-22
Kevin Rue-Albrecht (05:55:16): > Hi all, let me know what you think of the updates to iSEE2018. > I’ve added a couple ofNote, which may be better written differently. Feel free to edit/remove if it’s too obvious
Aaron Lun (05:58:35): > Looks pretty good to me.
Federico Marini (08:05:41) (in thread): > that lom system is a plague
Federico Marini (08:05:55) (in thread): > sadly very much spread throughout germany
Federico Marini (08:06:09) (in thread): > even thougher for us being in a medical center
Federico Marini (08:06:52) (in thread): > if I am not mistaken: > 1/3 of the points to the first author, > 1/3 to the last,
Federico Marini (08:07:14) (in thread): > 1/3 to all whoever is in between, of course spread out if more are there
Aaron Lun (09:19:32) (in thread): > What about joint first/last?
Federico Marini (10:22:11) (in thread): > depends
Federico Marini (10:22:20) (in thread): > one internal, one external: full to the internal
Federico Marini (10:22:25) (in thread): > two internals: split
Federico Marini (10:22:36) (in thread): > VERY ENCOURAGING for intramural coops
Federico Marini (10:23:05) (in thread): > I feel some shame in reporting this. but this is how many uni medical centers tick
Aaron Lun (10:23:12) (in thread): > jesus
Charlotte Soneson (11:00:05): > I added a description of the CyTOF tour to the README.@Kevin Rue-Albrecht- thanks for spotting the typo in the script! Sorry for the slow reaction, I’m a bit out of sync these days while touristing in San Francisco:slightly_smiling_face:
Kevin Rue-Albrecht (11:13:19): > No worries. It was also good practice for me to read through your script - I still haven’t tested out the PBMC 4k, btw. Did anyone else than Aaron run the tour script?
Kevin Rue-Albrecht (11:14:55): > One of the joys of small academic groups: I’m up for lab meeting again this week -_-
Aaron Lun (13:45:17): > The sce should be available from Dropbox somewhere. I think I gave Fed the link.
Aaron Lun (13:45:26): > Well, I must have, because otherwise we wouldn’t have transferred it.
Federico Marini (14:50:25): > hehe
Federico Marini (14:50:28): > lemme check
Federico Marini (14:52:26): > (unrelated,@Charlotte Soneson: I hope lior’s new anger round is on you guys)
Federico Marini (14:55:18): > so, I can’t recall how i did get the sce object from you, aaron
Federico Marini (14:55:29): > I guess you shared a dropboxx link?
Federico Marini (14:58:02): > but otherwise now it is stated how to get it
Federico Marini (14:59:24): > I can try and get the three demos running one of these days
Federico Marini (14:59:48): > so that they are also reachable in the 1-click mode, provided reviewer or editors are lazy
Federico Marini (15:40:06): > Ok, found it.@Kevin Rue-Albrecht, this contains the original version - Attachment: Attachment > See app materials on https://www.dropbox.com/sh/ohw3ikibo1ymku4/AADT6idzq7K0RgAGkE18e-mKa?dl=0
Kevin Rue-Albrecht (17:22:20): > argh thanks Fede. I had forgotten about that folder. Found the sce now. Cheers!
2018-04-23
Federico Marini (12:14:21): > had a meeting with a collaborator today and had the chance to run the tours live
Federico Marini (12:14:48): > positive feedback. Both from their side, as well as from my side from the 0-bug usability
Federico Marini (12:15:14): > plus a bravo to@Aaron Lunfor the idea of marking clearly theactions to be taken by the user
Federico Marini (13:30:39): > Sooo. For the lazy ones, and for our commodity: here’s a couple of links for showcasing our creaturehttp://shiny.imbei.uni-mainz.de:3838/iSEE_PBMC4k/http://shiny.imbei.uni-mainz.de:3838/iSEE_cytof/
Federico Marini (13:30:47): > the one for the TCGA is coming up too
Federico Marini (13:31:05): > the only thought is that it is quite a biggy set to load
Federico Marini (13:31:38): > about 1G
Federico Marini (13:32:03): > I am permissive with the timeout when the app is loading, but this is putting some load indeed
Kevin Rue-Albrecht (13:43:37): > Thanks for the effort, in any case!
Federico Marini (13:54:47): > There goes the TCGA one too. Keep in mind, the machine where the whole things run is far from a hi profile one
Federico Marini (13:54:49): > http://shiny.imbei.uni-mainz.de:3838/iSEE_TCGA/
Federico Marini (13:58:35) (in thread): > for the pbmc4k, I got the most recent version
Kevin Rue-Albrecht (14:02:13): > no news from the F1000 yet, btw?
Federico Marini (14:03:14): > maybe aaron knows more
Aaron Lun (21:10:24): > nope
2018-04-24
Aaron Lun (09:30:29): > > Hi Aaron, > > > > Great to hear from you and no worries at all about the workflow, I can imagine its a hectic time of year! The paper sounds really interesting. I’ve spoken to our production and dev teams about potentially integrating this (as you said we like to be accommodating!). We did look into shiny apps before but unfortunately we can’t host the apps on our own servers primarily due to security reasons (plot.lyenables us to create the smart URL to embed the interactive figure but not host it on our own servers). Something we can do though is have a HTML iframe in the article which pulls the image. We’ve done one case of this on our Wellcome platform-https://wellcomeopenresearch.org/articles/2-26/v1which is pulling this-https://livedataoxford.shinyapps.io/1488296544-b93rqzpw1qgokzg/Although obviously this means the apps stays on your own server unfortunately. Also what is in the iframe doesn’t go to pubmed, as it isn’t part of the article so we also have the data behind the visualisation on that Wellcome example in Figshare. > > > > Let me know if this is something that could work at all- would be great to host the apps in this capacity if this is feasible for you.
Aaron Lun (09:32:27): > So, we don’t get everything we asked for, but we’re pretty close.
Aaron Lun (09:32:56): > I can host the apps on the CI shiny sever next week, which might be more stable than@Federico Marini’s.
Aaron Lun (09:33:07): > I say next week because we want to wait for the release…
Aaron Lun (09:33:25): > In the meantime, can@Federico Marinicheck how ExperimentHub behaves on a Shiny server? Will it try to download it fresh every time?
Federico Marini (09:34:01): > EH seems to be good
Federico Marini (09:34:44): > I had to edit a little the code from Kevin in the app.R file because I rather load the fully-prepared sce object
Federico Marini (09:35:02): > instead of doing the processing from EH1 to sce “live”
Kevin Rue-Albrecht (09:35:12): > right, i was getting bothered re-running the tSNE every time
Federico Marini (09:35:29): > so, I did the operations once (irlba, tsne, …)
Federico Marini (09:35:34): > and then saved
Federico Marini (09:35:40): > btw
Federico Marini (09:35:55): > did anyone of you try out the UMAP algorithm?
Federico Marini (09:36:13): > which seems to be the new sheriff in town, in terms of dim red
Kevin Rue-Albrecht (09:36:14): > I spotted it a while, but not yet
Federico Marini (09:36:42): > therewas a recent ms on arxiv (or biorxiv) with a plausibly meaningful comparison
Charlotte Soneson (09:38:17): > Yeah, I saw that too and I wanted to try, but haven’t gotten around to it yet…
Aaron Lun (09:38:44): > I am told that the comparison is not particularly convincing, as it is just “well it looks better”.
Aaron Lun (09:39:01): > But I haven’t read it myself.
Aaron Lun (09:43:32): > @Charlotte SonesonAre you already awake? I couldn’t get any sleep here.
Federico Marini (09:44:14): > Probably apart from looking slightly better it could be that it does run faster because it seems to scale better?
Aaron Lun (09:44:36): > The speed is a definite improvement, from what I hear
Aaron Lun (09:45:00): > Though all of this dimensionality reduction stuff is a bit meh
Aaron Lun (09:45:12): > Just a competition to see who gives the prettiest pictures, really
Aaron Lun (09:45:45): > There must be so much information loss from compression into two dimensions, we’re already seeing approximations of approximations
Aaron Lun (09:49:13): > Well, perhaps it does matter. I don’t know
Charlotte Soneson (09:52:45) (in thread): > Yeah, it’s hard to sleep when it’s really afternoon…but we arrived in San Francisco on Friday so I got a bit accustomed.
Aaron Lun (10:07:16) (in thread): > you in the main building?
Charlotte Soneson (10:07:42) (in thread): > Nope, we’re in one of the villas by the ocean
Aaron Lun (10:10:15) (in thread): > Oh, I meant like, right now. I guess you’re still in your rooms, then.
Charlotte Soneson (10:10:48) (in thread): > Aha, yes, still have to get ready. I’ll be there for breakfast at 8 I think.
Aaron Lun (10:12:00) (in thread): > Is that in the main buildling? I haven’t gone inside other than to check in
Charlotte Soneson (10:13:48) (in thread): > Yeah, that’s what it sounded like yesterday. Upstairs.
Aaron Lun (10:14:06) (in thread): > ok
Aaron Lun (12:12:33): > <!channel>In any case, I think F1000 is the way to go. I know that@Federico Mariniwon’t get his LOMs points - and that’s unfortunate - but I think the ability to have the app embedded in the article is too good to pass up.
Federico Marini (12:13:02): > gotta fetch the freaking points from somewhere else:smile:
Kevin Rue-Albrecht (12:16:48): > from the F1000 reply, I was just wondering about whatwillto Pubmed and picked a random f1000 (that turns out to have 20 figures) for example:https://www.ncbi.nlm.nih.gov/pubmed/27909575 - Attachment (ncbi.nlm.nih.gov): A step-by-step workflow for low-level analysis of single-cell RNA-seq data with Bioconductor. - PubMed - NCBI > Version 2. F1000Res. 2016 Aug 31 [revised 2016 Oct 31];5:2122. eCollection 2016.
Aaron Lun (12:18:14): > Yeah, seems pretty dodgy
Kevin Rue-Albrecht (12:18:19): > Do we want to include any static figure, like the one that Aaron prepared for the BAN? Or are we going ‘full interactivity’ ?
Aaron Lun (12:18:38): > I guess we’ll have our cake and eat it tooo.
Aaron Lun (12:20:16): > Do you guys want to start fleshing out the text for a short article in F1000?
Aaron Lun (12:20:28): > @Charlotte Sonesonand I will be out of it this week
Aaron Lun (12:20:46): > stuck in San Fran at the CZI meeting
Kevin Rue-Albrecht (12:20:51): > Anyway, I’ll have to read up more about the f1000 format to give relevant suggestion as to the additional content that we can now flesh out
Kevin Rue-Albrecht (12:21:47): > Any interest in a representation of the ECM, to highlight the fallback mechanism ?
Federico Marini (12:22:43): > I’d say we can easily stress out that the app is a solution for showcasing any dataset, and point to the use with the tours. that was kept to the minimal in the BAN version
Aaron Lun (12:23:23): > You can probably talk about the ECM in 1 paragraph
Aaron Lun (12:23:33): > Yes, and you can talk about the tours in more detail
Kevin Rue-Albrecht (12:27:53): > I was just thinking how we could have a section per tour, like methods papers typically have a section per data set that they demonstrate their tool (e.g., synthetic data, data set 1, data set 2)
Aaron Lun (12:28:40): > Sure.
Kevin Rue-Albrecht (12:28:41): > Anyway. Are we working back on Google Docs to kick that off again, or is there a better place for this?
Aaron Lun (12:29:09): > Just work straight from GH. If it’s just you and Fed, there shouldn’t be too many conflicts yet. Me and CS will look at it later.
Kevin Rue-Albrecht (12:32:14): > Right. I think that’s what we’re looking for then:https://www.overleaf.com/latex/templates/f1000research-journal-article-template/tmshcjpczjbb#.Wt9b5i-ZNBx - Attachment (overleaf.com): F1000Research Journal Article Template - LaTeX Template on Overleaf > This template has been created for authors submitting articles to F1000Research. It is designed for easy editing online with Overleaf, and users can submit t…
Aaron Lun (12:33:21): > Yep. Money is not so much a matter, but don’t go overboard when you don’t need to.
Kevin Rue-Albrecht (12:34:51): > eh. sure. i’m not following the link with money though:sweat_smile:
Aaron Lun (12:35:12): > article length => cost
Kevin Rue-Albrecht (12:36:09): > Oh. right. Again, i need to read their author guidelines to avoid unnecessary questions on those aspects.
Aaron Lun (12:36:30): > I never worried about it because last two times I published in there for free.
Federico Marini (12:40:06): > can we work locally with the template from them?
Kevin Rue-Albrecht (12:40:56): > eh I’m not what you mean by that, because right now I picture you and me working ‘locally’ on a different document each
Federico Marini (12:42:07): > ok the question was unclear. do we have to use overleaf or can we use it locally no probs?
Aaron Lun (12:42:17): > Might consider chucking things on overleaf and then recommit locally once you sort out the major things
Kevin Rue-Albrecht (12:44:34): > I just got the overleaf editor open right now, and indeed, it seems like a nice place to type stuff, that we can copy-paste back into our local editor before pushing to GH. Neat.
Kevin Rue-Albrecht (12:46:18): > Thinking back about the BAN, I understand better Fede’s question: I wonder whether there are any ‘dependencies’ in terms of accompanying TeX files that we should have locally to make it compile
Aaron Lun (12:46:27): > Yeah… it gets boring after a while though, especially due to the lack of versioning.
Kevin Rue-Albrecht (12:46:39): > Although again, we would just have to paste the latest GH commit into the Overleaf editor to visualise it
Charlotte Soneson (12:46:49): > https://www.overleaf.com/help/233-how-do-i-connect-an-overleaf-project-with-a-repo-on-github-gitlab-or-bitbucket#.Wt9fWxZlCEc - Attachment (overleaf.com): How do I connect an Overleaf project with a repo on GitHub, GitLab or BitBucket? - Help and FAQs - Overleaf > There are two main ways to use Overleaf with a project on GitHub, GitLab or BitBucket: add your Overleaf project as a remote, or add it as a submodule. The i…
Kevin Rue-Albrecht (12:47:01): > … and here’s the voice of sanity (thanks@Charlotte Soneson!)
Charlotte Soneson (12:47:48): > Or laziness, depending on how you see it:wink:
Kevin Rue-Albrecht (12:49:50): > In academia, that what I call ‘sanity’. Work is complex enough as it is:wink:
Federico Marini (12:50:35): > I was asking because i could not find the whole .sty and similar
Federico Marini (12:52:15): > I guess we get to these if we do the …https://www.overleaf.com/blog/195-new-collaborate-online-and-offline-with-overleaf-and-git-beta - Attachment (overleaf.com): New: Collaborate Online and Offline with Overleaf and Git (beta) - Overleaf Blog > One of your most requested features - a link between Overleaf and Git - is now available in beta release
Aaron Lun (12:53:02): > well, you guys fiddle with it and see how you go.
Federico Marini (12:56:27): > What’S your preferred location for starting? Should we overwrite the main.tex or have a subfolder with all the F1000 stuff?
Aaron Lun (12:56:57): > Just copy the main tex into the new template and overwrite it.
Aaron Lun (12:57:08): > Commit often, etc.
Aaron Lun (12:57:15): > And new lines for new sentences!
Federico Marini (12:57:20): > :slightly_smiling_face:
Federico Marini (12:57:43): > I thought we could keep it separated from the bioinfo submission
Federico Marini (12:58:03): > so that all the styles are just in one folder for each format, blah blah.
Aaron Lun (12:58:40): > Well, there will only be one manuscript - you can make branches if you like.
Federico Marini (12:59:06): > also true:slightly_smiling_face:
Federico Marini (13:00:09): > Ok once I get their .sty file I’ll just put it in our folder
Federico Marini (13:18:56): > caption.sty is potentially conflicting with the f1000 format
Federico Marini (13:19:14): > just as a heads up in case you also got that
Federico Marini (13:25:07): > @Kevin Rue-Albrecht: i can take care of the “transmutation” to f1000res, so we do not double the efforts
Federico Marini (13:26:18): > we are currently ignoring sty files, and as of now we should all get it by hand by cloning the repo from overleaf. Should we do an exception for this case?
Federico Marini (13:26:53): > (or, is there a better way of doing this with a makefile - pinging good ol’@Aaron Lunon this)
Aaron Lun (13:27:18): > You could probablywgetit from somewhere, right?
Federico Marini (13:27:47): > I had to clone a random overleaf project
Federico Marini (13:28:08): > I guess it is also there as a file in awgettable way?
Federico Marini (13:28:15): > but i did not check deeper
Aaron Lun (13:28:20): > dunno, I’ve long forgotten.
Aaron Lun (13:28:52): > You can just chuck the sty and image onto Dropbox and wget from there for the time being.
Federico Marini (13:56:08): > As of now, first draft is here
Federico Marini (13:56:08): > https://github.com/LTLA/iSEE2018/tree/f1000res
Federico Marini (13:56:21): > ported over most of whatever we had in the BAN format
Aaron Lun (13:56:37): > don’t need the frog
Aaron Lun (13:56:41): > or the sample bib
Federico Marini (13:56:57): > Yep, these will go away
Federico Marini (13:58:05): > on overleaf, I started using this
Federico Marini (13:58:06): > https://www.overleaf.com/15765478mqsvydndbhfv - Attachment (overleaf.com): iSEE: Interactive SummarizedExperiment Explorer - Overleaf > The online platform for scientific writing. Overleaf is free: start writing now with one click. No sign-up required. Great on your iPad.
Federico Marini (13:58:24): > not compiling 100% because we miss the figure as of now
Aaron Lun (13:58:43): > yeah, looks good. Suggest fleshing out everything.
Aaron Lun (13:58:55): > Chuck in the figure and add placeholders for the tour figure locations.
Federico Marini (14:00:09): > I marked the open things with some% TODOs
Federico Marini (14:01:15): > but as it is now, the overleaf version is 0.99:1 to what i have locally
Aaron Lun (14:01:37): > ok
Federico Marini (14:02:03): > can you all view/edit the link I sent?
Federico Marini (14:11:45): > (can we switch to author-year cit style? It always makes it easier to read IMHO)
Charlotte Soneson (14:36:41) (in thread): > Seems to work for me
Kevin Rue-Albrecht (14:55:44): > I can fiddle with the link that you shared above, but I’m not signed in yet. I suppose that doesn’t matter if we track it on GH.
Federico Marini (16:04:41): > I see overleaf as of now as a platform to just check that what we are compiling without troubles also compilestherewithout trouble
Aaron Lun (17:46:08): > FIne with author/year, see what F1000 uses tho.
Federico Marini (17:50:10): > it is likely at our discretion. there’S a commented line in the template
Aaron Lun (17:52:46): > Is that the case in the final published articles?
Federico Marini (17:54:43): > need to check. but at least we can use it while writing
Charlotte Soneson (17:55:09): > > References can be listed in any standard referencing style as long as it is consistent between references within a given article. >
Charlotte Soneson (17:55:16): > https://f1000research.com/for-authors/article-guidelines/software-tool-articles
Aaron Lun (17:55:33): > hm.
Aaron Lun (17:55:43): > okay, well, knock yourself out then.
Federico Marini (17:56:04): > being the numbers the default one, probably many dont bother changing it
Federico Marini (17:56:34): > i just pleasantly see name-year so that I dont have to turn a couple of pages to see what works it refers to
Aaron Lun (17:56:54): > I like numbers.
Charlotte Soneson (17:58:09): > I usually find it easier with the name-year style
Kevin Rue-Albrecht (18:20:02): > I tend to agree with Charlotte and Fede that I prefer author-year more while writing (and reading), but then I usually set it back to the default for the submitted version (as most journals wouldn’t accept it otherwise anyway). > Looking at the overleaf manuscript now, I’d say the numbered references might be look more compact and make the text easier to read, given the two-column format.
Aaron Lun (18:21:48): > YEAH!
Aaron Lun (18:21:55): > NUMBERS FTW
2018-04-25
Federico Marini (03:17:51): > nein nein nein nein nein!:smile:
Federico Marini (03:18:19): > btw. somewhat positively jealous of you guys involved in the nice CZI thingo
Federico Marini (03:18:31): > but proud you are there holding the bioc flag
Davide Risso (11:58:39): > And Martin mentioned iSEE during the presentation! So get that paper out! ;)
Kevin Rue-Albrecht (11:59:17): > daaaaamn… any streamed video of the talk?
Federico Marini (12:08:39): > :love_letter:martin
Federico Marini (12:08:40): > :slightly_smiling_face:
Federico Marini (12:09:14): > there has to be a video… after all it is from the facebook group, it would be strange if theydon’trecord data:stuck_out_tongue:
Aaron Lun (12:09:21): > lol
Federico Marini (12:09:52): > I noticed a couple more of stargazers, btw
Federico Marini (12:10:03): > so the shoutout could have already done something
Kevin Rue-Albrecht (12:10:16): > ah. been a while since i didn’t see 26 go up
Kevin Rue-Albrecht (12:10:32): > oh no wait. i’m up to date then
Federico Marini (12:10:44): > oh
Federico Marini (12:10:54): > I thought a couple left, and few new came
Kevin Rue-Albrecht (12:11:15): > still. the funniest thing that I saw was the peak in visits to the GH the day that we tweeted about it for the first time
Davide Risso (12:11:37) (in thread): > They record data but they keep it for themselves and later sell it
Davide Risso (12:11:49) (in thread): > so no video of the talk for us to see
Charlotte Soneson (12:11:54): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-04-25 at 18.11.11.pngand commented: “New”:slightly_smiling_face: - File (PNG): Screen Shot 2018-04-25 at 18.11.11.png
Kevin Rue-Albrecht (12:12:17): > LOL
Kevin Rue-Albrecht (12:12:44): > did you toggle off and on again, Fede?
Kevin Rue-Albrecht (12:12:46): > :stuck_out_tongue:
Federico Marini (12:12:56): > eheheh
Federico Marini (12:13:08): > the mobile version allows only to star
Federico Marini (12:13:15): > and not view the gazers
Federico Marini (12:13:17): > :disappointed:
Federico Marini (12:13:30): > and I would not be that unbiased if I starred that
Kevin Rue-Albrecht (12:13:55): > oh i got caught with that as well. Though I seem to remember that holding does show the *gazers
Federico Marini (12:14:07): > aah+
Kevin Rue-Albrecht (12:15:40): > oh no, i know what i did now: scroll to the bottom, click “Website version” and then you get access to the button to view them
Kevin Rue-Albrecht (12:16:21): > btw, a 27th one just appeared.. but don’t get excited.. its ’just me:joy:
Federico Marini (12:40:57) (in thread): > schade schade marmelade…
Federico Marini (12:41:05) (in thread): > they really want to sell it?!
Federico Marini (12:41:20) (in thread): > none of you sneaking in with your phones?:stuck_out_tongue:
Aaron Lun (23:06:18) (in thread): > Was thinking of this@hcorrada. It would be great if we could have a “genome browser” panel in our iSEE interface, allowing us to scan and select features (and show row-level data) by genomic coordinate. Is this even remotely possible? Do the Websockets play nice (or at all) with Shiny?
2018-04-26
Charlotte Soneson (00:12:45): > main example used during the single-cell visualization workshop tonight:http://tabulamuris.cxg.czi.technology/
Aaron Lun (00:36:23): > Nothing new.
Aaron Lun (00:36:38): > Probably a lot faster.
Aaron Lun (00:46:02): > But if you’re downsampling, it doesn’t really matter anyway
Aaron Lun (01:36:29): > Peeps: let’s push this damn MS forward. I want to submit immediately after the release, this space is getting crowded and I want us to put down a flag to mark the spot.
Kevin Rue-Albrecht (04:06:36): > Right. I’ll have a proper look at the manuscript tonight. Got distracted by reviews that came back yesterday for a manuscript that my PhD group has been submitting for the past couple of years -_-
hcorrada (05:20:22) (in thread): > we (w/@Jayaram Kancherla) have been working on components that make it easy to embed epivizr visualizations into arbitrary html. So far, we have rmarkdown all set, we ran into some issues with shiny (we want to run the data exchange through shiny’s websocket connection instead of opening our own) but working with you on this would give us impetus to fixing that. Any dataset/analysis we should use as motivating example? We can have a discussion around it once we start thinking of options.
Aaron Lun (12:19:41): > Was talking to@Charlotte Sonesonyesterday. There’s a couple of features we should look to adding, that I was kind of reluctant to do at first but I guess is sort of unavoidable: (1) re-dimensionality reduction of a restricted subset (based on user-supplied functions), and (2) better and formalized support for gene sets.
Kevin Rue-Albrecht (12:49:50): > Both make sense. I’ll spend time on the manuscript tonight though
Aaron Lun (14:09:01) (in thread): > Sounds encouraging. We’re mostly working with genes ATM, so our thoughts on visualization would be in the direction of getting a genomic track with some gene models. The ideal would be to be able to select genes and have those selections passed to the Shiny server. Being able to visualize a few tracks of data would also be nice (though we still have to think about how to do that for genes, as we’ll be usually only having one datapoint for the entire gene).
Kevin Rue-Albrecht (14:27:56): > Can any of you compiledmain.texlocally? I’m running into errors likeunknown option 'labelsep=period' for package 'caption'
Kevin Rue-Albrecht (14:28:34): > might be mytexlive/2016installation that’s out of date?
Aaron Lun (14:29:29): > Haven’t tried yet. Probably your tex if it’s working on OL.
Kevin Rue-Albrecht (14:38:05): > argh.. kicking off 5 GB of TexLive 2017 over the internet.
Kevin Rue-Albrecht (14:38:48): > I genuinely hope that solves the issue. Anyway, in the meantime dinner and I’ll work on OL
Federico Marini (14:47:37): > nooooo
Federico Marini (14:47:40): > wait kev
Kevin Rue-Albrecht (14:47:57): > is the online thing saving the state?
Federico Marini (14:48:08): > you need to delete the caption.sty leftover from the ban template
Federico Marini (14:48:26): > it does AFAIK
Federico Marini (14:48:36): > but this one is just a nasty tex error
Kevin Rue-Albrecht (14:49:21): > ah ok you scared me, i was typing on OL, and i thought you were freaking out because it might have been making new versions of the manuscript. But I just checked and the version history still only lists your changes
Federico Marini (14:49:44): > that was one of the reasons for pushing pro- separate folders
Federico Marini (14:49:56): > my nooooo was for the full texlive
Federico Marini (14:50:01): > :stuck_out_tongue:
Kevin Rue-Albrecht (14:53:48): > it’s working locally, now that I deleted thecaption.styfile@Aaron Lunand@Charlotte Sonesonmight want to take note of this one, for sanity
Charlotte Soneson (14:54:30): > So are you editing locally and pushing to Overleaf in the end, or editing directly on Overleaf?
Federico Marini (14:55:19): > I started locally for the bulk
Federico Marini (14:55:48): > and used overleaf just as a check for “it’S working also online”
Federico Marini (14:56:10): > OL has not that big support for version control
Kevin Rue-Albrecht (14:56:13): > i’m also planning to edit locally and also compile locally for checking before pushing to GH
Kevin Rue-Albrecht (14:56:40): > we probably want to keep the number of versions on OL to a minimum
Kevin Rue-Albrecht (14:56:58): > sort of a ‘push upstream’ kind of scenario
Charlotte Soneson (14:57:03): > Ok, cool, just so that we are doing the same:slightly_smiling_face:
Federico Marini (14:57:46): > yup:slightly_smiling_face:
Federico Marini (15:01:58): > @Aaron Lunand@Charlotte Soneson, what do you mean precisely with point 1 above?
Charlotte Soneson (15:15:23): > Basically redo the dimensionality reduction on only the selected subset of the cells. The “user-supplied functions” should define how the dim.red. should be done (PCA, t-SNE, etc).
Federico Marini (15:16:04): > So “allowing” some analysis, if I got you right?
Federico Marini (15:16:34): > With some sensible limits
Federico Marini (15:16:53): > I am not dogmatic against it, at all
Federico Marini (15:17:34): > same goes for the better (centralized?) support for gene sets
Kevin Rue-Albrecht (17:34:19): > Yes, i think it will be up to the users to download the gene sets (e.g., MSigDb). We could supportread.gmtfor instance (https://www.rdocumentation.org/packages/qusage/versions/2.4.0/topics/read.gmt) andGSEABasehas atoGmtfunction. - Attachment (rdocumentation.org): read.gmt function | R Documentation > This function reads in and parses information from the MSigDB’s .gmt files. Pathway information will be returned as a list of gene sets.
Kevin Rue-Albrecht (17:35:00): > there might be an equivalent totximportfor gene sets that I don’t know about though
Aaron Lun (18:39:48) (in thread): > So… something likerenderEpivis, if that does not sound too ambitious. Possibly similar to what DataTable offers.
Jayaram Kancherla (19:11:18) (in thread): > We do have a render function inepivizrChartpackage that work’s similar to DataTable’s render and generates multiple visualizations. I think the key for me to figure out is to having selections passed back to the shiny server. I remember seeing a TCGA version of iSEE, is the vignette for that available anywhere ? or if you have a preferred dataset, let me know and we’ll start with that.
Aaron Lun (19:13:10) (in thread): > It would probably be easiest to work fromexample(iSEE, ask=FALSE), which uses a fairly small data set that is convenient for testing.
Aaron Lun (19:13:20) (in thread): > I haven’t filled it with any genomic coordinates, though.
Aaron Lun (19:15:20) (in thread): > And theiSEEinternals are pretty intense, so I’m not sure you want to try to add a new panel type yourself (though you are more than welcome to try). If you provide a MWE of a shiny browser using epivizr, I can pick it apart to figure out how it fits intoiSEE.
Jayaram Kancherla (19:42:52) (in thread): > Yup that works. I’ll let you know once i have a working prototype. I can work with you to get it integrated into iSEE
Aaron Lun (19:43:55) (in thread): > Sounds good, thanks.
Kevin Rue-Albrecht (19:55:22): > Alright. Going to bed here. Wish I could do more, but it’ll have to wait this weekend
Kevin Rue-Albrecht (19:56:22): > Basically, I’ve split up the current text into more sections that are more digestible, and give us space to flesh out certain points in each of them
Kevin Rue-Albrecht (19:58:04): > Section titles are work in progress, especially the one that only describes the SummarizedExperiment input. Which was weird anyway, as the section title used to be ‘description of the iSEE interface’
Aaron Lun (19:59:04): > I’ve had about 10 hours sleep over the past 70 hours.
Kevin Rue-Albrecht (20:00:26): > i don’t envy that
Kevin Rue-Albrecht (20:02:12): > but for now, I have to go dream how to continue a project for Steve tomorrow. At least I’ll feel that I deserve a bit of iSEE playtime this weekend:stuck_out_tongue:
Aaron Lun (20:04:35): > Sweet dreams:heart:
2018-04-27
Charlotte Soneson (10:02:52): > @Aaron Lunyour wish has been granted:upside_down_face:http://www.bbc.com/news/entertainment-arts-43924609
Kevin Rue-Albrecht (10:06:45): > are they releasing an iSEE jingle?
Charlotte Soneson (10:07:49): > Probably:slightly_smiling_face:
Kevin Rue-Albrecht (11:02:07): > Question to the crowd: at the moment, we still have hard-coded the maximum number of levels for a factor to 24, while I implemented that as a potentially configurable option destined to bold users (https://github.com/csoneson/iSEE/blob/a2027cb6115bae640d81a514f90d82ae9dca903f/R/plotting.R#L1601). > At the moment, we don’t give user visible control over that value > (https://github.com/csoneson/iSEE/search?utf8=✓&q=.is_groupable&type=) and I was wondering how that would be best implemented, without an extra argument toiSEE( ...) - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Federico Marini (11:23:31): > yo, peeps
Federico Marini (11:23:44): > got some flesh added to the ms body
Federico Marini (11:24:56): > in brief: > - i tried to stick to the section naming style which is common for such articles on f1000res > - maybe some sections are oh-not-so-needed, or compactable with others > - ui description and functionality are there > - something more on tours
Federico Marini (11:25:34): > … all now onhttps://github.com/LTLA/iSEE2018/tree/f1000res
Federico Marini (11:26:04): > @Federico Marini pinned a message to this channel. - Attachment (overleaf.com): iSEE: Interactive SummarizedExperiment Explorer - Overleaf > The online platform for scientific writing. Overleaf is free: start writing now with one click. No sign-up required. Great on your iPad.
Aaron Lun (11:27:28) (in thread): > Just an extra arg should be sufficient.
Federico Marini (11:27:30): > plushttps://www.overleaf.com/15765478mqsvydndbhfv#/59995311/-> I pinned the address for this - Attachment (overleaf.com): iSEE: Interactive SummarizedExperiment Explorer - Overleaf > The online platform for scientific writing. Overleaf is free: start writing now with one click. No sign-up required. Great on your iPad.
Federico Marini (11:27:45) (in thread): > Yup
Federico Marini (11:28:10) (in thread): > and if someone has >24 categories, it is likely a power user, and (s)he will change the hard-coded value:smile:
Kevin Rue-Albrecht (11:31:53) (in thread): > Call that power user ‘viper’:wink:
Aaron Lun (12:52:08): > I will work on this as soon as I get back. Can you push from OL?
Federico Marini (13:46:09): > I pushed to OL already (well, copied over)
Aaron Lun (13:48:34): > Oh, I meant more generally, can a person push to GH from OL?
Federico Marini (13:49:02): > I did not try that out
Aaron Lun (13:49:10): > I mean, how are people doing it?
Federico Marini (13:49:18): > likely, but it was too much in a danger zone:stuck_out_tongue:
Aaron Lun (13:49:27): > Editing on OL, copying back to desktop, and then commit + push to GH?
Federico Marini (13:49:47): > would make sense, I guess. Also from their side to support it.
Federico Marini (13:50:01): > can be that it is indeed possible, in one of their many guides/docs
Aaron Lun (13:50:05): > But we’re not doing it now.
Aaron Lun (13:50:22): > Okay. Let’s be explicit. How are you currently editing the MS?
Federico Marini (13:52:55): > tex, locally, with good ol’ texworks
Federico Marini (13:53:04): > compilin on mine
Aaron Lun (13:53:12): > So OL isn’t involved at all
Aaron Lun (13:53:21): > Good.
Federico Marini (13:53:22): > and then every “git push” I copy-paste to OL
Federico Marini (13:53:24): > yep
Federico Marini (13:55:25): > How was SF, apart from the few sleeptime?
Federico Marini (14:00:35): > @Kevin Rue-Albrecht:https://twitter.com/fabian_theis/status/989895577625214976 - Attachment (twitter): Attachment > Super useful hacking @cziscience meeting, where @falexwolf, @davisjmcc and A Butler added compatibility of scanpy, Seurat and SingleCellExperiment. https://pbs.twimg.com/media/DbzQx31VAAEWVQ1.jpg
Federico Marini (14:04:57): > I fixed a couple of things in the pkg itself, where we referred in the vignette to old nomenclature of the param groups
Federico Marini (14:05:07): > is it worth a push to upstream or not?
Aaron Lun (14:06:36): > Docs or code?
Federico Marini (14:06:51): > vignette
Federico Marini (14:07:09): > https://github.com/csoneson/iSEE/tree/fewfixes - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Aaron Lun (14:07:20): > Should be harmless enuf
Federico Marini (14:07:35): > fine
Federico Marini (14:08:09): > What maybe is worth doing (but we have few few time) is update the default tour. I liked a lot your “Action” coding
Aaron Lun (14:27:30) (in thread): > John doesn’t know how to code anymore, really.
Kevin Rue-Albrecht (14:31:03) (in thread): > So much for my SeuratConverter ;)
Kevin Rue-Albrecht (14:31:13) (in thread): > Well, happy there’s something out there
Kevin Rue-Albrecht (14:34:15): > You haven’t pushed the fix yet, have you? If you do, I could push it upstream tonight
Kevin Rue-Albrecht (14:35:05): > I’m just at a leaving party right now. Can do when I get home, if you like
Kevin Rue-Albrecht (14:38:42): > If it’s not core app code, it’s unlikely to break anything before the release and I can just test it in my side for sanity
Federico Marini (14:38:49): > Let me try first to also do the updated tour
Federico Marini (14:38:54): > no code
Federico Marini (14:39:00): > only text & anchors
Federico Marini (14:39:14): > some are un-anchored after the latest renaming
Federico Marini (14:39:35): > So, nothing transcendental, but still
Kevin Rue-Albrecht (14:39:50): > Ah right. Harmless, but nice to fix indeed as we still have the chance
Kevin Rue-Albrecht (14:40:48): > Would avoid the unnecessary hassle to post things back to the release branch. I wouldn’t like that idea
Kevin Rue-Albrecht (14:45:03): > Now it tempts me to throw in that controllable limit on the number of factor levels… but is that something probably best for extra testing?
Aaron Lun (14:51:33): > Depends on how easily it can get passed down. Factor values are pretty low-level, there’s a lot of arguments to a lot of functions that you’ll need to add.
Kevin Rue-Albrecht (15:22:34): > Ouch yeah didn’t realise that off the top of my head. Let’s keep it for the next release then
Federico Marini (15:23:02): > I wasn’T even thinking of touching upstream again until rel
Federico Marini (15:23:40): > but then when describing the ui I realized I did not include a button in the tour… and then the hurricane started
Kevin Rue-Albrecht (15:24:37): > Me neither. I thought frozen NEWS was the end of it. I only looked back at the release schedule now that you mentioned the small fixes
Kevin Rue-Albrecht (15:25:43): > I think those are small enough to be push upstream without being worthy of News
Federico Marini (15:45:12): > ok, the tour polish should be finished
Federico Marini (15:48:29): > quick check local, then pushing
Federico Marini (15:57:16): > Should be all fine
Federico Marini (15:57:32): > I’ll leave it up as open PR for you to check
Aaron Lun (16:17:32): > looks fine to me
Federico Marini (16:53:36): > Ok, I’d merge then
Federico Marini (16:53:59): > and ask@Kevin Rue-Albrechtto proceed with pushing upstream
Kevin Rue-Albrecht (17:36:46): > what’s the story, then? PR looks good. what’s the story about the Rd file that changed? a leftover that hadn’t gone through roxygen yet?
Kevin Rue-Albrecht (18:07:31): > I spotted a couple more typo as I was taking the updated tour. I’ll push in the next few minutes
Kevin Rue-Albrecht (18:09:42): > Also, i don’t know if it’s just me, but the last bit about the buttons at the top right of the dashboard header behaves a bit strangely for me. Namely, if i click the ‘Next’ button, the dropdown of buttons disappears and the intro window jumps to the top left corner
Kevin Rue-Albrecht (18:10:54): > Instead, I like to navigate tours by using the left-right arrows, which is more convenient (my keyboard doesn’t move around as much as the intro window), and more importantly here: it doesn’t close the dropdown of the dashboard
Aaron Lun (20:54:19): > Comeon, merge this fucker.
2018-04-28
Kevin Rue-Albrecht (06:54:12): > Merged and pushed this morning. I’m looking at the ms iSEE2018 now
Federico Marini (08:12:22) (in thread): > that behavior in imperfect, but it was the best i could get with the current rintrojs knowledge I have
Federico Marini (08:16:33): > “unrelated”: have you seen this?
Federico Marini (08:16:34): > https://twitter.com/kieranrcampbell/status/989984308378169344 - Attachment (twitter): Attachment > Patent application for applying tSNE to single-cell data - https://patents.google.com/patent/US20180046755A1/en
Federico Marini (08:16:43): > WTF
Kevin Rue-Albrecht (08:20:59) (in thread): > Yep no worry. I imagined that and would have done the same. Anyway, I added the ‘right arrow’ bit on the first intro step. People might figure it out themselves
Kevin Rue-Albrecht (08:21:40) (in thread): > so long as the information is there, people will get the basics and keep learning by doing
Kevin Rue-Albrecht (08:24:01): > is that even possible?
Kevin Rue-Albrecht (08:26:02): > if i understand correctly, they don’t have a particularly good application history either : “2013 … Abandoned”
Kevin Rue-Albrecht (08:39:34): > i’m in a picky mood today: just spotted the discrepancy of collapsible panel names for the heat map: ‘Feature params.’ and ‘Column data params.’ > I’d suggest changing the first to ‘Row data params.’ at some point, for consistency with conventions through the app > It didn’t hit me until now, as we’ve started mentioning the collapsible boxes by name in the manuscript
Kevin Rue-Albrecht (10:00:36): > Separately, I think we’ll need to add soon enough some documentation guiding users through the ‘pre-configuration’ of apps, which for me is a HUGE bonus for us. Can’t say how happy I am that I can open up an app with some things already set up the way I like without having to go through the same 100 clicks
Kevin Rue-Albrecht (10:01:32): > I’m still warming up going through@Federico Marini’s latest updates, but I’m thinking that it’d be a worthy point to make in the paper.
Kevin Rue-Albrecht (12:55:38): > alright, don’t hit me: i tried a first go at author contributions, too.
Kevin Rue-Albrecht (12:56:14): > anyway. I just go through the ECM section of the draft. Anyone alive around here?
Aaron Lun (14:34:11) (in thread): > WHAT THE FUCK
Kevin Rue-Albrecht (14:34:27) (in thread): > ?
Aaron Lun (14:34:33) (in thread): > The whole point
Kevin Rue-Albrecht (14:36:47) (in thread): > you’ll have to use a few more words to explain
Aaron Lun (14:38:50) (in thread): > of patenting that applicatoin
Aaron Lun (14:41:11) (in thread): > That’s deliberate, actually, as we never use row data.
Aaron Lun (14:41:21) (in thread): > That panel only controls the feature selection.
Kevin Rue-Albrecht (14:41:34) (in thread): > ok, i think i see what you mean. Call me naive, but I didn’t even think that kind of patent was possible. I mean no one patented the t-test that has been used to put every single drug on the market
Aaron Lun (14:41:48) (in thread): > While the column data panel is fair enough, because it’s the column data that we’re showing on the plot.
Aaron Lun (14:42:06) (in thread): > I think we should patent something too. Use of Shiny for single-cell data.
Kevin Rue-Albrecht (14:42:32) (in thread): > ok nevermind then
Kevin Rue-Albrecht (14:44:02) (in thread): > true story: i feel from the sky when my first postdoc prof asked me whether we could commercialise my TVTB package, which has basically 3-4 functions wrapped around Gviz
Aaron Lun (14:44:05): > Do I need to look at the app, or can I go straight to the MS? All changes were just doc-related right?
Kevin Rue-Albrecht (14:44:29): > give me just another 5 min on the MS: i’m touching the conclusion right now
Kevin Rue-Albrecht (14:45:30): > i’ve been at it all afternoon, I’m ready to pass the relay over after that
Aaron Lun (14:46:10): > Did someone commit f1000 styles? I see the header PNG but not the style.
Kevin Rue-Albrecht (14:46:29): > no, Fede pointed out a few days ago that sty files are not tracked
Kevin Rue-Albrecht (14:46:43): > you can easily download it from OL, and stick in your local folder
Aaron Lun (14:46:54): > well, why do we have te PNG committed, then?
Kevin Rue-Albrecht (14:47:20): > dunno. a design choice related to BAN ?
Kevin Rue-Albrecht (14:47:53): > also delete the caption.sty as per this message - Attachment: Attachment > it’s working locally, now that I deleted the caption.sty file @Aaron Lun and @Charlotte Soneson might want to take note of this one, for sanity
Aaron Lun (14:51:01): > you know, if you want to commit the sty, you just edit the.gitignore.
Kevin Rue-Albrecht (14:51:42): > ok, i’ll do that
Kevin Rue-Albrecht (14:55:07): > alright, over to you
Aaron Lun (14:55:43): > Wait. You addedallthe Sty files?
Aaron Lun (14:55:59): > facepalm
Kevin Rue-Albrecht (14:56:57): > well, all the ones that I had on my side. Damn, they’reallleftovers from the BAN?
Kevin Rue-Albrecht (14:56:58): > argh
Aaron Lun (14:57:05): > I’ll fix it
Kevin Rue-Albrecht (14:57:29): > sorry - that was a poor end to a long afternoon
Kevin Rue-Albrecht (14:58:24): > anyway. no need to tell you, but don’t be sentimental with the text. I borrowed a few pieces from the vignette, to get some material in there. I don’t mind seeing anything further edited
Federico Marini (14:59:08): > back to the patent story, this thing is so wrong on so many sides. AS IF dana pe’er with her damn viSNE ever solved one single problem. plus, even the thought of patenting a method NOT FROM THEM
Federico Marini (14:59:53) (in thread): > I tried indeed to touch on that point when I mention the customization at startup
Federico Marini (15:00:33): > is viSNE btw the one that even required matlab?!
Kevin Rue-Albrecht (15:05:22) (in thread): > Can’t remember not I wrote anything more on it or just thought it
Kevin Rue-Albrecht (15:05:42) (in thread): > I’ve got other stuff to tend to now. So feel free to have a play
Aaron Lun (15:23:02): > Probably the text is a bit too detailed.
Aaron Lun (15:23:58): > Also, someone didn’t get the memo about 1 line = 1 sentence.
Kevin Rue-Albrecht (15:25:25): > Might be me. I’m trying to be mindful of this, but when i worked on a paragraph I was pushing some of the text a few lines down, and might have missed a few places to stitch it properly back together
Aaron Lun (16:12:54): > @Federico Marinineed rintrojs citation
Aaron Lun (16:18:41): > NVM
Aaron Lun (16:22:25): > I feel the need - the need for sleep. I will continue with this tomorrow, but in the meantime, I need people to provide some screenshots of each of the apps. These will be placeholders for the interactive figures.
Aaron Lun (16:23:00): > FYI the current functionalities section has been fully reorganized. Will proceed to the use cases tomorrow.
Aaron Lun (16:24:36): > Actually, NVM on the screenshots, I see the Wellcome open research example doesn’t have it anyway.
Kevin Rue-Albrecht (18:21:23): > Alright. off to sleep. Nice rewrite Aaron, it reads a lot smoother. Sweet dreams and take some well deserved rest. I’ve fixed up occurrences of ‘data sets’ and ‘web pages’ (they’re both two words). We should probably just flesh out the ‘use cases’ section, to push it across the line. Maybe give ourselves a few days to think about what more could go in there, too, I suppose.
2018-04-29
Aaron Lun (11:26:01): > @Federico Marinithe allen dataset needs to launch with the tour already started. Suggest manually specifyingtour=.
Aaron Lun (11:27:19): > Also, should we split theapp.Rinto a separatedata.Rdescribing how to obtain the SCE object?
Aaron Lun (11:31:14): > I remember Fed saying something about modifying the script to get it to work on the server
Kevin Rue-Albrecht (11:33:58): > - I like the idea of_data.Rfor tours > - I don’t know how the allen tour is set up on IMBEI, given that it doesn’t have a script yet in iSEE2018
Aaron Lun (11:41:49): > All tour operators need to have a disclaimer at the front of the tour, telling users to restart the tour from the top-right button (see the PBMC tour).
Aaron Lun (11:41:56): > Also, why the fuck is the PBMC tour so slow to load? The TCGA is lightning quick.
Kevin Rue-Albrecht (11:42:29): > unrelated: i’ve thrown in a stub of author contributions. I just noticed that i’ve put dots between initials (inspired from the Lun&McCarthy workflow), but this doesn’t match with the absence of dots in the funding paragraph. I don’t mind either way, but we should probably stick with one
Kevin Rue-Albrecht (11:43:52) (in thread): > did you just try now? TCGA might have been quick because I was also just playing with it, if there’s any interaction between sessions about shared resources
Kevin Rue-Albrecht (11:43:58) (in thread): > TCGA was slow for me
Aaron Lun (11:44:56) (in thread): > hm. Okay.
Kevin Rue-Albrecht (11:45:42) (in thread): > indeed, the PBMC one opened in ~5-10s for me right now
Aaron Lun (11:46:38): > What does tCGA stand for again? The Cancer Genome Atlas?
Kevin Rue-Albrecht (11:46:55): > yup.
Aaron Lun (11:47:01): > is 7000 the entire lot?
Kevin Rue-Albrecht (11:47:35): > apparently not, I realised later that there is a bigger one of ~9k samples
Kevin Rue-Albrecht (11:47:44): > when control (non-cancer) samples are included
Kevin Rue-Albrecht (11:48:07): > at the time, I just grabbed the first data set in the experimenthub “EH1”
Kevin Rue-Albrecht (11:48:30): > the bigger one is ~EH149 or something like that
Kevin Rue-Albrecht (11:49:41): > > > query(eh, "The Cancer Genome Atlas") > ExperimentHub with 4 records > # snapshotDate(): 2018-04-27 > # $dataprovider: GEO, TCGA > # $species: Homo sapiens, Homo sapien > # $rdataclass: ExpressionSet, SummarizedExperiment > # additional mcols(): taxonomyid, genome, description, coordinate_1_based, maintainer, > # rdatadateadded, preparerclass, tags, rdatapath, sourceurl, sourcetype > # retrieve records with, e.g., 'object[["EH1"]]' > > title > EH1 | RNA-Sequencing and clinical data for 7706 tumor samples from The Cancer Genome Atlas > EH1043 | RNA-Sequencing and clinical data for 9246 tumor samples from The Cancer Genome Atlas > EH1044 | RNA-Sequencing and clinical data for 741 normal samples from The Cancer Genome Atlas > EH1108 | TCGA >
Kevin Rue-Albrecht (11:49:50): > akaEH1043
Kevin Rue-Albrecht (11:51:22): > (I’ll have to check but I’m pretty sure “9246 tumor samples” is a lie. That indeed the extra samples come at least in part from healthy people)
Aaron Lun (11:51:24): > Anyway, downsampling is broken on the TCGA violin plot. Many of the points seem to lie exactly on the vertical line, and turning off downsampling breaks the app.
Kevin Rue-Albrecht (11:51:36): > damn.
Aaron Lun (11:52:04): > hell, I can’t even get the violin plot any more
Kevin Rue-Albrecht (11:55:43): > not sure what you did, but i’m playing with the DS checkbox and the TCGA app behaves ok
Kevin Rue-Albrecht (11:56:53): > i think i see what you mean about the data points on the vertical line - some of them do overlap - but are you sure it’s not just a matter of resolution?
Aaron Lun (11:57:23): > No. You see if you turn on downsampling, all of the dots are not scattered along the width of the violin.
Aaron Lun (11:57:50): > This is because the x-scatter coordinates are calculated using the full set of points, but should be calculated using the jittered set.
Kevin Rue-Albrecht (11:58:22): > @Kevin Rue-Albrechtuploaded a file:? - File (PNG): ?
Aaron Lun (11:58:40): > That’s not what I see. Make sure you turn on downsampling.
Aaron Lun (11:58:54): > @Aaron Lunuploaded a file:image.png - File (PNG): image.png
Kevin Rue-Albrecht (11:59:10): > oh yeah, with DS, that’s what i see too
Aaron Lun (11:59:34): > Bascially, we need to move the_setupfunctions out of the extract_plotting_data function, into a section where downsampling actually takes effect.
Aaron Lun (12:32:10): > Anyway, I am happy with the text. I will pass my version onto John to check (as I need to do, because he has to check if it’s got the CRUK on it). Suggest that everyone does the same with their PIs, if necessary.
Kevin Rue-Albrecht (12:45:45): > well, i checked with Steve when I spotted you’re acknowledgement of John’s funding, as I offered to do the same for him. However, he eventually decided that he probably not be a good idea, as my position is jointly funded by two sources, and mentioning either could create some conflicts (of interest or otherwise)
Aaron Lun (12:47:31): > What. How?
Aaron Lun (12:47:38): > Conflicts of interest with academic funding?
Aaron Lun (12:47:48): > I mean, you’re not funded by Rstudio or anything.
Kevin Rue-Albrecht (12:49:57): > i think more something along the lines of those sources of funding potentially wishing to include this as an output of their grants and/or request senior authorships, if you remember how Steve didn’t insist on this, despite being my direct supervisor/employer
Aaron Lun (12:50:42): > Hm.
Aaron Lun (12:50:46): > Well, your call.
Aaron Lun (12:53:28): > Also, I’ll send what we have to Vicky to see whether we’re on the right track.
Kevin Rue-Albrecht (12:53:34): > probably worth offering her a read of the manuscript, now that it’s in good shape
Kevin Rue-Albrecht (12:54:32): > That’s a good idea too. I’m genuinely curious about things we may be missing
Aaron Lun (13:14:09): > Allright<!channel>I’ve given you all a number of things to do for 3.8.
Aaron Lun (13:14:22): > See the GH issues for further details.
Kevin Rue-Albrecht (15:44:47): > <!channel>anyone feels like trying:docker run --name=isee_session --rm -p 4000:1234 -i kevinrue/isee_allen:0.99.5? > Followed by opening your fav’ browser tolocalhost:4000With thanks to@Davis McCarthy
Kevin Rue-Albrecht (17:49:56): > If I’m correct, anyone can now distribute self-contained instances of iSEE applications by adding their own SCE object in an RDS file and a launcher R script similar to our tours. > I’ve added the allen example in our Dropbox
2018-04-30
Kevin Rue-Albrecht (04:42:58): > While the first docker download will take a while, I think any subsequent app should be lighting-fast, as it should re-use the common ‘bioconductor/devel_core2’ and ‘kevin/isee’ containers. > I’m making a new container for the the tcga tour, to test the hypothesis.
Aaron Lun (05:59:48): > Merging f1000res2 with master unless told otherwise.
Kevin Rue-Albrecht (05:59:59): > it looked good to me
Aaron Lun (06:02:11): > Man, the recalculation of reduced dims is a real pain in the ass
Aaron Lun (06:02:24): > Will have to think about it deeply.
Kevin Rue-Albrecht (06:02:38): > i’ll have to go for a meeting very soon.. but how so?
Aaron Lun (06:02:53): > See the GH for some comments
Kevin Rue-Albrecht (06:03:03): > ok. catch up later
Federico Marini (08:02:48) (in thread): > I saw you got that already. Thanks for taking care of it
Federico Marini (08:03:10) (in thread): > The separation could be a good idea.
Federico Marini (08:04:59) (in thread): > The edits I needed to perform are the following: > - commented out the data-related parts, read in the rds > - removed the part where we say > > app <- iSEE(sce) > runApp(app) > > because the shiny server complains to not userunApp. This, well, because it already tries to do so
Federico Marini (08:05:37) (in thread): > Against the separation would be one thing only: it is practical to have the whole thing in one, and the tour in the other one, and that would just do the job
Aaron Lun (08:42:32) (in thread): > meh, we’ll do what we need to do.
Federico Marini (08:47:48) (in thread): > Are the sections numbered intended to be so ?
Federico Marini (08:48:00) (in thread): > Competing interests/grants/co
Federico Marini (08:50:56) (in thread): > (plus I think they were listed as subsections in the template?)
Aaron Lun (08:51:36) (in thread): > Looking at my last actual F1000Res paper, they seem to full sections.
Aaron Lun (08:51:45) (in thread): > Doesn’t really matter, it won’t look like that in the final paper.
Aaron Lun (08:51:59) (in thread): > But they shouldn’t be numbered. Can’t remember if I turned that off.
Federico Marini (08:52:22) (in thread): > pulled now and I see numbers
Federico Marini (08:52:29) (in thread): > sounds like I’m on acid:smile:
Aaron Lun (08:55:16) (in thread): > okay, just whack a*in there.
Federico Marini (09:05:35) (in thread): > edited, committed, merged:wink:
Aaron Lun (09:06:49) (in thread): > :+1:
Aaron Lun (10:07:42): > Could someone add the shiny server links to the iSEE2018 README?
Federico Marini (11:18:04): > Since we are at editing…
Federico Marini (11:18:06): > - data only for thesce-> data.R > - tours is read in app.R > - tour content, obviously, in the txt
Federico Marini (11:18:21): > So each use case should have three .R files
Federico Marini (11:18:47): > (allen has two as it fetches the tour from the pkg, and edits one step by hand by appending the note)
Aaron Lun (11:22:15): > two R and one text
Federico Marini (11:22:23): > yep, sorry
Federico Marini (11:31:17): > I started here:https://github.com/LTLA/iSEE2018/tree/tourtour/tours
Federico Marini (11:31:31): > feel free to pick up on this branch
Aaron Lun (12:26:01): > pbmc tour’s “data” is pretty much just a comment on how to generate the SCE from another repo. Didn’t think it was worth deploying to an EH package, as it isveryspecific to the emptyDrops paper.
Aaron Lun (12:34:46): > Happy to be told otherwise, though.
Kevin Rue-Albrecht (12:36:17): > sounds fair … if you don’t need to to anything for the PBMC, updating the TCGA is up for takes:stuck_out_tongue:
Kevin Rue-Albrecht (12:36:44): > otherwise, i’ll get to it later
Aaron Lun (12:37:15): > off to dinner with a candidate
Kevin Rue-Albrecht (12:38:23): > maybe I should candidate too sometimes …:stuck_out_tongue_winking_eye:
2018-05-01
Aaron Lun (08:41:43): > Yes! shiny has been purged from scater and scran! ggplot2 has been purged from scran! Whoo!
Kevin Rue-Albrecht (08:43:27): > did that just happen? i’ve opened the scran GH (MarioniLab) and can’t see this in the last few commits
Aaron Lun (08:43:42): > runnign through CHECK now.
Kevin Rue-Albrecht (08:44:42): > and the only references to ggplot that I can see are in the shiny app, is that right? (I wasn’t aware of any ggplot in scran)
Kevin Rue-Albrecht (08:45:37): > ohhhh:heart_eyes:.Deprecated(new="iSEE::iSEE")https://github.com/MarioniLab/scran/blob/fba9cab1349ce1aa6e846baecf9255ae53ee18e0/R/exploreData.R#L12
Aaron Lun (09:04:49): > Also, ggplot2’s default discrete colour scheme sucks balls. Can’t distinguish between green and blue.
Kevin Rue-Albrecht (09:07:56): > i don’t have a problem about a different default, although i don’t have a suggestion either
Aaron Lun (10:49:26): > Whoever has push access should sync the Bioc and GH repos for the version bump.
Kevin Rue-Albrecht (10:50:36): > sure
Kevin Rue-Albrecht (11:05:56): > incoming PR should take care of that
Aaron Lun (11:06:11): > :vomit:
Aaron Lun (11:06:13): > oops
Aaron Lun (11:06:15): > :+1:
Kevin Rue-Albrecht (11:06:19): > lol
Kevin Rue-Albrecht (11:06:37): > i had a hesitation, given that we can’t push directly tomasteron GH
Aaron Lun (11:21:42): > I think you’re missing a version bump. We should have gotten another commit from val.
Kevin Rue-Albrecht (11:27:59): > :face_with_symbols_on_mouth:i grabbed the release branch. However, when I checkout upstream/master, I still see 0.99.5
Kevin Rue-Albrecht (11:28:11): > I don’t think it’s up yet
Kevin Rue-Albrecht (11:28:20): > better wait this evening I guess
Aaron Lun (11:29:34): > Grab the master, I have vbumps for scran and scater
Aaron Lun (11:29:38): > from val
Kevin Rue-Albrecht (11:32:34): > incoming
Aaron Lun (18:11:09): > I’m going to changefeatExprstofeatAssay, for “Feature Assay Plots”. I will do this and then suggest pushing this to release ASAP, to avoid people setting up scripts usingfeatExprs. Suggest@Kevin Rue-Albrechtdo the same for discrete => categorical, especially in the ECM interface (but also more generally).
Aaron Lun (18:15:45): > @Federico Mariniheads up - once the above changes go through, package vignette + iSEE2018 tours will need updating.
Aaron Lun (18:27:19): > Basically, we’re sneaking past the usual deprecation cycle, because CBF.
Aaron Lun (18:27:37): > Not that it really matters, I doubt we accumulated that many power users so quickly.
Aaron Lun (18:27:47): > Other than us, of course.
Aaron Lun (18:27:56): > Which is why I’m telling you all this.
2018-05-02
Kevin Rue-Albrecht (02:32:06) (in thread): > writing it down for to do an evening this week
Kevin Rue-Albrecht (02:32:22) (in thread): > and i haven’t forgotten about the tours files to update
Kevin Rue-Albrecht (04:53:25): > @Kevin Rue-Albrechtuploaded a file:isee_rafiki.pngand commented: (8) Naaaaaaaaants ingonyama bagithi Baba (8) - File (PNG): isee_rafiki.png
Kevin Rue-Albrecht (06:53:34): > too much?:sweat_smile:
Aaron Lun (06:55:25): > Sithi uhm ingonyama
Federico Marini (06:56:04): > one of us with a blog should actually use that in the post
Kevin Rue-Albrecht (06:57:04): > what was that thing you found,@Federico Marinithat shows an image while Shiny elements are being rendered? ^^
Federico Marini (06:57:22): > shinycustomloader or similar?
Kevin Rue-Albrecht (06:58:13): > yeah. i’m just teasing, as this image wouldn’t have its place in the app, of course, plus the custom loader makes better animations
Federico Marini (06:58:48): > the easter egg edition of iSEE
Federico Marini (06:58:50): > :slightly_smiling_face:
Aaron Lun (10:18:10): > Renaming is almost complete. Who has not usedgit cherry-pick?
Federico Marini (11:31:24): > I did not
Federico Marini (11:33:13): > is there a branch online already to work upon?
Aaron Lun (11:35:49): > Yes,RELEASE_3_7. Be VERY careful when working on this branch. Usecherry-pickto select bugfix commits frommasterto bring in. Do notmergewith mster.
Aaron Lun (11:36:15): > Itwilldiverge from themaster; this is expected.
Aaron Lun (11:37:08): > Anyway, I have converted it tofeatAssay. Suggest you re-install on the server and update the tours.
Federico Marini (11:43:51): > I can continue on yourrenamed
Federico Marini (11:44:02): > and we do all the changes once it is done done?
Aaron Lun (11:50:22): > Yes, update the tour as well.
Aaron Lun (11:50:40): > But keep the number of commits low, otherwise it will be painful.
Federico Marini (11:51:05): > grepping and replacing should work well I guess?
Federico Marini (11:51:19): > If we use cleverly^featExpr
Aaron Lun (11:51:25): > I’ve already done it.
Aaron Lun (11:51:40): > on the rest of the package
Federico Marini (11:51:40): > I’ll tag along then
Aaron Lun (11:51:40): > so yes
Aaron Lun (11:51:55): > Make sure you also check foreature expression
Aaron Lun (11:52:07): > Just to change the name as well.
Federico Marini (11:52:12): > Ok
Federico Marini (12:06:11): > Ok the tour had minimal traces
Aaron Lun (12:09:00): > oops, clicked on the wrong thing
Aaron Lun (12:09:04): > Should talk here.
Federico Marini (12:09:38): > no prob. Anyway:grep -irnis a wonderful tool:slightly_smiling_face:
Federico Marini (12:10:05): > (sedeven better, but I am not so familiar with all options)
Aaron Lun (12:47:24): > Okay, whoever has push access to BioC should be very careful, as RELEASE_3_7 has just been updated, and if you push you will also push RELEASE_3_7.
Aaron Lun (12:54:18): > It’s all done, but double-check RELEASE_3_7 (start the app, fuck with it). Then someone push it to BioC ASAP.
Kevin Rue-Albrecht (13:19:47): > will do when I get home
Kevin Rue-Albrecht (13:20:01): > sorry - journal club + long chat
Kevin Rue-Albrecht (13:22:07): > just to be clear: you mean thatRELEASE_3_7is ready to be pushed upstream, AFA-you’re-A ?
Kevin Rue-Albrecht (14:26:52): > alright, I’vegit merge origin/RELEASE_3_7 RELEASE_3_7and now I’m building locally
Kevin Rue-Albrecht (14:27:56): > if all is correct, all I need isgit push upstream RELEASE_3_7, right?
Kevin Rue-Albrecht (14:28:35): > https://bioconductor.org/developers/how-to/git/bug-fix-in-release-and-devel/says yes
Federico Marini (14:29:09): > I am the least appropriate person to ask on the cherry picking
Federico Marini (14:29:20): > :disappointed:
Kevin Rue-Albrecht (14:31:18): > well… is there any more cherry picking to do? i have dropped out of the conversation today, but@Aaron Lun’s last message included ‘it’s all done’
Federico Marini (14:32:16): > I am updating the whole on the MS side, but I had the feeling the pkg itself was done
Kevin Rue-Albrecht (14:32:22): > (although I haven’t touched the ECM interface as mentioned in the thread yesterday, I’ll look now, but it sounds like you guys nailed it withsedor equivalent)
Federico Marini (14:33:54): > I guess Aaron went with a sed-like bazooka over all .R files
Federico Marini (14:34:01): > plus the vignette was also fixed
Federico Marini (14:42:13): > Som the ms repo branchtourtourhas a PR open for renaming and structuring the tours
Federico Marini (14:42:18): > for your eyes only:slightly_smiling_face:
Kevin Rue-Albrecht (14:55:23): > so that;s the place where I can split into_appand_data
Federico Marini (14:55:34): > yep
Federico Marini (14:55:50): > sometimes the _data is one line or two
Federico Marini (14:56:15): > but if you want, you can be explicit and put all the steps. say, from EH1 to sce, which in your case fits well
Kevin Rue-Albrecht (17:35:10) (in thread): > mhh… I’m looking at thatdiscrete->categoricalupdate. I suppose that should also affect the code.. right? damn, those package-wide updates make me nervous. Especially when the word discrete appears in functions likescale_discrete
Kevin Rue-Albrecht (17:36:42) (in thread): > actually.. now that makes me think, ifdiscretehas been adopted for things likeggplot2, why wouldn’t we keep using it as it is right now?
Kevin Rue-Albrecht (17:44:40) (in thread): > @Federico Mariniquick one: so for the iSEE2018 _data / _app …
Kevin Rue-Albrecht (17:45:19) (in thread): > what we expect users to do is: > - rundata.R- then runapp.R… right?
Kevin Rue-Albrecht (17:45:48) (in thread): > or shouldapp.Rsourcedata.R?
Kevin Rue-Albrecht (17:55:37): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-02 22.55.06.pngand commented: God I love t-SNE sometimes - File (PNG): Screenshot 2018-05-02 22.55.06.png
Aaron Lun (18:00:58): > Yes, theiSEEstuff is all done from my side. No more cherry-picking, just push when you’re happy.
Aaron Lun (18:01:42): > Also, my GH issue said to change discrete => categorical, but I realized that ggplot2 was using discrete (Bad!) anyway, so oh well, let’s just leave it as it is. But worth mentioning in the internal docs and in the ECM that discrete = categorical, not integer.
Aaron Lun (18:03:00) (in thread): > Yes, that was what I concluded in the end.
Aaron Lun (18:03:15) (in thread): > See the GH issue. Or somewhere where I said that.
Kevin Rue-Albrecht (18:03:43): > sure, but do you want that on the release branch as well?
Aaron Lun (18:05:00): > No, that’s not critical.
Kevin Rue-Albrecht (18:05:26) (in thread): > argh.. didn’t realise you replied to the GH.
Kevin Rue-Albrecht (18:06:00): > ok, well i’ve played with the release branch compiled locally, and I couldn’t make it bug, so I think i’ll push upstream
Kevin Rue-Albrecht (18:07:43): > Meanwhile, for the use of bold/italic in the tours, I’ve just tried replacing all the pieces of info that I currently show in bold (TCGA tour), by italic, and I’m sorry to say that it looks crap
Kevin Rue-Albrecht (18:09:03): > Considering that I don’t giveActioninstructions in the tour, I actually think it doesn’t hurt so much. So long as each tour is internally consistent. Opinions?
Aaron Lun (18:14:39): > Colours?
Kevin Rue-Albrecht (18:14:44): > ow
Kevin Rue-Albrecht (18:14:56): > didn’t think of that one
Aaron Lun (18:15:09): > But I think you have too many things in bold anyway.
Aaron Lun (18:15:22): > I mean, I don’t think you need the stats in bold.
Kevin Rue-Albrecht (18:15:31): > numbers of samples and genes, names of UI panels, that’s about it
Aaron Lun (18:15:45): > The numbers aren’t really that important, IMO
Aaron Lun (18:15:56): > The UI panels… well, maybe.
Kevin Rue-Albrecht (18:16:36): > True. I thought I was highlight key points (e.g. ‘look how many samples we can handle’), but users can probably figure that themselves
Kevin Rue-Albrecht (18:17:03): > I was thinking about code font, but that’s just brutal-looking in tooltips
Kevin Rue-Albrecht (18:28:23): > would you mind having a quick look at the README of iSEE2018 ?
Aaron Lun (18:33:28): > give me a few minutes, dealing with support
Kevin Rue-Albrecht (18:34:17): > no worries. i just fleshed out the description of running the tours, and added the links to the public demos that you suggested a few days ago
Aaron Lun (19:13:50): > Okay, ich bien back.
Aaron Lun (19:15:47): > So italics for the UI element names aren’t nice?
Aaron Lun (19:15:55): > I thought it would be okay…
2018-05-03
Federico Marini (03:36:12): > my 2 cents, given I did not know we could use html code until a few days ago:stuck_out_tongue:
Federico Marini (03:36:35): > I’m fine with any version, as long as we keep consistent across tours
Federico Marini (03:37:04): > the Action in bold is excellent to not be missed, so I would leave probably just that inboldy?
Kevin Rue-Albrecht (03:43:58): > Yeah, I’ll move numbers back to normal font, and italicise the pane names. Later today or this weekend at the latest
Kevin Rue-Albrecht (03:46:54): > That was initially to showcase the size of stat sets handled and expressions specific to the iSEE « language » . But tours are probably not the place for that (or at least not one outlier tour)
Aaron Lun (05:15:33): > Where’s my full stop?
Aaron Lun (05:46:04): > Also,@Federico Marini, I was wondering whether we could actually put the question mark icon in the disclaimer for restarting the tour - perhaps this would be a lot easier to parse.
Federico Marini (05:46:56): > you mean like a visual cue to find it in the corner?
Aaron Lun (05:47:12): > yeah.
Federico Marini (05:47:24): > lemme check
Aaron Lun (05:47:33): > If arbitrary HTML is allowed, it should be possible.
Federico Marini (05:47:43): > I hope the fa icons are includable
Federico Marini (05:47:46): > if so, then yes
Federico Marini (05:50:38): > doesn’T show up:disappointed:
Federico Marini (05:51:39): > I mean, per se it would not be complicate to insert them in a full html page
Federico Marini (05:51:42): > see here:
Federico Marini (05:51:43): > https://www.w3schools.com/icons/fontawesome_icons_intro.asp - Attachment (w3schools.com): Font Awesome Intro > Well organized and easy to understand Web building tutorials with lots of examples of how to use HTML, CSS, JavaScript, SQL, PHP, and XML.
Federico Marini (05:52:06): > I don’t know if we just put that line in the isee main body, will it work?
Aaron Lun (05:52:07): > But they don’t fit easily into the intro tour.
Aaron Lun (05:52:08): > Hm.
Federico Marini (05:52:36): > we cannot ad dit to the head..
Aaron Lun (05:52:57): > Well, fuck that then
Federico Marini (05:53:45): > wait a sec
Federico Marini (06:00:03): > ok
Federico Marini (06:00:08): > not working, even so:
Federico Marini (06:00:15): > > tags$head( > tags$link( > rel="stylesheet", > href="[https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css](https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css)") > ) >
Federico Marini (06:00:20): > with this in the body
Aaron Lun (06:00:35): > fuck it
Federico Marini (06:01:05): > would have been nice, I admit it
Kevin Rue-Albrecht (07:04:14) (in thread): > What do you mean? I see full stops everywhere in the README.
Aaron Lun (07:37:23) (in thread): > on tourtour?
Aaron Lun (07:37:40) (in thread): > in the itemize list, where my comments are.
Federico Marini (07:53:27): > Side info: I setup the bioconda recipe in the meanwhile
Federico Marini (07:53:33): > PR is here:https://github.com/bioconda/bioconda-recipes/pull/8803 - Attachment (GitHub): adding iSEE (and one of its dependencies, scRNAseq) from R/Bioconductor by federicomarini · Pull Request #8803 · bioconda/bioconda-recipes > I have read the guidelines for bioconda recipes. This PR adds a new recipe. AFAIK, this recipe is directly relevant to the biological sciences (otherwise, please submit to the more general purpos…
Federico Marini (07:53:50): > but we need to wait a little for them to build the whole R stack on 3.5.0
Kevin Rue-Albrecht (07:56:28): > i’m about to run off to catch a bus to the airport now. will be offline for a bit
Kevin Rue-Albrecht (07:56:47): > i just gave you something to toy with in the README
Kevin Rue-Albrecht (07:56:54): > inspired from BiocStickers
Kevin Rue-Albrecht (08:00:19) (in thread): > oh. i could swear i fixed them
Kevin Rue-Albrecht (08:00:37) (in thread): > anyway. i’ll catch up on the bux. they have wifi i believe
Kevin Rue-Albrecht (11:07:12): > @Kevin Rue-Albrechtuploaded a file:Slack for iOS Uploadand commented: Why windows… why?:face_vomiting: - File (PNG): Slack for iOS Upload
Aaron Lun (11:10:43): > I think everyone’s breaking on Windows.
Kevin Rue-Albrecht (11:13:12): > Yeah I looked at the error message and it seems pretty core to the R installation. Beyond package dependencies
Aaron Lun (14:13:12): > The README doesn’t get BiocStyle, so someone should hard-link iSEE’s Bioconductor page.
Aaron Lun (14:13:21): > And fix the grammar
Kevin Rue-Albrecht (17:22:33): > @Aaron Lundid you re-contact f1000 with the updated manuscript btw? If so, I assume you didn’t get a reply yet. Or are we still waiting until all the tours are updated/coordinated? Just curious.
Aaron Lun (17:23:45): > Yes, I have, I asked them about some details of how the iframe would work, especially because the app is big and won’t fit in a subbox.
Aaron Lun (17:24:02): > Also, John won’t read the MS til the WE, and charlotte gets back next week, i think.
Kevin Rue-Albrecht (17:25:46): > Ok. Thanks. I’m away now until Tuesday. I still need to update the tcga tour and some discrete/categorical statements in the ECM
Kevin Rue-Albrecht (17:26:10): > The latter being more a “devel” lower priority thing
Kevin Rue-Albrecht (17:30:23): > And for now I’d put faceting on the back burner, even though I’m the one who brought it up again recently. I anticipate heavy work on handling facets as domains and brushes (I think we’d need to make the custom lasso look even more like a shiny brush to store all the relevant information. > Bottom line is i just can’t afford the time right now. Need to get other things out of the way first
Aaron Lun (17:37:42): > Sure. Your zoo, your monkeys.
2018-05-04
Federico Marini (03:18:34): > Look who’s there:slightly_smiling_face:
Federico Marini (03:18:36): > http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006135 - Attachment (journals.plos.org): beachmat: A Bioconductor C++ API for accessing high-throughput biological data from a variety of R matrix types > Biological experiments involving genomics or other high-throughput assays typically yield a data matrix that can be explored and analyzed using the R programming language with packages from the Bioconductor project. Improvements in the throughput of these assays have resulted in an explosion of data even from routine experiments, which poses a challenge to the existing computational infrastructure for statistical data analysis. For example, single-cell RNA sequencing (scRNA-seq) experiments frequently generate large matrices containing expression values for each gene in each cell, requiring sparse or file-backed representations for memory-efficient manipulation in R. These alternative representations are not easily compatible with high-performance C++ code used for computationally intensive tasks in existing R/Bioconductor packages. Here, we describe a C++ interface named beachmat, which enables agnostic data access from various matrix representations. This allows package developers to write efficient C++ code that is interoperable with dense, sparse and file-backed matrices, amongst others. We evaluated the performance of beachmat for accessing data from each matrix representation using both simulated and real scRNA-seq data, and defined a clear memory/speed trade-off to motivate the choice of an appropriate representation. We also demonstrate how beachmat can be incorporated into the code of other packages to drive analyses of a very large scRNA-seq data set.
Aaron Lun (05:10:31): > Oh yeah, I still have to green-light the proofs
Aaron Lun (05:10:42): > Do you like my gratituous use of colour?
Federico Marini (05:13:18): > Did they want it?
Aaron Lun (05:20:59): > No
Aaron Lun (05:21:14): > When I started making the figures, I had a nice colour code.
Aaron Lun (05:21:24): > But then there were too many things to colour code, and I ran out of colours
Aaron Lun (05:21:33): > what you see are the remnants of my attempts to standardize the colours
Aaron Lun (05:21:45): > Specifically, in the various supplementary figures.
Federico Marini (05:44:00): > did not check yet there
Federico Marini (05:45:38): > Ouh ok, quite some color scheme attempts:smile:
Aaron Lun (05:45:55): > It would have all been okay, but the reviewers wanted some figures in the main text as well
Aaron Lun (05:46:05): > so the main text figures just look like I randomly picked out colours
Federico Marini (05:46:10): > Go go straight lines!
Aaron Lun (14:53:52): > When people get the chance, I suggest checking outrenamed. I did some work to try to reduce the size of the reported code by avoiding generation of unnecessary variables when they are not needed (e.g.,plot.data.allwhen no restriction is active, orplot.data.prewhen no downsampling is being performed). This was mostly okay, but the horizontal violin plots were a real nightmare with respect to which axis was going where (asplot.data.allis generatedbeforeX/Y are flipped). See if you can break the violin plots with any combination of horizontal-ness, downsampling and selection to restrict.
2018-05-05
Aaron Lun (11:39:09): > Uh - known bug in the downsampler. Will get fixed as part ofcustomplot.
Charlotte Soneson (13:35:19): > I have made some small modifications to the manuscript (in theedits_cbranch). A couple of remaining issues: > 1. The figure says “Feature expression” while the caption says “Feature assay”. > 2. The figure says “Heatmap” while the rest of the manuscript says “Heat map” (it had both variants before, I changed to make it consistent). > 3. The sentence “Despite its name, this plot type is applicable to data modalities beyond gene expression.” on page 2 may not be necessary any more after renaming to feature assay? > 4. I guess we are not really consistent with British/American spelling - e.g. customisable and visualize (or maybe it’s ok to mix). > 5. The links in the ms all point tohttp://shiny.imbei.uni-mainz.de:3838/, but I guess that is because of the line wrapping (and if we are anyway embedding the apps it will not matter).
Aaron Lun (13:39:06): > 2; heatmap should probably be one word.
Aaron Lun (13:39:11): > 1; figure screenshot needs updating.
Aaron Lun (13:39:18): > 3; yes, that can be removed.
Aaron Lun (13:39:27): > 4; probably should be british, then.
Aaron Lun (13:39:42): > 5; Do they? I thought they were correct.
Charlotte Soneson (13:40:08) (in thread): > at least hovering over them points me to the general page
Charlotte Soneson (13:41:02) (in thread): > I’ll fix this
Charlotte Soneson (13:42:29) (in thread): > removed this
Aaron Lun (13:53:07): > <!channel>Anyway, on some more exciting news,customplotis now working. To use, do something like this: > > example("Custom iSEE plots") > iSEE(sce, customColFun=list(PCA2=CUSTOM)) > > … set the custom function to “PCA2” in the app, receive a selection from anywhere, set it to restrict and brush on the transmitting plot; the custom column plot will then recompute the PCA on the restricted subset.
Aaron Lun (13:54:02): > It takes a few seconds; I’m not doing a particularly fast or sensible PCA.CUSTOMis just a demo function.
Aaron Lun (13:56:27): > Honestly, though, this two-space indenting makes me sick.
Aaron Lun (13:56:52): > Someday I’m not going to be able to control myself and will convert all of them to the rightful 4 space indent.
Aaron Lun (14:16:17): > Anyway, back to the custom plots; there are a number of implications that arise from having coordinates that change with selection, in particular upon transmissionfromthe custom plot. Ideally all brush and lassos would be cleared upon recalculation of the coordinates; however, this is not easy to enforce.
Aaron Lun (14:24:58) (in thread): > Updated on Dropbox.
Kevin Rue-Albrecht (14:32:59) (in thread): > I don’t want to make a big deal of it, but I was always told that it was two words. Anyway both are common enough that I don’t mind
Aaron Lun (14:33:29) (in thread): > Boo. BOO!:ghost:
Kevin Rue-Albrecht (14:34:57) (in thread): > I’ve fixed it in AllClasses.R as part as an ongoing discrete -> categorical update (that I still haven’t pushed)
Aaron Lun (14:35:37) (in thread): > Just to be clear about that; you’re just changing the docs, right? No need to change the actual functions, esp. as ggplot2 uses discrete. Might as well be consistent if we’re going to be loose with our language.
Aaron Lun (14:43:44): > In any case, I’m going home.
Kevin Rue-Albrecht (17:42:02) (in thread): > Yes - just the docs. In other words, things like@all_discretestays as such. > The only things happening around functions are the indent, and removing some oldprintstatements that were already commented out anyway.
Kevin Rue-Albrecht (17:42:23) (in thread): > I had only kept them so far in case of need for debug
Aaron Lun (17:44:28) (in thread): > Okay. Watch out forplotting.R, that’s gone under a fair amount of work. Probably best to work fromcustomplot
Aaron Lun (17:44:46) (in thread): > It’ll be completed tomorrow, and ready to merge withmaster.
Kevin Rue-Albrecht (17:54:57) (in thread): > Ok. I’ve created a new branch from master locally “categorical”. But I can always merge customplot in it before I push anything
Kevin Rue-Albrecht (17:55:55) (in thread): > I’ll be on my way back to England tomorrow, as easyJet cancelled my Tuesday flights due to strikes in the north italian air control staff
Kevin Rue-Albrecht (17:56:59) (in thread): > Don’t wait on me to merge anything, I’ll see what’s there when I come back online
Aaron Lun (18:05:09) (in thread): > ;p;
Aaron Lun (18:05:12) (in thread): > lol
2018-05-06
Kevin Rue-Albrecht (11:49:43) (in thread): > On the bus back to Oxford. We were sitting ~2h on the plane in Napoli before we got a takeoff window, due to the thoughtful strikes of Marseille air traffic control
Kevin Rue-Albrecht (11:50:26) (in thread): > i’ll add that to my selection of answers for “do you ever think of going back to France?”
Kevin Rue-Albrecht (11:51:41) (in thread): > Anyway, I pushed thecategoricalbranch, which doesn’t include any functional change, only doc. Except for having mergedcustomplot, of course
Aaron Lun (12:40:45): > I am starting to feel that we should not have anyvalidatecalls in our plotting functions. Consider a plot that fails safely withvalidate. That’s all well and good, but the failure means that the stored coordinates inxydo not get added or updated inpObjects$all_coordinates. This causes problems if the current plot is transmitting a selection to any other plots, as the result ofbrushedPointsormgcv::inwill now be nonsensical or fail unsafely.
Aaron Lun (12:43:49): > This has not been a problem so far as the design of the interface means that the standard plots can never be invalid. The exception is with heatmaps, which may be invalidated with an empty list of features; or with custom column plots, which can be invalid if no custom function is selected. The former case is less of a problem as heatmaps can never transmit anyway. For the latter, I force the function to returnsomethingto fillall_coordinates, even if it is just an empty data.frame inxy.
Aaron Lun (12:45:44): > Regardless of what you think about that, the fact is that we don’t actuallyneedany of thevalidatecalls inplotting.R, because the user cannot possibly generate invalid inputs.
Aaron Lun (12:58:05) (in thread): > Or more simply just > > example("Custom iSEE plots", ask=FALSE) >
Aaron Lun (13:28:31): > Custom column plots merged and now available; enjoy the fun.
Aaron Lun (13:29:26): > It’s pretty craaazy
Federico Marini (14:32:08): > I’ll have a look later:slightly_smiling_face:
Aaron Lun (18:10:03): > ARE YOU NOT ENTERTAINED!?
Aaron Lun (18:10:27): > IS THAT NOT WHY YOU ARE HERE?
2018-05-07
Federico Marini (03:13:21): > eheheh:smile:
Federico Marini (03:13:43): > came for the entertainment, stayed for the …
Federico Marini (03:13:53): > code consistency?
Federico Marini (03:26:37): > I just checked the example:wink:
Federico Marini (04:58:44): > Small nice thing: picking up a line w kevin, can we work on finalizing the logo?
Federico Marini (04:59:12): > I could use the presentation for a couple of slides on this
Federico Marini (04:59:24): > one could be rafiki holding the hex sticker:slightly_smiling_face:
Federico Marini (05:01:34): > which brings us to the hexsticker topic - shall we vote for two figures to fill up the goggles?
Federico Marini (05:10:51): > I got it started over here:
Federico Marini (05:10:52): > https://github.com/federicomarini/BiocStickers - Attachment (GitHub): federicomarini/BiocStickers > BiocStickers - Stickers for some Bioconductor packages - feel free to contribute and/or modify.
Federico Marini (05:12:24): > and sent you the collaboration invitation so you can also get push access
Kevin Rue-Albrecht (05:23:33): > ‘started’ i can see a PNG and PDF files, but none of them display anything. Is that normal?
Federico Marini (05:26:23): > yep. I justtouched the files
Aaron Lun (07:20:11): > So, should we clear out thevalidatestatements fromplotting.R?
Aaron Lun (07:20:39): > Or otherwise replace them with a dummy plot, as incustom_plot.R.
Federico Marini (10:16:24): > https://imgflip.com/i/29srs8 - Attachment (Imgflip): I don’t always… > An image tagged i don’t always
Aaron Lun (10:41:33): > lol
Federico Marini (10:44:48): > if something funnier pops up to your minds, please share:slightly_smiling_face:
Federico Marini (10:45:08): > I’ll probably put one of these at the end of the prese
Federico Marini (10:45:54): > Since I am the last thing before the evening break, it’s worth to give them some nice reminder about the talk:smile:
Kevin Rue-Albrecht (12:05:07): > @Kevin Rue-Albrechtuploaded a file:“… when I do, I see”and commented: Stick the caption on this picture:stuck_out_tongue_winking_eye: - File (JPEG): “… when I do, I see”
Aaron Lun (12:06:15): > Well, if you guys have so much time on your hands, there are tests for downsampling and heatmap that are missing.
2018-05-08
Charlotte Soneson (03:16:15): > I think I changed all the American spelling to British…I think the only inconsistencies now areSummarizedExperimentandExperimentColorMap, but that’s not much to do about:slightly_smiling_face:I’ll merge it in a little while if no one complains. Talking to Federico yesterday we were wondering if we should try to submit so that the ms might be available before his presentation next week? I think it reads nicely, and Mark has OK-ed it as well.
Aaron Lun (04:13:39): > Waiting for an OK from John, will get one today. Still waiting on Vicky about the dev team from their end, but that process might be accelerated if we have an active submission. Can someone try linking overleaf to our git repo, or should I just upload the files manually?
Federico Marini (04:24:38): > Probably it is a one-time only thing
Federico Marini (04:24:52): > so I think there should be no need of linking to the git repo
Aaron Lun (04:25:35): > Probably not.
Aaron Lun (05:25:33): > Just nudged vicky about this.
Kevin Rue-Albrecht (05:31:08) (in thread): > Do you mean UK-US inconsistencies?
Charlotte Soneson (05:31:53) (in thread): > Yes, I changed all “-ize” to “-ise” etc, but I can’t really changeSummarizedExperiment…
Kevin Rue-Albrecht (05:32:09) (in thread): > I meant aboutExperimentColorMap, sorry
Charlotte Soneson (05:32:21) (in thread): > Also there, I changed all color to colour in the text
Kevin Rue-Albrecht (05:32:21) (in thread): > ahhhhcolor, sorry
Kevin Rue-Albrecht (05:33:09) (in thread): > I’ve always been trained with UK English, so I usually stick with that fairly robustly, but indeed for class names I do make the exception
Charlotte Soneson (05:34:26) (in thread): > Yeah…I don’t know what I’ve been trained with really, probably a mix depending on where I have been:slightly_smiling_face:I think I tend to use American spelling, but it is slowly changing by being around Australians…
Kevin Rue-Albrecht (05:38:13) (in thread): > I had a super carefully/picky (he says it himself: ‘pedantic’) Irish PhD supervisor, it was a consistent UK English with him. > He also had this awesome way to break ties about general spelling too: “Google the two spelling, take the one that returns most hits”
Kevin Rue-Albrecht (05:38:49) (in thread): > e.g. “Heatmap: About 3,820,000 results”, “heat map: About 15,100,000 results”
Charlotte Soneson (05:38:55) (in thread): > :slightly_smiling_face:
Kevin Rue-Albrecht (05:41:21) (in thread): > Sometimes it also works even more clearly when you google one, and Google suggests “Did you mean ”
Kevin Rue-Albrecht (05:42:12) (in thread): > Anyway, gotta dive back and debug a few pipelines today. Sometimes I’m tempted to rewrite stuff from scratch…
Aaron Lun (05:50:19): > Vicky is on the chase.
Aaron Lun (11:08:29): > Ugh. John says “sometime this week”.
Federico Marini (15:57:44): > oh, bummer
Aaron Lun (17:52:46): > Yeah.
Aaron Lun (17:52:55): > I’ve pressed the issue.
2018-05-09
Aaron Lun (17:36:00): > <!channel>We are go for submission tomorrow. Couple of comments for John that I’ll read.
Aaron Lun (17:36:30): > But keep your diaries open - I’ll do some final edits before lunch, and submit sometime after.
Aaron Lun (19:14:24): > Hello?
Aaron Lun (19:14:29): > Anyone home?
2018-05-10
Kevin Rue-Albrecht (04:16:19): > I’m here. I’m just completely blocked until me, the HPC manager, or Steve figure out why I can’t get a pipeline to run on the cluster:sweat:
Kevin Rue-Albrecht (04:17:27): > Can’t look at a screen anymore when I get home:face_vomiting:
Aaron Lun (04:22:37): > In any case, we should think of reviewers. I think I needed 5. Jimmy Mac is probably one.
Aaron Lun (04:22:52): > I haven’t talked to Mike Love about this, so perhaps he’s another?
Aaron Lun (04:23:33): > Or Ryan Thompson.
Aaron Lun (04:23:50): > But then it’s basically just another Bioconductor meeting group.
Kevin Rue-Albrecht (04:23:59): > Mike Love is among our Stargazers on GH. I suppose that’s alright?
Aaron Lun (04:24:09): > I think so.
Charlotte Soneson (04:30:00): > I haven’t talked to Mike about this either, but we do have other collaborations (both finished and ongoing)
Aaron Lun (04:38:14): > Oh. Well, that’s no good then.
Aaron Lun (04:44:34): > Did any of your PIs provide useful comments on the MS?
Aaron Lun (04:45:35): > What about Laurent as a reviewer?
Charlotte Soneson (04:46:10): > Mark said that it looked great:slightly_smiling_face:and pointed out that the links did not seem to go to the actual apps, but to the main shiny server
Charlotte Soneson (04:46:29): > Laurent sounds like a good reviewer indeed
Aaron Lun (04:46:50): > I still don’t see how the links go to the main server
Charlotte Soneson (04:47:16): > Ok. If I hover over one of the links in the pdf, the link I see stops at :3838
Charlotte Soneson (04:47:27): > But I think it may be because of the line break
Aaron Lun (04:47:59): > I don’t get anything in my PDF viewer.
Aaron Lun (04:48:13): > I think it’s just Preview’s problem.
Charlotte Soneson (04:48:18): > possible
Aaron Lun (04:48:19): > Or whatever you’re using.
Charlotte Soneson (04:48:53): > And in any case they will be replaced by the iframes so I don’t think it’s a problem
Charlotte Soneson (04:57:31): > For reviewers: Wolfgang? Someone from the shinyMethyl group? Stephanie Hicks? Alejandro Reyes?
Aaron Lun (04:58:59): > Dammit can’t spell kevin’s name
Aaron Lun (05:02:26): > I don’t know Alejandro well, so he probably is a good reviewer.
Aaron Lun (05:02:45): > @Federico MariniWhat’s the name for the German federal ministry … in german?
Charlotte Soneson (05:04:56): > Jeff Leek?
Aaron Lun (05:06:10): > I really don’t know him at all. Besides for some interaction during a Biostatistics submission
Charlotte Soneson (05:07:06): > More or less same for me.
Charlotte Soneson (05:10:09): > I guess we have all demoed the app in group meetings etc., and at least I got some comments from the group there, so I would maybe include the Robinson group in the acknowledgements - don’t know if it’s the same for the rest of you
Aaron Lun (05:14:13) (in thread): > Bundesministerium für Bildung und Forschung
Aaron Lun (05:15:33): > It is, but I wouldn’t bother thanking group members unless they gave me specific tangible comments.
Charlotte Soneson (05:21:48): > Sure. Mark definitely did give specific comments, but then we had a more general discussion following that, which involved many of the group members. I honestly don’t remember who said what, I’d rather be inclusive:slightly_smiling_face:
Aaron Lun (05:23:17): > Ugh. Well, it can be dealt with during review, I don’t want to have to upload the thing again. Besides, I think Marks’ group can be considered part of the Bioconductor community, so it’s technically covered.
Aaron Lun (05:23:29): > Anyway, ready to drop the bomb.
Charlotte Soneson (05:23:58): > Ah, it’s already uploaded. Ok, then go for it. Who did you suggest as reviewers in the end?
Aaron Lun (05:24:12): > That comes later.
Aaron Lun (05:26:18): > But does anyone actually know Jimmy Mac’s email?
Charlotte Soneson (05:31:43): > Nope
Aaron Lun (05:33:16): > Found it eventually.
Aaron Lun (05:35:32): > Alejandro has 3 different email addresses, dammit
Aaron Lun (05:36:49): > Okay, currently have James, Ryan, Laurent, Alejandro. Need 1 more.
Aaron Lun (05:37:43): > Guess we’ll go with Jeff, for a bit of variety.
Aaron Lun (05:39:31): > @Federico Marini@Kevin Rue-Albrechtconfirm that you haven’t collaborated with the guys above in the past 3 years.
Aaron Lun (05:41:36): > iSEE2018 is now public.
Kevin Rue-Albrecht (06:09:13): > I can confirm.. considering that I can’t even tell what last names you mean for James and Ryan. I assume Laurent Gatto? (I found Alejandro Reyes earlier in the conversation)
Aaron Lun (06:15:34): > James MacDonald and Ryan Thompson
Aaron Lun (06:15:48): > Yes, Laurent Gatto, Alejandro Reyes, and Jeff Leek.
Kevin Rue-Albrecht (06:15:53): > ok that’s a definite no, then, about collaborations with them
Kevin Rue-Albrecht (06:18:35): > and yes, I presented a very early version of iSEE at a lab meeting (in January), but I didn’t get constructive feedback. Rather reminders to give higher priority to internal group projects:confused:
Aaron Lun (06:24:33): > Come on Fed, wake up.
Charlotte Soneson (06:26:15): > Just FYI, it is a public holiday on the continent today
Aaron Lun (06:31:28): > WHAT
Aaron Lun (06:31:44): > I didn’t realize Eurovision was that important
Aaron Lun (06:31:52): > It’s not even the final yet
Charlotte Soneson (06:33:48): > Eh well, we need to prepare for the semifinal (although not Germany I guess, since they are already in the final by default)
Kevin Rue-Albrecht (06:42:32): > Just checking: I can see the submission listed in “My Research”, but I can click on it or anything (e.g. see the manuscript). Is that normal?
Federico Marini (06:46:50) (in thread): > green light from my side
Federico Marini (06:47:08): > sorry guys I was missing around
Federico Marini (06:47:39): > got news from my mum that grandma is since yesterday on morphine pump with continuous flow
Kevin Rue-Albrecht (06:48:05): > oww. i’m really sorry to hear it
Federico Marini (06:48:10): > and I am currently on the way (rail) to italy to try and say a proper goodbye
Charlotte Soneson (06:48:41): > so sorry:disappointed:
Federico Marini (06:48:42): > feels awful per se, plus that it comes down so quick and unexpected
Federico Marini (06:49:08): > and mostly concurring with a plan to have a few days with the kids around before leaving direction budapest
Federico Marini (06:49:37): > thank you folks:pray:
Federico Marini (06:50:01) (in thread): > BMBF, yes, that is right
Federico Marini (06:50:58): > other good names: Luke Zappia from the oshlack group?
Federico Marini (06:51:57): > Lorena Pantano or someone from that group?
Federico Marini (06:52:17): > Ana Conesa or whomever from the spanish cohort of RNA-seq persons?
Federico Marini (06:52:41): > Kasper Hansen? Vince Carey?
Federico Marini (06:52:52): > ok, I’m done with the shotgun:slightly_smiling_face:
Aaron Lun (06:53:12): > Luke might be good. Though I haven’t forgotten how he badmouthed my simulations in the splatter paper.
Aaron Lun (06:53:32): > Also Lorena. Don’t think I’ve spoken to Ana.
Aaron Lun (06:54:21) (in thread): > Overleaf seems to have created a new F1000 account for me, so I can’t even get to it using my old account. ¯*(ツ)*/¯
Federico Marini (06:54:25): > Jeff Leek might be good even if he’S not that much into “this this”
Federico Marini (06:54:43): > a good one: Leonardo Collado Torres
Federico Marini (06:54:43): > the recount guy
Aaron Lun (06:54:48): > Yes, I thought of him too.
Federico Marini (06:55:01): > Levi Waldron could also be excellent
Federico Marini (06:56:27): > If more names pop up, I’ll send them around
Federico Marini (06:58:54): > or Sandrine, Dudoit?
Aaron Lun (06:58:58): > I’ll throw Luke, Lorena, Ana and Leo into the mix. That’s 9, that should keep them happy.
Federico Marini (06:58:58): > Davide Risso, but he might be too close to us?
Aaron Lun (06:59:02): > Yeah, too close.
Federico Marini (06:59:12): > damn, italian connection:smile:
Kevin Rue-Albrecht (06:59:16) (in thread): > ouch. well i didn’t have an F1000 account yet, so I just created one with my kennedy email, and immediately saw the entry in “my submissions’
Kevin Rue-Albrecht (06:59:22) (in thread): > oh well
Federico Marini (06:59:26): > Okie dokie
Federico Marini (06:59:43): > sorry, connection has more hiccups than expected
Aaron Lun (07:00:01): > @Kevin Rue-Albrecht@Charlotte Sonesonplease check if you’re on a paper with luke, lorena, ana or leo.
Federico Marini (07:00:46): > I’ll be offline for a bit on the swiss trains later
Charlotte Soneson (07:01:08): > Yes, they aren’t as modern as the German ones with internet connection…
Federico Marini (07:01:48): > Should you need me for anything urgent, please do ping me on the mobile - i think it is safe to post it here on slack?
Charlotte Soneson (07:02:36): > I am involved in a kind of collaboration with Leo (or at least the Jaffe group), but we haven’t actually spoken. No papers with the others, I was the external expert for the defence of one of Ana’s PhD students some years ago.
Federico Marini (07:03:03): > :iphone:+49 157 84730828
Kevin Rue-Albrecht (07:33:30) (in thread): > I’m not on a paper with any of them.
Aaron Lun (07:49:00): > Okay. final reviewer rundown. James MacDonald, Ryan Thompson, Laurent Gatto, Alejandro Reyes, Jeff Leek, Luke Zappia, Ana Conesa and Lorena Pantano.
Aaron Lun (07:49:20): > 8 refs should be enough.
Kevin Rue-Albrecht (07:53:22): > That looks all good to me. I’ve met and/or chatted with some, but no shared publications or project
Aaron Lun (07:58:55): > Gender balance is a bit fucked up, but meh.
Aaron Lun (07:59:13): > Submitting in exactly 10 minutes.
Kevin Rue-Albrecht (07:59:53): > the 10th at 13:10, or something like that?
Aaron Lun (08:00:02): > Dunno.
Aaron Lun (08:14:19): > BOOM
Aaron Lun (08:14:20): > Done.
Aaron Lun (08:55:09): > I’ll send something to the OpenAccess team now, actually.,
Aaron Lun (08:55:17): > Toilet first, tho
Aaron Lun (09:48:55): > “Please provide a brief summary of your paper for a non-specialist audience. This will help us in our efforts to communicate your Cancer Research UK-funded research more widely.”
Aaron Lun (09:49:05): > > Data are to scientists what toys are to young children; we like to be able to see it, touch it, play with it. This process of “data exploration” is essential for achieving a deep understanding of the data set and, ultimately, of the underlying truth regarding biological processes and disease mechanisms. The iSEE software package provides a general, flexible and customizable tool for interactive visualization of high-throughput biological data sets, with applications ranging from exploring gene expression in cancer patients to inspecting protein expression in single cells. The iSEE interface allows scientists to explore and present experimental data, metadata and analysis results (and the relationships between them) in an easy and intuitive manner, while supporting best practices in reproducibility by tracking the code used for each visualization.
Charlotte Soneson (09:55:12): > :+1:
Kevin Rue-Albrecht (11:11:01): > i like it !
Aaron Lun (14:11:37): > Currently waiting for F1000 to greenlight the paper for production. This will allow me to ask the OA team to accept the invoice for the publication charge. Once the invoice is processed, proofs will then come back from F1000, which we will have to approve, etc. etc. Each step has some delay, so I really don’t think this will be done before the 14th.
Aaron Lun (14:13:45): > On another note - message from Vicky: > > Sorry for the delay, we’ve just been discussing the best way to go about this. The good news is that we will be able to expand the iframe- it should work like the rest of the popups in the article page as below- but we also have other options if this isn’t suitable. > > > > We’re still working out how to implement this and it is most likely looking like it will be the end of our next sprint, which will be the end of June. However if you want to go ahead with your submission that’s completely fine and we can just apply this after your article has been published (if that is what happens first) or instead hold it at production stage till we’re ready to put this in place.
Aaron Lun (14:13:58): > … which I just said okay to.
2018-05-11
Federico Marini (11:31:02): > Chiming in later on this, or I don’t know whether we already touched it
Federico Marini (11:31:13): > Do we split the publication fee?
Aaron Lun (11:32:56): > I will check whether UCam’s OpenAccess team will pay for it.
Kevin Rue-Albrecht (11:34:28): > A while back I checked with an Oxford OA website, but they would not have covered oxford bioinfo, I could check again whether F1000 would fall within their scope
Federico Marini (11:35:47): > I can ask the team at the unimedizin as well. They have specific funds for that
Federico Marini (11:36:48): > Thing is , “can we split it”? Alla romana, or any other option, but is it possible for F1000 to get that fee from more sources
Aaron Lun (11:37:26): > I don’t think we need to complicate this. Besides, I’m corresponding, so at least give me the privilege of paying.
Aaron Lun (11:37:40): > Got to have done something.
Charlotte Soneson (11:37:45): > We should have 50% off the publication fee since I (and Aaron too, I think) have reviewed for them the last year
Kevin Rue-Albrecht (11:37:46): > lol
Kevin Rue-Albrecht (11:38:17): > FWIW I was looking athttp://openaccess.ox.ac.uk/wellcome-and-coaf/ - Attachment (openaccess.ox.ac.uk): Wellcome Trust and Charity Open Access Fund (COAF) > Current Policy What do I do at Oxford? Oxford has received a Wellcome/COAF block grant for 1 October 2017 to 30 September 2018 to
Aaron Lun (11:39:01): > Yeah, I’ve already filled out UCam’s equivalent.
Kevin Rue-Albrecht (11:39:02): > Good to know that F1000 Research is in the list, if only for future reference (Thanks Aaron!)
Kevin Rue-Albrecht (11:40:55): > But good opportunity for me to learn about this kind of publication support. I might contact them, even if only to know more about it
Federico Marini (11:44:32): > Anyway, thank for pushing & submitting
Federico Marini (11:45:01): > I’ll check last minute on tuesday at 16.20 shortly before the presentation whether it is already out:slightly_smiling_face:
Federico Marini (11:45:21): > and if this is the case, add it - or mention it properly:wink:
Aaron Lun (11:49:29): > I would doubt it - turnaround usually takes a week, and that’s with all steps going smoothly.
Aaron Lun (11:49:40): > But you never know. Maybe they work on WE’s.
Aaron Lun (11:49:55): > Well, the University’s OA team definitely won’t.
Federico Marini (11:50:00): > eheheeh
2018-05-12
Kevin Rue-Albrecht (07:10:22): > I gotta say, reading the confirmation email from F1000: “Before accepting the article: we will check […]; and that there is sufficient detail to enable others to replicate the study (if applicable).” > I think we’re covered on this last one:slightly_smiling_face:
Federico Marini (13:20:29): > the young lady sitting in front of me on the train just whisperediSEE, iSEEwhile I was finishing up the presentation
Federico Marini (13:20:54): > I needed to check whether my sunglasses were way too mirrory:smile:
Aaron Lun (13:21:14): > lol
Aaron Lun (13:21:58): > Or she really likes ion chromatogrpahy
Aaron Lun (13:22:05): > or integrated circuits
Federico Marini (13:22:51): > I’mma pick “bioc developer with clear insights on who was included in the recent release”
Aaron Lun (14:21:04): > eh?
Federico Marini (14:22:20): > just profilin’ her
Kevin Rue-Albrecht (14:26:19): > Maybe she was profiling you Fede… maybe she’s preparing competition foriSEE.. somethink likeBLIND… Bayesian Latent Inference of Negative-binomial Distributions:stuck_out_tongue::stuck_out_tongue::stuck_out_tongue:
2018-05-13
Federico Marini (08:04:30): > hey iSEE team
Federico Marini (08:05:03): > if you write me here your physical (work) address, I’m happy to send you a postcard from Budapest:slightly_smiling_face:
Kevin Rue-Albrecht (08:19:04): > Thats’s kind of you! Here goes: > Kennedy Institute Of Rheumatology > Roosevelt Drive, Oxford OX3 7FY
Kevin Rue-Albrecht (08:20:16): > I’m half-tempted to give you my home address though, but given that my Nature Methods subscription reaches me at work, I’ll trust them to accept a postcard:slightly_smiling_face:
Federico Marini (10:11:08): > :slightly_smiling_face:
Kevin Rue-Albrecht (12:38:25): > @Kevin Rue-Albrechtuploaded a file:sizeFactor issue during sanitize?and commented:@Aaron Lun, I think we got a bug with sizeFactors in iSEE. - File (PNG): sizeFactor issue during sanitize?
Kevin Rue-Albrecht (12:38:43): > ERCC should be quoted
Aaron Lun (12:38:46): > Yes, there shoul dbe quotes there
Aaron Lun (12:39:00): > probably missing adeparse
Kevin Rue-Albrecht (12:39:15): > I’m checking with you, but I think I’m up to date with all packages
Aaron Lun (12:39:39): > Should be a simple fix. Just dig through.sanitize_memoryand add the deparse where it reports it.
Kevin Rue-Albrecht (12:39:53): > ok, so i’m not crazy or out of date
Kevin Rue-Albrecht (12:40:46): > should be fixed in a single commit, and probably pushed to release as well, noticeable bug for those who work with this
Aaron Lun (12:41:40): > Can you do that?
Kevin Rue-Albrecht (12:42:36): > I mean cherry pick the commit to release, and bump up there as well. You know, for important fixes on the release branches
Kevin Rue-Albrecht (12:43:30): > I prefer to avoid when I can, but yes, I’ve done it before
Aaron Lun (12:52:15): > Okay, good. Should be a one-liner once you find it.
Kevin Rue-Albrecht (12:52:33): > spotted it already - just building locally for testing
Kevin Rue-Albrecht (12:53:23): > amusingly, a few lines below, thespikeNameswere deparsed correctly. A case of missed copy-paste:wink:
Aaron Lun (12:55:23): > I remember when I added that function. I was double-team teaching a stats course with Cata Vallejos, and I got bored.
Kevin Rue-Albrecht (12:55:42): > hehe.. just teasin’
Kevin Rue-Albrecht (12:55:49): > works now
Kevin Rue-Albrecht (12:56:40): > just wondering about making the new colnames cleaner, though, because of spaces and brackets turning into dots (screenshot incomign)
Kevin Rue-Albrecht (12:56:47): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-13 17.55.59.png - File (PNG): Screenshot 2018-05-13 17.55.59.png
Aaron Lun (13:04:22): > Guh
Aaron Lun (13:04:41): > what the fuck is datatable doing
Kevin Rue-Albrecht (13:04:44): > god, forgot about the set of commits you sent in last week. Oh well, doesn’t prevent me fromcherry-picking things to release
Aaron Lun (13:04:46): > see if we can turn offcheck.names
Kevin Rue-Albrecht (13:06:19): > looks like it: > > > data.frame("sizeFactors(se)" = 1, check.names = FALSE) > sizeFactors(se) > 1 1 >
Kevin Rue-Albrecht (13:08:24): > although it’s weird: > > > df <- data.frame(a =1, b=2) > > df[,"sizeFactors(se)"] <- 3 > > df > a b sizeFactors(se) > 1 1 2 3 > > [<-doesn’t seem to have a problem with names here
Aaron Lun (13:10:49): > No, it’s not there. It’s probably inDT
Kevin Rue-Albrecht (13:11:21): > argh, got you
Kevin Rue-Albrecht (13:11:32): > for a moment, I thought it was aDataFramething
Kevin Rue-Albrecht (13:16:48): > damn.. seems like i mixed a few things in my “quotes” branch. Need to clear the slate and make a new clean branch -_- > I was toying with docker on a separate branch. I don’t get how that commit got in here
Kevin Rue-Albrecht (13:23:12): > alright, that’s one thing in placequotes2is the one to merge. I’ll delete the other one.
Kevin Rue-Albrecht (13:24:40): > Btw, thanks to Davis’ help a few weeks ago, I’m getting the hang of docker containers. I was doing them manually and locally so far, but I’ve started testing automated builds today. Still running though.
Kevin Rue-Albrecht (13:25:18): > Don’t if you guys can see this link, I think it’s public:https://hub.docker.com/r/kevinrue/isee/builds/
Kevin Rue-Albrecht (13:29:15): > The only limitation I’ve found so far was that the Docker container that I made with the TCGA data set crashed before showing any data. But I think thatdocker run ... --memory="10G"would solve that (value to be defined for memory, all I know is that the default 2G does not seem to be enough)
Kevin Rue-Albrecht (13:29:52): > From what I could see, a container with theallendata was using ~1G at runtime
Kevin Rue-Albrecht (13:53:16): > quick debug note (I started cooking here)print(head(tmp_gene_data))athttps://github.com/csoneson/iSEE/blob/quotes2/R/iSEE-main.R#L1366already shows that thecolnamesare ‘dotted’ - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Kevin Rue-Albrecht (13:58:21): > yeah ok - it’s thegene_data <- as.data.frame(rowData(se))that messes up things
Kevin Rue-Albrecht (13:59:29): > Seems likedata.frame(rowData(sce), check.names = FALSE)solves it. Note thatas.data.framedoes not have acheck.namesargument
Aaron Lun (13:59:58): > yes.
Aaron Lun (14:00:00): > good
Kevin Rue-Albrecht (14:00:58): > FWIW:DThandles ‘fancy’ colnames seemlessly
Kevin Rue-Albrecht (14:03:12): > Looks like Docker completed:https://hub.docker.com/r/kevinrue/isee/
Kevin Rue-Albrecht (14:05:34): > I’ll explain more later, but this basically means that people can build docker apps that include a custom data set and tour by pulling this image, and adding at least 3 files: 1) Dockerfile, 2) launcher.R script, and 3) sce.rds. > Bonus is 4) tour.txt
Kevin Rue-Albrecht (14:06:00): > e.g.https://hub.docker.com/r/kevinrue/isee_allen/
Kevin Rue-Albrecht (19:13:50): > I’ll leavequotes2open for now, if anyone wants to throw in a nested DF to extend code coverage a bit more
2018-05-14
Federico Marini (04:07:53): > Folks, one thing I noticed - sadly only now
Federico Marini (04:08:09): > we did not update devel to remove featExpr and rename it to featAssay
Federico Marini (04:08:25): > release is fine
Federico Marini (04:08:32): > but devel is still the old one
Kevin Rue-Albrecht (04:18:47): > argh is that so? I didn’t pay attention when running the app. > or is it just because I haven’t pusheddevelupstream?
Kevin Rue-Albrecht (04:19:36): > There was more activity on devel, so I only pushed the fixes to release at the time.
Aaron Lun (04:21:00): > well, someone should feel free to take care of it. I am busy all day today; new student.
Kevin Rue-Albrecht (04:21:16): > I can see the commit onmaster(i.e., devel) too :https://github.com/csoneson/iSEE/commit/514e330b56aa4974436d04a4e4107c631db2cac3 - Attachment (GitHub): Changed ‘Feature expression’ to ‘Feature assay’ plot name. · csoneson/iSEE@514e330 > Changed ‘featExpr’ to ‘featAssay’ in variables, arguments.
Kevin Rue-Albrecht (04:26:39): > (which makes sense, as every commit should go to master/devel first, and only then be cherry-picked back to release, where applicable)
Federico Marini (05:02:54): > true, but somehow we’re stuck at v1.1.0
Kevin Rue-Albrecht (05:03:59): > ? DESCRIPTION says 1.1.1 on GH, it’s just a matter of pushing upstream.
Federico Marini (05:04:30): > yep, I meant on Bioc devel
Federico Marini (05:05:20): > Still, users should try out release, and if they want true bleeding, they can go for github
Kevin Rue-Albrecht (05:05:45): > I found how to set up a DataFrame in colData late last night, that covers a bit more code in existing unit tests wheniSEE()is called. I would just like to add a couple of more specific unit tests for the specific functions in iSEE-extras.R
Aaron Lun (07:55:08): > ATTENTION ALL.
Aaron Lun (07:55:16): > Editorial team’s minor edits are visible in the latest commit.
Federico Marini (07:57:43): > uh
Federico Marini (07:58:10): > closing in?
Aaron Lun (08:00:16): > I will make a PR filling in the changes. Also need to re-add Mark Robinson to the acknowledgements, they deleted him.
Aaron Lun (08:00:40): > Fortunately iSEE already has a DOI.
Aaron Lun (08:01:13): > Are these DOI’s version specific? They can’t possibly be.
Federico Marini (08:06:17): > why should mark be gone?
Federico Marini (08:06:53): > doi should be fixed
Federico Marini (08:07:01): > it is the same for both rel and devel, as of now
Federico Marini (08:07:35): > Shall we tag release versions, plus the versioj at submission time?
Aaron Lun (08:08:06): > We need to provide archived source code at submission.
Aaron Lun (08:08:17): > which is fine, I’ll just point them to the URL.
Federico Marini (08:12:17): > okiedokie
Aaron Lun (08:16:54): > @Kevin Rue-Albrecht@Charlotte Soneson@Federico MariniSeeeditedfor the requested fixes. You have 40 minutes before I have a long meeting with someone else, so I want to get this done before then.
Kevin Rue-Albrecht (08:17:57) (in thread): > DOI are version specific, AFAIK, but they also produce one that always points to the latest version
Kevin Rue-Albrecht (08:18:25) (in thread): > oh wait you meant for the package, not the manuscript
Federico Marini (08:21:33): > spelling: license vs licence, what is the final choice?
Federico Marini (08:21:42): > they said ‘c’, we have ‘s’
Kevin Rue-Albrecht (08:21:44): > i think ‘s’ no?
Federico Marini (08:21:53): > otherwise, I’m fine with the rest of the edits
Kevin Rue-Albrecht (08:22:10): > oh no wait
Kevin Rue-Albrecht (08:22:15): > “license = grant a licence to.”
Kevin Rue-Albrecht (08:22:22): > (dixit Google)
Kevin Rue-Albrecht (08:23:21): > I don’t have a problem with the edits that I see onedited. Where did you get the Editorial team’s minor edits from@Aaron Lun?
Federico Marini (08:23:26): > hm
Aaron Lun (08:23:32): > OL
Federico Marini (08:23:43): > but we have aLICENSEfile in the pkg
Kevin Rue-Albrecht (08:23:53): > Anything we’re supposed to do? Seems all pretty minor, so I don’t see anything I could add
Federico Marini (08:24:35): > (which btw we need to edit a little, it is still in the draft form, see the output of$ head LICENSE
Charlotte Soneson (08:24:50): > Yeah, edits look fine to me too. I guess I should explicitly check that Mark is fine with being named in the Acknowledgements, I sent a message but I think he’s in a committee meeting right now. Can’t imagine that it should be a problem though
Aaron Lun (08:25:05): > Someone fix the license and tell me when it’s done.
Federico Marini (08:25:33): > I would but can’t - somewhat the wifi here is blocking git push
Federico Marini (08:25:49): > (and i am fine with any order)
Aaron Lun (08:26:03): > Oh, if that’s the only change, then I’ll just throw it in.
Federico Marini (08:26:25): > just this line basically : > > Copyright (c) 2017 Aaron Lun, Charlotte Soneson, Federico Marini, Kevin > Rue-Albrecht (left in rigorous alphabetical order :slightly_smiling_face: ) >
Kevin Rue-Albrecht (08:27:53): > Btw, on OL, I can only see a version that was last edited 14 days ago by Fede (https://www.overleaf.com/15765478mqsvydndbhfv#/59995311/)
Aaron Lun (08:28:18): > No, this is a private thingy.
Aaron Lun (08:28:23): > Do youwantto look at it?
Aaron Lun (08:28:34): > I mean, I just overwrote the tex file.
Kevin Rue-Albrecht (08:28:35): > ah ok forget it then
Kevin Rue-Albrecht (08:29:14): > Was mostly curiosity. I can compiled the PDF locally, if it’s just to have the updated PDF
Aaron Lun (08:32:23): > Well, you can do that onmaster
Aaron Lun (08:32:27): > It’s literally the same shit
Aaron Lun (08:35:12): > Nonetheless, can someone clean up the LICENSE to make it a bit more sober?
Aaron Lun (08:40:06): > Any edits onedited?
Aaron Lun (08:42:29): > Okay, well, it’s merged in.
Aaron Lun (08:45:25): > Going in after I come back from the toilet.
Aaron Lun (08:54:45): > Going once, going twice…
Aaron Lun (08:54:53): > You know, it wouldn’t hurt to have a thumbs up or something.
Charlotte Soneson (08:55:20): > Go for it. I haven’t gotten a reply from Mark but I don’t think it will be any problems
Aaron Lun (08:55:46): > I seriously doubt that anyone would complain about being present in the Acknow.
Aaron Lun (08:56:13): > Well, we can wait if you feel it’s a thing
Charlotte Soneson (08:56:55): > I think it’s fine. It may be 1-2 hours until he’s out of the meeting anyway.
Aaron Lun (08:57:25): > Okay, I”m going to go for it then. They won’t publish until they get paid anyway!
Charlotte Soneson (08:57:37): > Exactly:slightly_smiling_face:
Aaron Lun (08:58:36): > It is done.
Aaron Lun (08:58:47): > Waiting for our OA team to get back to me with money details.
Charlotte Soneson (08:58:49): > :+1:
Federico Marini (09:06:57): > cool-io
Kevin Rue-Albrecht (09:11:27): > Thanks!
Kevin Rue-Albrecht (09:12:48): > Sorry, I’m knee deep into an issue compiling R-3.5.0 on our cluster. Can’t figure out whymakefails with an error that includes/gfs/apps/src/R-3.5.0/src/main/grep.c:108: undefined reference topcre_assign_jit_stack’`. > The error is reported in various places, but I can’t find a clear fix for it
Federico Marini (09:13:21): > do you need source? otherwise I use marutter’s ppa
Kevin Rue-Albrecht (09:13:52): > Yeah. We’ve got this ‘module’ system
Kevin Rue-Albrecht (09:14:16): > Need to compile from source and link modules to individual installs
Federico Marini (09:14:16): > dEEEmn
Federico Marini (09:28:06): > huh. bad news folks (at least, folks wanting a sticker too): there is one sticker/person as of now
Federico Marini (09:28:18): > dunno if they throw in more later on
Kevin Rue-Albrecht (09:28:59): > haha no worries
Federico Marini (09:29:40): > on top of somewhat crappy wifi, but well
Charlotte Soneson (09:32:07): > Are they trying to make you focus on the talks?:wink:
Federico Marini (09:33:19): > today it is workshop day
Federico Marini (09:33:24): > so yes
Federico Marini (09:33:44): > but sometimes you just want to google things to help yourself in the hands on, dammit
Charlotte Soneson (11:40:33) (in thread): > Mark replied, all good, as expected.
Aaron Lun (13:36:17): > You guys can merge on iSEE2018, right?
Aaron Lun (13:36:36): > I realized that I had permissions to push directly tomaster, even though you guys probably don’t
Charlotte Soneson (13:36:42): > I think I have, yes
Kevin Rue-Albrecht (13:48:54): > I think we all have permission onmaster. I sort of remember doing it myself at the very start
Kevin Rue-Albrecht (13:49:22): > anyone wants to check my PR, or can I merge it right now?
Aaron Lun (13:50:04): > Nike.
Kevin Rue-Albrecht (13:50:15): > I’ve standardised the use of fonts (italics instead of bold), while adding color to reference panel types
Aaron Lun (13:50:27): > Oh jesus
Kevin Rue-Albrecht (13:50:48): > is that a no?
Aaron Lun (13:51:01): > Keeping the colours in sync will be a real pain
Aaron Lun (13:51:13): > As we add new panels, the old colours may not be preserved
Aaron Lun (13:51:24): > as they occupy too many extremes on the colour space
Aaron Lun (13:51:34): > so we’ll have to jiggle the colours a bit to make space for new panel types
Kevin Rue-Albrecht (13:51:34): > mhh i see (no joke)
Aaron Lun (13:51:39): > e.g., genome vis.
Aaron Lun (13:51:47): > I mean, the custom panel already is a bit of a stupid colour
Aaron Lun (13:51:57): > it’s like a shit, basically,.
Aaron Lun (13:52:32): > Perhaps just use one innocuous colour to refer to any UI elements.
Aaron Lun (13:52:44): > Blue, perhaps?
Aaron Lun (13:53:01): > Blue + italics should make it stand out in a separate manner from bold.
Aaron Lun (13:53:09): > But don’t overuse it, otherwise the whole thing will show up as blue.
Kevin Rue-Albrecht (13:53:56): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-14 18.53.31.pngand commented: darn, it was looking cute with the current colors, but I get - File (PNG): Screenshot 2018-05-14 18.53.31.png
Kevin Rue-Albrecht (13:54:00): > at the moment it looks like this
Kevin Rue-Albrecht (13:54:21): > that’s exactly the color code of the blue used for redDim panels right now
Aaron Lun (13:54:21): > Oh, that’s pretty nice
Kevin Rue-Albrecht (13:54:40) (in thread): > i’ll just say that you picked the color:wink:
Aaron Lun (13:54:52) (in thread): > But just use one colour for all UI elements.
Kevin Rue-Albrecht (13:55:09) (in thread): > sure
Aaron Lun (13:55:25) (in thread): > Bold we definitely need to keep for actions. Either that, or use bold + red for actionable elements.
Aaron Lun (13:55:38) (in thread): > … which might not be a bad idea.
Kevin Rue-Albrecht (13:58:20) (in thread): > Right. For now, I’ll just switch the color to blue in the TCGA tour. > I’m not particularly fond of messing with the other tours, as we’ve kind of been responsible for one each. > However that also resulted in the current difficulty to keep them synced
Aaron Lun (13:58:38) (in thread): > Okay. How about this. Blue normal text for UI elements, bold text for Action to be performed, red bold text for warnings (should there be any).@Federico Marini@Charlotte Sonesonthoughts?
Kevin Rue-Albrecht (13:59:27) (in thread): > I’ve got a “Note” in mine. > At the moment, it’s all normal font (except for a word in the note that I meant to emphasise)
Kevin Rue-Albrecht (14:00:32): > @Kevin Rue-Albrechtuploaded a file:Tour “Note” - File (PNG): Tour “Note”
Charlotte Soneson (14:01:23): > I have no problem with that scheme
Aaron Lun (14:04:38) (in thread): > You could probably say “Note that”
Kevin Rue-Albrecht (14:06:26) (in thread): > Good point. I think the current version is a remnant of my initial attempt using an italic font.
Kevin Rue-Albrecht (14:07:46) (in thread): > i like the result. incoming
Aaron Lun (14:23:38): > What colour do people want to use for red? Note that blue will also clash with hyperlink blue.
Aaron Lun (14:24:44): > Suggest pure red,[color content]
Kevin Rue-Albrecht (14:27:44): > pure red looks fine for the warnings.
Aaron Lun (14:28:04): > I suggest the disclaimer at the front should be a pure red.
Kevin Rue-Albrecht (14:28:47): > what disclaimer?
Aaron Lun (14:29:09): > Noting that users should restart the tour if it isn’t working. Don’t you have one on yours.
Kevin Rue-Albrecht (14:30:45): > The CyTOF either, from what I can see
Kevin Rue-Albrecht (14:33:16): > and separately, we’re coming back on the ‘blue for panel names’, which clashes with hypertext? Maybe darker blue?
Aaron Lun (14:34:30): > A darker blue would work
Kevin Rue-Albrecht (14:35:51): > Btw, did I miss something: the only disclaimer I can see about tour restart is in your PBMC, right?
Kevin Rue-Albrecht (14:36:42): > How do you like#16048c(seehttps://www.google.com/search?client=safari&rls=en&q=color+picker&ie=UTF-8&oe=UTF-8)
Aaron Lun (14:38:38): > Slack will do this for yout,[color content]
Aaron Lun (14:39:23): > That looks a bit too dark? is it distinguishable from black?
Kevin Rue-Albrecht (14:39:26): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-14 19.39.18.png - File (PNG): Screenshot 2018-05-14 19.39.18.png
Kevin Rue-Albrecht (14:39:34): > exactly what I was going to say, in fact
Aaron Lun (14:39:36): > Yeah, i can’t see that easily.
Kevin Rue-Albrecht (14:39:48): > even zoomed like this, barely different
Aaron Lun (14:39:55): > Perhaps something like[color content]
Kevin Rue-Albrecht (14:40:11): > yep, there’s still space to lighten before hitting the hypertext
Kevin Rue-Albrecht (14:41:26): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-14 19.41.14.png - File (PNG): Screenshot 2018-05-14 19.41.14.png
Kevin Rue-Albrecht (14:41:31): > chicken dinner i think
Aaron Lun (14:41:36): > Good enough.
Aaron Lun (14:41:53): > I would say, though, to make references to UI elements sparingly. That’s because the tour inherently references UI elements by its very nature, so it is not usually necessary to refer to it again in the text, unless for pedagogical purposes.
Aaron Lun (14:42:14): > So, for example, the PBMC tour just says, “This box” or “This plot”
Kevin Rue-Albrecht (14:42:22): > yep, i initially meant to help references to linked panels
Kevin Rue-Albrecht (14:42:59): > “remember that orange panel from before?”
Aaron Lun (14:43:46): > yeah okay
Kevin Rue-Albrecht (14:43:57): > which is kinda lost now that we apply a uniform color, actually
Kevin Rue-Albrecht (14:44:52): > now that I think of it, I could just italicize those names in regular black color, likeVisual parametersis already done
Aaron Lun (14:45:57): > Well, italics is useful elsehwere, e.g., gene names.
Aaron Lun (14:46:01): > package names
Aaron Lun (14:46:02): > method names
Aaron Lun (14:46:30): > Maybe we could give it a grey background shading?
Kevin Rue-Albrecht (14:46:45): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-14 19.46.28.pngand commented: is that what you were expecting? - File (PNG): Screenshot 2018-05-14 19.46.28.png
Aaron Lun (14:47:27): > Yes, but I will update the disclaimer, so don’t change that yet.
Kevin Rue-Albrecht (14:48:26) (in thread): > alright, need to summarize again here: we’re talking about panel names in blue font on a grey background?
Kevin Rue-Albrecht (14:49:00) (in thread): > sounds a bit much, or I misunderstood something here
Aaron Lun (14:49:17) (in thread): > Black font, grey background.
Kevin Rue-Albrecht (14:50:01) (in thread): > rightgoes to check w3schools:musical_note:
Kevin Rue-Albrecht (14:52:49) (in thread): > ah it’s not<font>as for the color, but<span>(if anyone else didn’t know)
Aaron Lun (14:54:32) (in thread): > fuck it, I reached critical “don’t care” threshold.
Aaron Lun (14:56:19): > Okay. Pushed PBMC.
Aaron Lun (14:56:47): > Could you open the wiki page to put down these guidelines?
Kevin Rue-Albrecht (14:56:58) (in thread): > ok well just before you wander off, I’ll point out that I just said crap: doesn’t seem to work. I got it from that page:https://www.computerhope.com/issues/ch001391.htm - Attachment (computerhope.com): Highlighting text in color using HTML and CSS > Steps and information on highlighting text in any color using HTML and CSS on your web page.
Kevin Rue-Albrecht (14:58:07) (in thread): > Meanwhile, their<mark>looks nice (a sort of pale yellow)
Kevin Rue-Albrecht (14:58:28): > open? you mean write one?
Aaron Lun (14:58:35): > yeah…
Aaron Lun (14:58:39): > Or something.
Aaron Lun (14:58:47): > Well, you know waht, I’ll od it.
Kevin Rue-Albrecht (14:59:44): > As you wish, I was just trying to get that damn highlighting before having dinner (it’s basically ready, so I’m running out of time)
Aaron Lun (14:59:57): > Don’t bother.
Kevin Rue-Albrecht (15:00:12): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-14 20.00.01.png - File (PNG): Screenshot 2018-05-14 20.00.01.png
Kevin Rue-Albrecht (15:00:20): > that’s the<mark>HTML
Kevin Rue-Albrecht (15:01:02): > I think I just figured how to change the default yellow
Aaron Lun (15:01:34): > Don’t think that looks any better, TBH, but let me know.
Aaron Lun (15:02:58) (in thread): > It looked a lot stronger on their screen
Kevin Rue-Albrecht (15:03:13) (in thread): > true.
Aaron Lun (15:03:25): > Certainly easier than writing out<font ...>
Kevin Rue-Albrecht (15:04:03): > damn I can’t figure how to pass it directly in the<mark>tag. All they show is: > > <style> > mark { > background-color: yellow; > color: black; > } > </style> > <mark>blah</mark> >
Kevin Rue-Albrecht (15:05:23): > alright gotta go, i’ll pick this up later
Aaron Lun (15:07:17): > I think the blue colour is less intrusive.
Aaron Lun (15:07:25): > Anyway, wiki guidelines are up.
Kevin Rue-Albrecht (16:00:56): > Cool, i’ll revert to dark blue. It’s good enough for ‘lexical’ items
Aaron Lun (16:08:09): > Good. I’m going home now.
Aaron Lun (16:08:44): > Nag the other two to update their tours
Kevin Rue-Albrecht (16:09:13): > on a separate note, there’s “only” 4 functions left to unit test in iSEE-extras.R.
Aaron Lun (16:09:38): > heatmaps better be one of them.
Aaron Lun (16:09:51): > Okay, I really want to go home
Kevin Rue-Albrecht (17:11:38): > I hate to leave it in this state, but writing a seemingly superfluous test for.sanitize_SE_inputspotted a bug (I think), whereby in my case the last nested column is not added correctly. > Can someone check outquotes2?test_iSEE-extras.R?
Kevin Rue-Albrecht (17:11:54): > The relevant code is aroundhttps://github.com/csoneson/iSEE/blob/quotes2/R/iSEE-extras.R#L780
Kevin Rue-Albrecht (17:13:19): > I’m not sure exactly why the evaluation of commands is done in the loop, in the code. For the unit test, I naivelyeval-ed the list of commands, which produced the result that I would expect
Federico Marini (19:00:34) (in thread): > Fine for the coding
Federico Marini (19:01:26) (in thread): > shall we pick the warning color for say “solarized”? if there is one..
Federico Marini (19:01:43) (in thread): > something like that users might already see for warnings normally
2018-05-15
Federico Marini (03:28:01): > I can’t update my parts at least these 2 days- can some of you take over that for me?
Federico Marini (03:28:44): > To motivate you:
Federico Marini (03:29:14): > @Federico Mariniuploaded a file:Photo on 15.05.18 at 09.28.jpg - File (JPEG): Photo on 15.05.18 at 09.28.jpg
Kevin Rue-Albrecht (04:03:36): > small bad news: I got some impossible deadlined just dumped on me, and I don’t want iSEE blamed for me not meeting said deadlines.@Aaron Luncan you have a look at the unit test for nested DataFrames on branchquotes2intest_iSEE-extras.R? Except if I messed up the unit test itself, I think the code fails to extract a field, in my scenario.
Charlotte Soneson (04:44:01): > I updated my tour. If I get it right the one that’s still left to do is theintro_firststeps.txt. I can take a look at it now.
Aaron Lun (05:02:29): > Thanks C
Federico Marini (05:11:00): > @Aaron Lunstill missing your address if you fancy a postcard:slightly_smiling_face:
Aaron Lun (05:11:09): > No, I’ll pass thx
Federico Marini (05:11:59): > k:wink:
Charlotte Soneson (05:28:44): > Ok, standardizedfirststepstour currently onstandardizetourbranch
Aaron Lun (05:30:04): > BTW we don’t need the disclaimer for the firststeps tour.
Charlotte Soneson (05:30:25): > Well, maybe not
Aaron Lun (05:30:27): > As people generally aren’t instantiating it upon session flush
Kevin Rue-Albrecht (05:31:27): > I agree with Aaron on this one
Charlotte Soneson (05:32:24): > removed
Federico Marini (05:36:29): > -> the disclaimer is indeed there in iSEE2018 ->https://github.com/LTLA/iSEE2018/blob/master/tours/allen_app.R - Attachment (GitHub): LTLA/iSEE2018 > iSEE2018 - Code and manuscript files for the iSEE paper
Aaron Lun (05:39:11): > Yes, that’s fine
Kevin Rue-Albrecht (05:39:15): > Yeah, but in that case it makes sense, as the tour is launched immediately. That is not the case when running the example in the package
Federico Marini (05:40:18): > yup, therefore the edit inline
Charlotte Soneson (06:08:33): > I started looking at the heatmap issue when all values are identical (#192), a first fix is onheatfixbranch. The error appears only for centered data, since we are forcingggplot2to interpolate the three provided values, but there are no values to interpolate between. Thus, everything works fine if you set the upper and lower limits explicitly. Also for the non-centered case we don’t get the error, but rather a message urging the user to select lower and upper limits that are lower and higher than the value in the matrix, respectively. In the fix, I set the color to the middle value of the provided color scale in the problematic case, but I guess another option would be to give the same message as for the non-centered case and force people to set some limits.
Aaron Lun (06:10:48): > The fix sounds sensible.
Aaron Lun (06:11:25): > In this case it doesn’t matter as heatmaps do not transmit, but I want to avoid givingvalidatemessages for any potentially transmitting plot.
Charlotte Soneson (06:13:53): > Ok, then I’ll merge it in together with the standardizedfirststepstour
Aaron Lun (08:04:01): > Fix is confirmed, issue can be closed for the time being. However, I suggest we put in formal heatmap tests sooner rather than latter, where we can just progressively add tests for bugs as we encounter them.
Kevin Rue-Albrecht (08:29:33): > Thanks. Well done and sorry I left that on you. I’m struggling to keep up over here these days. Doesn’t help me focus on anything the way I’d like
Aaron Lun (10:25:08): > GIVE ME YOUR ORCIDS
Kevin Rue-Albrecht (10:25:25): > 0000-0003-3899-3872:wilted_flower:
Aaron Lun (10:26:11): > lol didn’t know you had an accent in the e
Kevin Rue-Albrecht (10:26:23): > had to have a French touch
Aaron Lun (10:27:38): > too many@Federico Marinion orcid
Kevin Rue-Albrecht (10:29:45): > Can’t find it in profile page…https://www.unimedizin-mainz.de/imbei/biostatistics-bioinformatics/the-team/federico-marini.html?L=1:wink:
Aaron Lun (10:43:02): > He is a non-person
Charlotte Soneson (11:40:39): > 0000-0003-3833-2169
Aaron Lun (12:46:23): > Wow, look at this “Low Bias Local Intrinsic Dimension Estimation from Expected Simplex Skewness”
Aaron Lun (12:46:30): > I understand each of those words only in isolation.
Kevin Rue-Albrecht (12:46:50): > ah. i lose then. Didn’t get Simplex
Kevin Rue-Albrecht (12:47:09): > (thanks Wikipedia)
Aaron Lun (12:47:12): > IT’s a virus, obviously.
Aaron Lun (12:47:24): > herpes FTW
Kevin Rue-Albrecht (12:48:25): > LBLIDE(f)ESS
Aaron Lun (12:48:28): > Also used in linear programming
Kevin Rue-Albrecht (12:49:32): > It’s probably good to see that for once, they probably focused more on the algorithm than the acronym ^^
Charlotte Soneson (12:49:55): > Are you planning to do dimensionality estimations?:slightly_smiling_face:
Charlotte Soneson (12:50:49): > Or are you just going through my ORCID record for fun?:wink:
Kevin Rue-Albrecht (12:51:06): > (when I say ‘they’, I mean ‘of all people’, btw)
Aaron Lun (12:51:08): > Yeah, I was thinking that I needed more simplexes in my life.
Aaron Lun (12:51:12): > NOT HERPES
Federico Marini (19:35:21) (in thread): > It might be the palinka speaking, but the acronym makes totally sense
Federico Marini (19:37:26): > my orcid:0000-0003-3252-7758
Federico Marini (19:42:18): > I got to talk with Barbara Borges (RStudio, many things rotating around shiny) tonight on the boat trip
Federico Marini (19:42:31): > (clever thing: she could not run away:slightly_smiling_face:)
Federico Marini (19:43:11): > but still, good thing, she also sees the sense of having the freehand selection for brushng
Federico Marini (19:43:50): > and she was pretty impressed of the pbmc4k demo which btw worked awesomely on the cellphone with tip tip tip lasso selections:smile:
Federico Marini (19:44:32): > I’ll sit up with her and a laptop tomorrow and try to pick up whether some upcoming news abt shiny might be relevant for us
2018-05-16
Aaron Lun (04:02:34): > Sounds good
Federico Marini (04:19:16): > small “bug” report when copy-pasting the code of the panel settings:
Federico Marini (04:19:36): > I copy the code, save it, but some panel types are not present
Aaron Lun (04:19:51): > You’re going to have to be more specific
Federico Marini (04:20:17): > yep, still typing:stuck_out_tongue:
Federico Marini (04:21:12): > > rowDataPlotArgs <- new('DataFrame', nrows=0L, rownames=paste0('rowDataPlot', seq_len(0))) > Error in validObject(.Object) : > invalid class "DataFrame" object: number of row names and number of rows differ > > rowDataPlotArgs[['YAxis']] <- character(0) > Error in rowDataPlotArgs[["YAxis"]] <- character(0) : > object 'rowDataPlotArgs' not found >
Aaron Lun (04:22:13): > should be sprintf instead of paste0
Federico Marini (04:23:14): > well spotted
Federico Marini (04:24:04): > I mean, in the end this does not lead to any mistake, but it is just otherwise unaesthetic to get the user to be pasting error-triggering code
Kevin Rue-Albrecht (04:31:51): > well if it’ssprintf, then there should be a placeholder%i
Kevin Rue-Albrecht (04:32:27): > sprintf('rowDataPlot%i', seq_len(1))andsprintf('rowDataPlot%i', seq_len(0))
Aaron Lun (04:32:48): > I’m sure you guys can sort it out.
Federico Marini (05:09:06): > FYI: Barbara’S live now
Federico Marini (05:09:07): > https://www.youtube.com/watch?v=rlt0GRara5g - Attachment (YouTube): eRum 2018 Wednesday - nagyHall
Kevin Rue-Albrecht (05:12:14): > oops, i just managed to join for the moment where apparently it crashed:sweat_smile:
Federico Marini (05:16:32): > yup - some pkg version conflict apparently:stuck_out_tongue:
Federico Marini (08:12:27): > -> we should just open up the issue on the shiny github repo
Federico Marini (08:12:38): > (and all +1 it :D)
Federico Marini (08:13:06): > but it is a totally reasonable and useful feat request, “that’s what she said”:wink:
Aaron Lun (08:41:29): > Did someone fix the bug, or what?
Federico Marini (08:43:17): > not yet - at least I could not, with the wifi somehow blocking push to github
Aaron Lun (14:21:59): > From Vicky: > > Just a quick update from dev- they’ve said we need your shiny app to be https as our pages are loaded over https and when we try to load this content as http it is getting blocked by the browser security. Is this possible for you to sort out? We’ll need this to do the iframe for definite and still looking into working in an expanded version of it for now.
Aaron Lun (14:30:10): > CRUK’s server is not served over https. I could ask, but it’s not going to happen quickly.
Aaron Lun (14:30:18): > Could someone explore the possibility of running this onshinyapps.io?
Aaron Lun (14:30:31): > We’ll put up with the monthly limit for the time being.
Aaron Lun (15:03:37): > @Kevin Rue-Albrecht@Federico Marini@Charlotte SonesonI find your collective lack of slackness disturbing. Where are you guys?
Aaron Lun (15:11:42): > Well, fine. I’m going home.
Federico Marini (16:48:09): > I was on the:airplane:. If I had been slacking there, it could have been a bad sign:smile:
Aaron Lun (16:49:41): > Lack o’ slack
Aaron Lun (16:49:47): > or slack slack
Federico Marini (16:53:22): > anyway, folks. I got some nice nice words from the audience- despite the not-so-overrepresentation of bioinfo
Federico Marini (16:53:35): > so in case you felt your ears whistling
Federico Marini (16:53:40): > :stuck_out_tongue:
Aaron Lun (17:35:10): > If you’ve some free time over there, can you let me know if iSEE works onshinyapps.io?
2018-05-17
Kevin Rue-Albrecht (03:10:05): > Sorry that I disappeared yesterday. Tons of stuff to debug here, plus I wasn’t feeling well. > Anyway, I’m curious to hear if anyone has a paidshinyapps.ioaccount, because we established early on that free accounts don’t have enough memory allowance, even for the relatively small allen data set
Aaron Lun (04:28:49): > Oh - really?
Aaron Lun (04:29:04): > Fuck
Aaron Lun (04:29:09): > how much memory do they allow, then?
Aaron Lun (04:29:11): > Must be nothing
Kevin Rue-Albrecht (04:30:41): > At the time, I uploaded it there, and it kept crashing when I was accessing theshinyapps.ioapp page. I somewhat remember Federico checking and confirming that on his machine the app was using more memory than available for free accounts onshinyapps.io
Kevin Rue-Albrecht (04:32:26): > If you have the patience, I also set up a Docker (https://hub.docker.com/r/kevinrue/isee_allen/), and using thedocker statscommand, I could monitor the container to use slightly over 1GB
Aaron Lun (04:33:09): > Gaah
Aaron Lun (04:34:48): > Well, okay, let me ask my peeps whether it is possible to get HTTPS for our shiny server.
Kevin Rue-Albrecht (04:49:18): > I’ve just pushed a fresh isee_allen image to Docker. Using version 1.1.0 of iSEE. If you want to try. However, I’m still doing the builds manually, and I can’t deal with automated builds right now.
Aaron Lun (05:06:35): > Where is the feature information for the app memory?
Aaron Lun (06:25:04): > Who the hell put ExperimentHub in our DESCRIPTION?
Aaron Lun (06:25:13): > shinyapp’s rsconnect is choking on it.
Kevin Rue-Albrecht (06:39:07): > I put ExperimentHub in our DESCRIPTION, when we started using the TCGA data. I think R CMD check complains otherwise. Still, packages under that header can be skipped during installation, withinstall.packages(dependencies=...), no?
Aaron Lun (06:39:37): > Why would R CMD CHECK complain about ExperimentHub if the TCGA data isn’t even in the package?
Kevin Rue-Albrecht (06:40:16): > well, we refer to some ExperimentHub commands in the vignette
Aaron Lun (06:40:38): > Seems unrelated to me.
Aaron Lun (06:40:47): > How we get a SummarizedExperiment object is irrelevant
Aaron Lun (06:41:17): > I mean, we could throw inEnhances: scran, csaw, diffHicif we want to go down that road.
Kevin Rue-Albrecht (06:42:04): > well, anyway, my question is rather: how is rsconnect chocking on it?
Aaron Lun (06:42:32): > dunno.
Aaron Lun (06:42:51): > Don’t care either, I gave up.
Kevin Rue-Albrecht (06:43:57): > Because installing the package shouldn’t be a problem. But obviously, the TCGA tour as implemented on iSEE2018 would attempt to create a cache and even download the data, the first time.
Aaron Lun (06:44:20): > Well, it shouldn’t do that anymore as we just give it the SCE as an RDS.
Kevin Rue-Albrecht (06:44:37): > oh right. that;s addressed then
Aaron Lun (06:44:38): > iSEE and ExperimentHub are more-or-less unrelated.
Aaron Lun (06:44:53): > I don’t think that iSEE “enhances” ExperimentHub any more than it enhances anything else.
Kevin Rue-Albrecht (06:45:47): > I guess so.
Kevin Rue-Albrecht (06:46:07): > I suppose you’ve read the “writing R extensions” bit about Enhances? > “Finally, the ‘Enhances’ field lists packages “enhanced” by the package at hand, e.g., by providing methods for classes from these packages, or ways to handle objects from these packages (so several packages have ‘Enhances: chron’ because they can handle datetime objects from chron even though they prefer R’s native datetime functions). Version requirements can be specified, but are currently not used. Such packages cannot be required to check the package: any tests which use them must be conditional on the presence of the package. (If your tests use e.g. a dataset from another package it should be in ‘Suggests’ and not ‘Enhances’.)”
Aaron Lun (06:46:37): > Yes.
Kevin Rue-Albrecht (06:46:42): > It was the “handle objects from these packages” that got me thinking
Aaron Lun (06:46:49): > Well, it can handle objects from anything.
Aaron Lun (06:47:04): > Surely you don’t intend to put all Bioc packages that generate SE objects?
Aaron Lun (06:47:24): > The only thing we’d really enhance is SE itself, and that’s already in Depends.
Kevin Rue-Albrecht (06:47:45): > Fair enough. Tbh, I don’t have a problem seeing this Enhances dropped
Aaron Lun (09:44:31): > Who has BioC push access again?@Federico Marini? Push the latest GH to BioC-devel, please.
Kevin Rue-Albrecht (09:44:40): > I do
Aaron Lun (09:44:44): > Oh.
Aaron Lun (09:44:55): > Okay, good. Wait for the latest PR…
Aaron Lun (09:45:55): > In the meantime, I told them to just proceed with the proofs with static links.
Kevin Rue-Albrecht (09:46:33): > Martin sent me the email, as I opened the issue on bioc/contributions. I’m genuinely curious to know what went through his mind when I submitted Charlotte’s repo that stated Fede as maintainer ^^
Aaron Lun (09:50:03): > lol
Aaron Lun (09:50:17): > Well, everyone gets a bite.
Aaron Lun (10:05:41): > DO IT
Kevin Rue-Albrecht (10:05:57): > ok
Kevin Rue-Albrecht (10:08:51): > done
Aaron Lun (10:13:01): > +1
Aaron Lun (10:13:07): > :+1:
2018-05-18
Aaron Lun (09:02:16): > Jesus: > > Unfortunately we cannot commit funds from the COAF block grant until after the article has passed through the open peer-review stage and been approved. This is consistent with the COAF Open Access policy which applies to peer-reviewed research articles. I’m sorry we cannot help at this stage.
Kevin Rue-Albrecht (09:02:46): > Does it sound that bad?
Kevin Rue-Albrecht (09:03:30): > I mean they’re not saying no. They’re just being bureaucratic
Charlotte Soneson (09:04:22): > Does this mean that they will do it once it is indexed? Or that they will decide then?
Kevin Rue-Albrecht (09:04:48): > I think the latter. Which obviously isn’t great as it leaves us in doubt til then
Aaron Lun (09:05:31): > Well, basically I need to get John to cough up the money
Kevin Rue-Albrecht (09:06:05): > I can check again, but I think the Oxford fund had the same statement on their website. Something like “contact us after the article is accepted”
Aaron Lun (09:06:16): > Technically it was “accepted”.
Kevin Rue-Albrecht (09:08:22): > Well, I don’t want to get in a game of semantics with you or the COAF people, but they do ‘imply’ (fairly explicitly) that accepted means approved by peer reviewers. F1000 has a somewhat different submission model. So far, we’ve only been accepted by the journal
Aaron Lun (09:08:58): > The confusing part is that F1000 is on the list of COAF-compliant publishers.
Kevin Rue-Albrecht (09:09:10): > True.
Aaron Lun (09:09:16): > So, I mean, if they’re on the list, can’t we use COAF money to pay for it?
Aaron Lun (09:09:26): > That’s the implication
Kevin Rue-Albrecht (09:09:41): > Still,http://openaccess.ox.ac.uk/wellcome-and-coaf/is pretty clear about “Before submission:” and “When article accepted:” - Attachment (openaccess.ox.ac.uk): Wellcome Trust and Charity Open Access Fund (COAF) > Current Policy What do I do at Oxford? Oxford has received a Wellcome/COAF block grant for 1 October 2017 to 30 September 2018 to
Aaron Lun (09:10:23): > Yeah, so was CRUK. So I followed the letter of the law there.
Aaron Lun (09:10:31): > Oh well.
Kevin Rue-Albrecht (09:10:41): > before being “accepted” (i.e. having the reviewer ticks, in F1000 verbiage), it’s only about “check this and that”
Kevin Rue-Albrecht (09:10:56): > “Please contact …” only comes after acceptance
Aaron Lun (09:11:15): > Again, letter of the law
Kevin Rue-Albrecht (09:11:39): > I’m gonna have to Google that
Kevin Rue-Albrecht (09:12:08): > ah. got it.
Aaron Lun (09:28:43): > In any case, I’ll beg and borrow some money from John.
Aaron Lun (09:28:46): > Get COAF to pay him back later.
Aaron Lun (09:28:55): > Well, it’s not like they’re asking for payment now anyway.
2018-05-20
Kevin Rue-Albrecht (05:28:28): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-20 10.27.57.png - File (PNG): Screenshot 2018-05-20 10.27.57.png
Aaron Lun (05:39:03): > what the hell is that
Kevin Rue-Albrecht (05:39:18): > the screenshot or what i did?
Kevin Rue-Albrecht (05:40:06): > screenshot: git history view in RStudio (restricted to the release branch)
Kevin Rue-Albrecht (05:40:40): > what I did: push the state of the upstream (Bioc) release branch to origin (csoneson GH)
Kevin Rue-Albrecht (05:41:08): > in the kevin remote (my GH) I was toying with docker, as Davis did a few months back
Kevin Rue-Albrecht (05:41:33): > gotta go, catch up later
Aaron Lun (05:41:43): > lol
2018-05-21
Jayaram Kancherla (10:22:05) (in thread): > hi@Aaron Lun, We now have a couple of vignettes integrating epiviz components with shiny. > > 1. General shiny integration -https://github.com/epiviz/epivizrChart/blob/shiny-provider/vignettes/IntegrationWithShiny.Rmd2. Using a SummarizedExperiment objecthttps://github.com/epiviz/epivizrChart/blob/shiny-provider/vignettes/VisualizeSumExp.RmdIf you have a test dataset (RangeSummarizedExperiment) that you would like us to use, let me know and i can test to see if there are any issues. - Attachment (GitHub): epiviz/epivizrChart > epivizrChart - epiviz genomic data visualization components to RMarkdown documents - Attachment (GitHub): epiviz/epivizrChart > epivizrChart - epiviz genomic data visualization components to RMarkdown documents
Aaron Lun (14:06:02) (in thread): > woah sweet
Aaron Lun (14:16:36) (in thread): > I’m overseas right now, but I’ll see if one of the others can play around with this.@Charlotte Soneson@Kevin Rue-Albrecht@Federico Marini?
Aaron Lun (14:17:06): > On an unrelated note, the space invader emoji is different in Tel Aviv.:exploding_head:
Kevin Rue-Albrecht (16:34:11) (in thread): > I’ll have a look ASAP. Getting rid of some 4-year old whole genome bisulfite sequencing right now
Aaron Lun (16:34:50) (in thread): > Sounds… as painful as it sounds.
Kevin Rue-Albrecht (16:34:50) (in thread): > this data set has imprinted the life out of me
2018-05-22
Aaron Lun (08:22:17): > Payment has gone through, we should be getting the proofs.
Kevin Rue-Albrecht (08:22:56): > Wow, I’m probably just not used to F1000, but… that was all quick!
Aaron Lun (08:23:57): > only 210 GBP as well.
Aaron Lun (08:24:02): > 10 times cheaper than plos.
Charlotte Soneson (08:24:11): > Did you get the 50% off for reviewing?
Aaron Lun (08:24:14): > yes
Charlotte Soneson (08:24:25): > Ok, cool.
Kevin Rue-Albrecht (08:24:54): > PS: I’ve just opened the vignette of epivizrChart last night, it’s:exploding_head:that the vignette includes the interactive plots too. Although I have to say that it kept freezing my RStudio session. Probably best to open the vignette in a web-browser
Jayaram Kancherla (08:42:47): > There a couple of plots with lots of data and rstudio’s inbuilt browser doesn’t handle it that well. chrome/firefox should be able to do it.
Aaron Lun (08:45:04): > Kevin, do you think it will be plug-and-play in a new panel type?
Kevin Rue-Albrecht (08:49:59): > Not sure. Haven’t dug enough yet
Jayaram Kancherla (09:00:51): > @Kevin Rue-Albrechtcheckout the vignettes for specific to shiny instead of the intro vignette (Integration with Shiny & SummarizedExperiment)
Kevin Rue-Albrecht (09:06:35): > Yep. Will do. Unfortunately, this needs to wait evenings and weekends these days. Countless pipelines to debug for the lab during the day -_-
Federico Marini (11:14:14) (in thread): > What does that look like?
Aaron Lun (14:08:36) (in thread): > A space invader from the game. WHereas in the UK it looked like a squid-thing.
Federico Marini (16:13:49): > Looking forward… Do you see something we can borrow?
Federico Marini (16:13:50): > https://github.com/dreamRs/shinyWidgets - Attachment (GitHub): dreamRs/shinyWidgets > shinyWidgets : Extend widgets available in shiny
Aaron Lun (16:23:25): > Nothing other than aesthetics-wise.
Federico Marini (17:33:12): > more on the substance side: are we good with the upcoming release of ggplot2 in mid june?
Federico Marini (17:33:29): > there might be some changes ahead which might break code
2018-05-23
Aaron Lun (02:46:41): > Um. Like what?
Federico Marini (03:18:07): > I read the announcement of the 2.3 version coming - headline and the invitation of pkg devs depending on ggplot2 to check in advance
Federico Marini (03:18:15): > but did not go deeper yet
Federico Marini (03:18:50): > we don’t do that extreme things - but still
Federico Marini (03:19:10): > I can try installing the dev version and see if all builds and checks as it should
Aaron Lun (11:45:43): > Worth giving it a go before everything shits itself, but we only use vanilla ggplot really. So we should be fine.
Charlotte Soneson (12:12:47): > One thought after preparing an iSEE session for some immunologists: would it be worth having a “select by colData” in the selection parameters - to let the users subset by a colData column instead of receiving a selection from another panel. In this case, they don’t really need the other panel, they just want to split the reduced dim plot by an annotation (facetting would be the ideal solution of course…)
Aaron Lun (12:25:32): > Not sure that would play nice with the existing observer infrastructure, as it would be a method of selection that doesn’t involve a brush or lasso somewhere. Perhaps you could give faceting a crack instead?
Charlotte Soneson (12:31:46): > Hm, ok
Aaron Lun (12:32:59): > I mean, it sounds like faceting is what theyreallywant here.
Charlotte Soneson (12:33:15): > Yes, definitely
Kevin Rue-Albrecht (12:33:38): > I can see how faceting would indirectly address the feature request (one facet would have the desired subset). However, I’d still advocate the use of a colDataPlot as this does exactly the subsetting in a large plot (i.e. not a potentially tiny facet)
Charlotte Soneson (12:34:51): > Actually in this case they want all facets. But I can imagine that sometimes you want only some of them, and you are right that the panels might be small (here there are only three groups I think)
Kevin Rue-Albrecht (12:35:21): > ah ok, so “split by colData” rather than “subset by colData”, you meant?
Charlotte Soneson (12:35:50): > Well, I meant subset, and then generate three plots, instead of having to have 6 panels.
Kevin Rue-Albrecht (12:36:01): > initially I thought they wanted a sort of dropdown that would allow them to select a subset of levels from a colData field
Aaron Lun (12:36:23): > Okay. For the feature request, faceting is the solution.
Aaron Lun (12:36:28): > If and when we get to it.
Aaron Lun (12:37:07): > More generally, it would be… challenging… to integrate non-brush/lasso options in “Selection parameters”.
Charlotte Soneson (12:37:15): > Yes, in this situation it would be preferable since this would allow coloring all facets at once
Aaron Lun (12:37:23): > I’m trying to think of exactly how challenging it would be… Probably a little, as the graphs wouldn’t make any sense.
Kevin Rue-Albrecht (12:37:42): > ok well, faceting should be (relatively) simple to put in place (there’ll be some design choices between _wrap and _grid), but indeed it’s the selections that scare me in that case
Kevin Rue-Albrecht (12:38:38): > I somewhat remember something about ‘domains’ in the shiny brush object, to store in which facet the brush was drawn
Aaron Lun (12:39:21): > Yes.
Aaron Lun (12:39:24): > It should be okay.
Aaron Lun (12:39:32): > We just have to update some of the functions that use brushes
Kevin Rue-Albrecht (12:39:34): > but on the plus sign I definitely remember that brushes can’t span multiple facets, so there would ever only be one ‘domain’ at any time
Aaron Lun (12:39:36): > e.g.identical_brushes
Federico Marini (15:21:04): > talking about feature request, i opened up thishttps://github.com/rstudio/shiny/issues/2072 - Attachment (GitHub): Feature request: freehand/lasso selection for brushing · Issue #2072 · rstudio/shiny > During a conversation at eRum2018 with @bborgesr, I managed to show a use case (out of potentially many) for this feature, which we currently (with some js flavour) implemented in our package, iSE…
Federico Marini (15:21:25): > if you fancy that, do drop a +1 reaction:wink:
Aaron Lun (15:49:13): > Done from me
2018-05-24
Kevin Rue-Albrecht (04:20:55): > I just realised we were featured herehttp://lazappi.id.au/2018/05/bioconductor-3-7-wrap-up/ - Attachment (lazappi): Bioconductor 3.7 wrap-up > The Bioconductor 3.7 release was announced this week. I thought I would have a look through the new packages and changes to existing packages and point out some of my highlights. The descriptions below are my summaries, if you want to see more detail you can read the full release notes here. Single-cell RNA-seq My interest is in single-cell RNA-seq analysis, so I am going to start off with packages related to this.
Kevin Rue-Albrecht (04:28:12): > Btw, how does it work with the Bioconductor gateway of F1000 ? Did our submission already include this, or is it something that can be updated ?
Federico Marini (07:37:35): > I assume it is a checkbox one can click, and my 2 cents is that Aaron did it:smile:
Aaron Lun (10:34:36): > Yes.
Aaron Lun (10:35:20): > Looks like we got a decent response from the shiny peeps
Federico Marini (10:36:12): > To me it sounded like a “it is definitely something useful, we’ll (somewhen unspecified) look into it and makes sense it is us to work on it”
Federico Marini (10:36:29): > which is pretty much the peachiest scenario I could think of
Aaron Lun (10:36:39): > Yeah, can’t hope for much more.
Aaron Lun (10:37:23): > If we have the closed lasso coordinates, we can just plot it manually on top as before.
Federico Marini (10:37:49): > yups. let’s wait’n’see
Federico Marini (10:38:11): > At least I got a good slot in catching up with Barbara at eRum
Federico Marini (10:38:23): > and she was genuinely stunned by the app:slightly_smiling_face:
Aaron Lun (10:38:49): > on the minus side, is your shiny server down?
Aaron Lun (10:38:53): > I can’t connect to it in Israel.
Federico Marini (10:39:09): > especially stunned because she said that to her the whole dashboard-thing did not look as nice as she wanted - yet people use it and use it
Federico Marini (10:39:13): > server seems to be up
Aaron Lun (10:39:17): > huh
Federico Marini (10:39:23): > blame it on Israel:smile:
Aaron Lun (10:39:28): > tried to show off the PBMC dataset, had to use a local version on my machine instead.
Aaron Lun (10:39:43): > Well in any case, the CI server will have HTTPS so we will be moving to that instead.
Federico Marini (10:40:07): > being CI cancer institute?
Aaron Lun (10:40:13): > yeah
Aaron Lun (10:40:21): > should be pretty beefy.
Federico Marini (10:40:27): > I guess so:slightly_smiling_face:
Aaron Lun (10:42:44): > The real question is, why do I always get taken aside by Israel’s airport security staff for enhanced screening?
Aaron Lun (10:43:25): > ’cause I’m smoking hot, that’s why. Probably set off an alarm.
Federico Marini (10:43:29): > As John would say, it’s the looks
Federico Marini (10:43:33): > :stuck_out_tongue:
Aaron Lun (10:43:55): > Was walking along a Tel Aviv beach next to some superfit ripped dudes
Aaron Lun (10:44:01): > And I was like, damn I fit right in.
Federico Marini (10:44:19): > I feel you bro
Federico Marini (10:44:43): > They also have their 6-pack
Federico Marini (10:44:58): > We have to compensate with the 6-packages
Aaron Lun (10:46:20): > There’s a small “park” with public gym equipment
Aaron Lun (10:46:24): > next to the beach
Aaron Lun (10:46:37): > and it’s full of these sweaty tanned ripped dudes
Aaron Lun (10:46:40): > and I’m walking past
Aaron Lun (10:46:49): > and I’m like, me, them, indistinguishable.
Aaron Lun (10:47:08): > Sort of like Where’s Wally (or Waldo, for the americans).
Federico Marini (10:47:58): > So the whole question is
Federico Marini (10:48:06): > why you at the security gates, and not them?
Aaron Lun (10:49:11): > A good point. Clearly the TSA have a keen eye for quality.
Aaron Lun (10:49:39): > Take the best, forget the rest.
Federico Marini (10:56:46): > Uh
Federico Marini (10:57:00): > I got a 7 min spot in an upcoming workshop
Aaron Lun (10:57:08): > Okay
Federico Marini (10:57:16): > a “pitch presentation” as they call it
Aaron Lun (10:57:19): > You going to talk about iSEE?
Federico Marini (10:57:39): > I’d touch on that in the lil’ time
Aaron Lun (10:57:46): > Cause I sold it pretty hard in the Weizmann.
Federico Marini (10:57:59): > Well done ripped-off boy:slightly_smiling_face:
Federico Marini (10:58:56): > audience will also include quite some companies, making a guess
Federico Marini (10:59:30): > since the keynotes are guys from Böhringer, one from TrOn (local translational oncology gGmbh), and one from Roche
Aaron Lun (11:29:16): > Gotta rake in the £££
Aaron Lun (11:29:18): > oops
Aaron Lun (11:29:21): > $$$
Federico Marini (11:29:28): > €€€
Federico Marini (11:29:32): > ¥¥¥
Aaron Lun (11:29:45): > Y’know I’ve never spent euros before
Aaron Lun (11:29:48): > EVER.
Aaron Lun (11:30:05): > I do, however, have 20 shekels in my wallet.
Aaron Lun (11:54:17): > https://support.bioconductor.org/p/109293/
Aaron Lun (11:54:27): > Yes, it’s called C++.
Kevin Rue-Albrecht (17:25:41): > First… brutal… prototype of faceting
Kevin Rue-Albrecht (17:26:03): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-24 22.25.18.png - File (PNG): Screenshot 2018-05-24 22.25.18.png
Kevin Rue-Albrecht (17:26:57): > seems to indicate some work required on the drawing of brushes by facet:
Kevin Rue-Albrecht (17:27:12): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-24 22.26.11.png - File (PNG): Screenshot 2018-05-24 22.26.11.png
Kevin Rue-Albrecht (17:50:45): > Note also that that the selection does not apply (“0%“) and by consequence does not transmit (i tried passing it to the colDataPlot, nothing lit up). I’m sure it was expected, but I’m just giving heads up.
Kevin Rue-Albrecht (17:51:22): > I’ll have to stop soon for tonight, but I’ll push it to a more stable place this weekend, and push to GH for you to look at
2018-05-25
Kevin Rue-Albrecht (06:25:26) (in thread): > Hi all, > I’ve quickly tried out the vignette, but for some reasondata(sumexp)fails.
Kevin Rue-Albrecht (06:26:07) (in thread): > I noticed I was not yet usinguseDevel, since the new release, so I’m in the process of updating now
Aaron Lun (07:58:58) (in thread): > Hm.
Federico Marini (08:08:53): > WARNING! WARNING!
Federico Marini (08:09:06): > ze concurrency has taken it to ze next level!
Federico Marini (08:09:08): > biorxiv.org/cgi/content/short/329102v1?rss=1
Aaron Lun (08:10:42): > lol
Charlotte Soneson (08:11:34): > > Users will need a gaming-class computer with a high-end graphics card (for example an NVIDIA GTX1080) running Windows 10, and an HTC Vive
Federico Marini (08:12:02): > zey vant to focus on a different user base
Aaron Lun (08:13:28): > No competition, I think.
Federico Marini (08:13:54): > ¯*(ツ)*/¯
Aaron Lun (08:14:07): > “Hey John, we need an Alienware machine.”
Federico Marini (08:14:09): > find me enough bioinformaticians with that setup
Federico Marini (08:14:17): > genau that:smile:
Aaron Lun (12:04:13): > @Federico Mariniis there any better way of keeping things in sync on the shiny server, compared to GH?
Aaron Lun (12:05:32): > especially as it seems I have to rename the files toapp.R?
Aaron Lun (12:08:08): > @Kevin Rue-Albrechtyourtcga_data.Rdoesn’t actually save the object to a file. And yourapp.Rdoesn’t actually read it from file.
Aaron Lun (12:09:16): > Same for@Charlotte Soneson.@Federico Marini, how on earth did you ever get these to run in the first place?
Aaron Lun (12:13:05): > Anyway, once you guys fix your tours, seehttps://marionilab.cruk.cam.ac.uk/iSEE_allen/for a HTTPS version. Get this done ASAP so I can tell the prod team.
Aaron Lun (14:06:05) (in thread): > In fact, yourdata.Rdoesn’t make much sense. What isexprs- log-counts or counts? Because you can’t compute the library size from the sum ofexprsif it’s log-counts, and you shouldn’t be running PCA on the raw counts.
Aaron Lun (14:20:35) (in thread): > Also, you still haven’t added the disclaimer to the start of your tour.
Aaron Lun (14:47:31) (in thread): > I’ve just done the disclaimer. But you should fix yourexprsthing. Suggest renaming them tocounts. Also, your log-CPM calculation is funny. Hard to see any upregulation of ERRB2 in breast cancers from the app.
Kevin Rue-Albrecht (14:48:31) (in thread): > I just got home. There was something about the disclaimer that you asked me to wait, last time I updated the TCGA tour. Anyway, thanks for doing it today
Aaron Lun (14:48:45) (in thread): > Hm, okay.
Aaron Lun (14:49:02) (in thread): > Most of the issues are fixed, but check the exprs.
Kevin Rue-Albrecht (14:49:31) (in thread): > .. and you also saved to file, I can see. I’ll try to understand what’s left to do
Aaron Lun (14:53:49) (in thread): > https://marionilab.cruk.cam.ac.uk/iSEE_tcga.
Kevin Rue-Albrecht (14:58:53) (in thread): > Nice. > The version I just loaded doesn’t have the disclaimer (that you added only 19 min ago, I’m aware)
Kevin Rue-Albrecht (15:00:10) (in thread): > Out of curiosity, to redeploy after an update, is it just about reinstalling the package on the server machine, or does something else need to be restarted?
Kevin Rue-Albrecht (15:00:30) (in thread): > (I haven’t had the admin power to manage a shiny server since my PhD)
Aaron Lun (15:00:31) (in thread): > Haven’t upploaded the new version to the server.
Aaron Lun (15:00:46) (in thread): > Nothing needs to be restarted, I think.
Aaron Lun (15:00:52) (in thread): > I’ve been updating things as I go along.
Kevin Rue-Albrecht (15:01:02) (in thread): > that’s neat
Charlotte Soneson (15:16:01) (in thread): > did you already fix this@Aaron Lun(if so, thanks)? I was in the middle of other GitHub fixes this evening.
Aaron Lun (15:16:32) (in thread): > More or less. Yourdata.Rcould do with some sprucing up - isn’t this an ExperimentHub package now?
Charlotte Soneson (15:17:06) (in thread): > Yes, it is. On my todo list for tomorrow. Also to fix the vignette in the same way.
Aaron Lun (15:47:39): > If someone is looking for things to do, I’ve starting cleaning up the language in the first steps tour. Stopped at the row data plot.
2018-05-26
Charlotte Soneson (01:33:00): > We have competition in the namespace:slightly_smiling_face:https://twitter.com/biorxivpreprint/status/1000243974236450816 - Attachment (twitter): Attachment > iSEE: Interface Structure, Evolution and Energy-based machine learning predictor of binding affinity changes upon mutations http://biorxiv.org/cgi/content/short/331280v1 #bioRxiv
Federico Marini (01:49:09): > get off of our yard!
Federico Marini (01:50:02) (in thread): > that was an alternative thought where we setup the isee2018 repo
Federico Marini (01:50:29) (in thread): > it was easier to maintain, but messy for humans to read/reviewers to check
Federico Marini (01:51:19) (in thread): > my suggestion is to keep a local copy of the repo on the server somewhere, and have a script update automatically pkg and app.R as well
Federico Marini (01:51:28) (in thread): > this for each dataset
Federico Marini (01:52:17) (in thread): > as for the configs on shiny
Federico Marini (01:52:33) (in thread): > I can send you the /srv/shiny-server config file
Federico Marini (01:52:55) (in thread): > I recall there is an important param to avoid crashes when an app takes long to load
Federico Marini (01:53:13) (in thread): > I am without my MBP now - charlotte knows:stuck_out_tongue:
Federico Marini (01:53:33) (in thread): > but I can recall it is somthing like app_init_timeout
Kevin Rue-Albrecht (06:31:14): > I hesitate between:rolling_on_the_floor_laughing:and:disappointed_relieved:I don’t know if you saw their GH repository (6 commits, even though they started before us, in October), but it’s definitely not an R ‘package’:https://github.com/haddocking/iSee - Attachment (GitHub): haddocking/iSee > Contribute to iSee development by creating an account on GitHub.
Aaron Lun (06:50:58): > Not a problem.
Aaron Lun (06:51:18): > We’re in an official repo first.
Aaron Lun (06:52:52): > Anyway, still cleaning up the intro tour - seefurtherfixesfor more details.
Kevin Rue-Albrecht (06:54:26): > I just got a doubt now, but can’t find an example anywhere: should irlba run the log2 counts like PCA, or the raw counts?
Kevin Rue-Albrecht (06:55:19): > The help page of irlba mentions sparse matrices: “A numeric real- or complex-valued matrix or real-valued sparse matrix.”
Aaron Lun (06:57:41): > It’s a PCA, so the log-counts.
Aaron Lun (06:57:49): > irlba just does an approximate PCA.
Aaron Lun (06:57:55): > Everything else is still the same.
Kevin Rue-Albrecht (06:58:59): > ok cool, i’ll fix that too then
Aaron Lun (09:08:47): > is this done?
Kevin Rue-Albrecht (09:47:41): > yeah - i just got a bit surprised by the updated feature expression plot after moving to edgeR cpm. I must say the distributions look a lot more like what I’m used too, but the difference between breast cancer and other cancers for ERBB2 looks less impressive. Anyway, Ready to merge, AFA I’m concerned
Kevin Rue-Albrecht (09:48:16): > I updated the text of the last tour step, to mirror the updated plot
Aaron Lun (09:53:18): > Less impressive? It already didn’t look that great from memory.
Kevin Rue-Albrecht (09:54:13): > True. It’s more like a proportion of breast cancer samples show the up-regulation.
Aaron Lun (10:26:58): > Can it not be made to look better? e.g. to focus on the relevant subset?
Aaron Lun (10:27:07): > Surely the original paper didn’t rest on this result.
Jayaram Kancherla (10:35:16) (in thread): > Hi Kevin, we haven’t pushed this to bioc-devel. Planning on doing it next week after you test it to see if we need to add/test more. This code is currently theshiny-providerbranch on github repo (https://github.com/epiviz/epivizrChart/tree/shiny-provider) - Attachment (GitHub): epiviz/epivizrChart > epivizrChart - epiviz genomic data visualization components to RMarkdown documents
Kevin Rue-Albrecht (11:56:34) (in thread): > Thanks. I spotted the branch, but didn’t get to checkout and build locally yet.
Kevin Rue-Albrecht (11:59:44) (in thread): > When you recently asked for an example RangedSummarizedExperiment, would one of our ‘tour datasets’ work for you? I don’t think any of them includes range data (yet). One could grab an Ensembl GTF and stash it in the rowData slot, but ideally we’d need a data set that has the original feature information.
Kevin Rue-Albrecht (12:02:29) (in thread): > I just checked and the pbmc4k tour has chromosome information for each feature@Aaron Lunwould it be possible to update the object with start-end position, and move it to the rowRanges rather than the mcols of the rowData ?
Aaron Lun (12:03:17) (in thread): > Well, yes, but I’d have to do it via annotation; it’s not present in the source data.
Kevin Rue-Albrecht (12:04:21) (in thread): > I imagine. I would have thought of doing the same for the TCGA data: I wouldn’t embark on sending a PR on the ExperimentHub object
Kevin Rue-Albrecht (12:06:03): > i haven’t been super stringent on my reading of the paper, but they do make a big point of the HER2/ERBB2 overexpression in breast cancer samples: 41 occurences of HER2 in the manuscripthttps://academic.oup.com/bioinformatics/article/31/22/3666/240143
Kevin Rue-Albrecht (12:08:48): > I wonder whether RPKM/TPM would make a huge difference in this case. Given that I use their counts, that’s the only difference I would see right now
Kevin Rue-Albrecht (12:09:28): > (another motivation for adding rowRanges to the TCGA tour, you’ll say ^^ )
Aaron Lun (12:12:48): > That shouldn’t make a damn difference, everything should cancel out in a within-gene comparison.
Kevin Rue-Albrecht (12:13:09): > duh. Saturday state of mind. Sorry
Kevin Rue-Albrecht (12:14:32): > In my defense, I’m cleaning up my first draft of a ‘faceting’ feature. Bit slow diving back in there after a few weeks head down in Oxford projects.
Aaron Lun (12:19:08): > You said the O-word.
Kevin Rue-Albrecht (12:19:13): > I drink?
Aaron Lun (12:19:46): > The O-word is banned on this channel!
Aaron Lun (12:20:10): > If you must refer to it, then you can say, “the other place”.
Kevin Rue-Albrecht (12:20:33): > I’ve been given the O-talk before, don’t worry:wink:
Aaron Lun (12:20:45): > Good.
Kevin Rue-Albrecht (12:20:57): > I just feel the need to rebel every now and then
Kevin Rue-Albrecht (13:04:10): > anyone feels like adding the UI to control the selection of colData/rowData covariates for faceting? (onfacetbranch)
Kevin Rue-Albrecht (13:05:00): > UI hasn’t been my prime field of expertise in iSEE so far.
Kevin Rue-Albrecht (13:14:45): > Basically, I’ve added two columns to theDataFrameof parameters forrowDataPlots andcolDataPlots, to store the row and column faceting covariates. So far I think I made the reasonable assumption that only colData and rowData covariates, respectively, can be used for faceting (notassay data using a feature name). > As usual, the covariates stored in the DataFrame are picked up by the plotting commands, to determine whether a faceting command is required, and if so, which one. > All that’s needed now are two selectize UI element (which collapsible box, to be defined) to control the row and column faceting covariates.
Kevin Rue-Albrecht (13:17:09): > Development notes: > - the selectize choices should include an additional"."item (first, preferably) that indicates no faceting required on that axis (X/Y) > - I would suggest to implement some smart filtering of “facet-able” covariates, which initially could be simply the result ofiSEE:::.is_groupable. We don’t want user to facet by a continuous variable
Aaron Lun (13:21:13): > It should go into visual parameters, surely?
Aaron Lun (13:21:49): > Because youcouldfacet an assay data plot. Why couldn’t you?
Kevin Rue-Albrecht (13:22:39): > what do you call an assay data plot? I meant facetingonan assay data for a particular feature
Kevin Rue-Albrecht (13:23:09): > as in “X axis = feature name”
Aaron Lun (13:26:48): > featAssayPlot
Aaron Lun (13:27:01): > I mean, you should be able to facet that.
Aaron Lun (13:27:08): > But no, not on the feature names.
Kevin Rue-Albrecht (13:27:56): > oh sorry, my bad, I also made that one facetable already, I didn’t word my previous message correctly
Kevin Rue-Albrecht (13:28:29): > it should say “the faceting is already implemented for both row-based and column-based plots”
Kevin Rue-Albrecht (13:28:40): > not just rowData and colData, sorry
Kevin Rue-Albrecht (13:28:47): > redDimPlot is handled too
Kevin Rue-Albrecht (13:29:24): > kudos on the.add_general_parameters_for_*_plotsfamily of functions btw, made my life so simple on this one
Aaron Lun (13:29:38): > Mm-hm.
Kevin Rue-Albrecht (13:30:17): > Can’t wait for the faceted heat map@Charlotte Soneson:stuck_out_tongue:
Charlotte Soneson (13:31:25): > Hmm…wonder how that would work
Kevin Rue-Albrecht (13:32:24): > Not sure how long it would take someone to set up the UI elements to control it dymanically, but a dirty way to test out the faceting is to change the defaults in.add_general_parameters_for_*_plots. I’ve left comments with those that I used (e.g.driver_1_sfor allen)
Aaron Lun (13:36:26): > I have to add a landing page for the Shiny server, so I won’t do this tonight.
Kevin Rue-Albrecht (13:38:19): > Ok no worries. I’ve been stuck on a few different things for Steve recently, so I’ll need to put in some effort in there this weekend. I can help fix up stuff for the faceting as we go along, but I confused myself reading through the dynamicUI code yesterday: I’d rather not make a mess of it, and leave it to some who’s more familiar with the coding pattern in there.
Aaron Lun (13:40:30): > Are we all done with the TCGA tour? Any progress on making the ERCBB2 prettier?
Kevin Rue-Albrecht (13:48:38): > no progress to report, no:sadface:
Aaron Lun (14:49:03): > You need to change the tour then. There is no way that I can see that ERBB2 is upregulated in BRCA mutants.
Kevin Rue-Albrecht (14:50:05): > Hm ok. Have you had a brief look at the paper (https://academic.oup.com/bioinformatics/article/31/22/3666/240143) if I missed anything there?
Aaron Lun (14:53:07): > They mention “control”.
Aaron Lun (14:56:05): > Are you actually comparing to the control?
Aaron Lun (14:56:09): > What the hell is the control?
Kevin Rue-Albrecht (14:56:19): > I don’t have the controls in this data set
Aaron Lun (14:56:54): > Perhaps we should get it, then.
Kevin Rue-Albrecht (14:57:56): > Yep ok. I think I spotted the Ehub record that also includes the controls after the whole tour was already in place, but at this point it’s worth having a look.
Aaron Lun (15:10:57): > @Charlotte Sonesonupdated cytof tour athttps://marionilab.cruk.cam.ac.uk/iSEE_cytof, check that everything works.
Aaron Lun (15:14:10): > I’ve had enough, am going home.@Kevin Rue-Albrechtlet me know when the TCGA is re-ready.
Charlotte Soneson (15:16:34) (in thread): > Looks good, thanks
Kevin Rue-Albrecht (15:43:53): > Will do. I was multitasking on other things. But I’ll get to it this weekend
Kevin Rue-Albrecht (17:03:19): > got it: the controls (aka ‘normal’) were stashed in a separate EHub record:EH1044 | RNA-Sequencing and clinical data for 741 normal samples from The Cancer Genome Atlas
Kevin Rue-Albrecht (17:10:21): > Although, I’m puzzled: > > > table(eh1044$type) > > BLCA BRCA CESC CHOL COAD ESCA GBM HNSC KICH KIRC KIRP LIHC LUAD LUSC PAAD PCPG PRAD READ SARC SKCM STAD THCA THYM UCEC > 19 113 3 9 41 13 5 44 25 72 32 50 59 51 4 3 52 10 2 1 37 59 2 35 >
Kevin Rue-Albrecht (17:47:17): > alright I’ve pushedtcga_cntlto iSEE2018, if someone could take a look atdata.R. I’m not convinced, and - if those controls indeed belong with the cancer samples - I don’t understand why the guys would have stored them as an ExpressionSet and a SummarizedExperiment, respectively. Also the controls (EH1044) have only two colData, which doesn’t even include gender. I had to clean up a couple of things (e.g. colData) before I could combine the two objects
Aaron Lun (19:10:08): > As in… you’re not convinced there’s differences in ERRB2 expression?
2018-05-27
Kevin Rue-Albrecht (04:45:11): > that, or I’m not convinced I’m doing this simplest thing right
Kevin Rue-Albrecht (04:45:36): > sticking two count matrices together, and comparing log2-CPMs
Kevin Rue-Albrecht (04:50:08): > what I mean is that the difference could well be significant, but looking at the two distributions of log-CPM doesn’t really impress me: the ‘control baseline’ looks fairly close to the BRCA group, for a gene that’s making the headline. Let me know what you think, if you can run the updated data.R on the tcga_cntl branch
Aaron Lun (06:22:40): > @Federico Marinithe tour didn’t point correctly to the dropdown elements - pressing “next” automatically cancelled the dropdown, and the next step of the tour just pointed to empty space - so I just removed those parts of the tour.
Aaron Lun (06:44:11) (in thread): > Yourdata.Rignores the fact that the controls are split up by tissue type.
Kevin Rue-Albrecht (06:47:00) (in thread): > Yup. Went a bit brute force I guess to create colData that have the same ncols for the controls as for the 7k cancer samples
Kevin Rue-Albrecht (06:48:11) (in thread): > I’ll just make a matrix of NA of the right dimensions, and turn it into a DataFrame later, instead of the ugly piece of code I pushed yesterday
Aaron Lun (06:48:32) (in thread): > There’s about a two-fold change for BRCA vs normal in ERBB2.
Kevin Rue-Albrecht (06:48:55) (in thread): > vs “BRCA-normal” you mean?
Kevin Rue-Albrecht (06:49:25) (in thread): > I’ll clean up my mess then (if you haven’t done so already?)
Aaron Lun (06:49:34) (in thread): > Yes, please do so.
Kevin Rue-Albrecht (06:49:50) (in thread): > Cool. Sorry for the confusion.
Aaron Lun (06:50:28) (in thread): > But in any case, I think you misunderstood the point of the paper. It wasn’t actually looking at the TCGA data for the ERBB2 upregulation, it was looking at the application of the TCGA Level 3 pipelines on their own dataset where they knew that ERBB2 was upregulated.
Aaron Lun (06:51:39) (in thread): > At no point do they really say that ERBB2 is upregulated in the TCGA BRCA samples - in fact, it seems like they’re trying to distinguish TCGA BRCA HER2-active samples from HER2-inactive samples, which means that ERBB2 isn’t actually up in all samples.
Aaron Lun (06:51:57) (in thread): > So perhaps that’s the wrong thing to look at.
Kevin Rue-Albrecht (06:54:09) (in thread): > I did get the point that it was a ‘reanalysis’ where they compared pipelines. I just didn’t go deep enough in the interpretation of their results, I must say. When I spotted 41 occurences of HER2 in the paper, I thought that was the ‘easiest’ thing to look at - but indeed not necessarily the ‘right’ one
Kevin Rue-Albrecht (06:57:27) (in thread): > Looking at the paper again, I guess it’d be nice to have a heat map of ‘HER pathway genes’, that’d go well with iSEE
Kevin Rue-Albrecht (08:45:17) (in thread): > did a screenshot show up anywhere in the channel or the thread? I saw it upload but can’t see it anywhere
Kevin Rue-Albrecht (08:45:57): > @Kevin Rue-Albrechtuploaded a file:HER2 in “BRCA vs. BRCA-cntl” - File (PNG): HER2 in “BRCA vs. BRCA-cntl”
Kevin Rue-Albrecht (08:47:14) (in thread): > @Aaron LunI’ve posted in the channel what I see when I use only the BRCA-cntl > I do take your point in this thread that the overall tour story would need to be revisited
Kevin Rue-Albrecht (08:49:07) (in thread): > Bluntly, I didn’t pick this TCGA data set because of a nice story. At the time, it was the largest data set that was readily available from the EHub
Aaron Lun (08:57:08) (in thread): > Possibly something could be said about the fact that there is a HER2+ subset.
Kevin Rue-Albrecht (09:10:20) (in thread): > indeed
Kevin Rue-Albrecht (10:13:02): > I’ve just pushed amore_discretebranch to GH, as the ‘extended’ TCGA data set exceeds the 24 levels currently enforced
Aaron Lun (10:20:29): > I don’t think that’s the problem. I think you should cut down what you’re showing in the demo.
Aaron Lun (10:20:56): > Can you imagine trying to fit 30 violin plots onto the screen?
Aaron Lun (10:21:01): > Let alone 30 colours
Kevin Rue-Albrecht (10:21:52): > I agree that for the tour, the problem is elsewhere, but Viper was already asking about that limit a few weeks back
Aaron Lun (10:22:21): > I should have a word with him then.
Kevin Rue-Albrecht (10:23:38): > I agree that 99% of the cases will work perfectly well with 24 levels (even fewer). However, I can imagine large screens and big projects like the TCGA that crosses the line even just by a couple of levels
Aaron Lun (10:24:24): > Well, rather than hard coding it, have a global option to be retrieved fromoptions(iSEE.maxlevels)or something.
Aaron Lun (10:24:31): > Or hell, don’t have a hard limit at all.
Kevin Rue-Albrecht (10:24:52): > tbh, I don’t think the actual value matters that much, continuous scales should immediately fly into several dozen values if not more
Aaron Lun (10:24:56): > If it’s a character or factor, then it’s discrete.
Kevin Rue-Albrecht (10:25:00): > true
Aaron Lun (10:25:22): > And if people can’t see the colours, that’s their own fault for putting in such a large number of factors.
Kevin Rue-Albrecht (10:25:35): > lol
Aaron Lun (10:49:39): > I don’t think we even need to have an upper limit. Just let the plots be made.
Kevin Rue-Albrecht (10:51:56): > mhh.. I haven’t done the mistake interactively in a long time, but I think ggplot can be fairly stupid when given a zillion character values (e.g. sample ids).
Kevin Rue-Albrecht (10:52:54): > i.e. taking ages to build a zillion discrete ticks with one data point each
Kevin Rue-Albrecht (10:53:26): > even smart users are not far away from a misclick that crashes a Shiny session
Kevin Rue-Albrecht (10:58:24): > Alright, as per GH, I think I’ve implemented the compromise: > - Infinite number of levels by default > - yet, expert users can regulate the limit through a session option
Kevin Rue-Albrecht (10:58:53): > Probably just needs documentation somewhere, if we agree on this implementation.
Aaron Lun (11:00:42): > Well, if the default is inf, then that doesn’t provide any protection to the type of user who would forget this.
Kevin Rue-Albrecht (11:02:30): > ok.. so maybe the compromise was actually to have the option, but with a limit set?
Kevin Rue-Albrecht (11:02:37): > like package that hasoptions(ucscChromosome=TRUE)by default or something, that users only learn about when their scripts crash
Kevin Rue-Albrecht (11:03:10): > ahh it’s Gviz, I remember know
Kevin Rue-Albrecht (11:04:54): > alright, so i put it back to 24 then
Aaron Lun (13:20:36): > Man, sitting in the office belting out ABBA hits
Aaron Lun (13:24:24): > In any case, this global options thing is a new feature and will not be in release, which is what the Shiny server is currently running on (and what people will be running on, if they want to reproduce the tour). So you’ll want to the TCGA tour to work within the current release constraints.
2018-05-28
Federico Marini (03:55:59) (in thread): > Yes, this behavior was known. I wanted to refer to the single elements, but it is ok to just keep the overarching element highlighted with the whole thing
Aaron Lun (06:57:16) (in thread): > Okay. Have a look at the the PR and merge it if the tour runs properly.
Kevin Rue-Albrecht (09:32:51) (in thread): > Sorry for the silence, I’ve been juggling a few things this weekend. I’m getting there. There are 22 coldata fields that deal with HER2 status in the Ehub object, and I’ve pulled the top 60 HER2 signature genes from their Suppl Table S4 (that’s not even cited in the main text - goes from S3 to S5..)
Kevin Rue-Albrecht (09:33:30) (in thread): > That adds a heat map that I am planning to annotated with HER2 immuno histochemistry status, as per their main text
Kevin Rue-Albrecht (11:17:43) (in thread): > Hm.. it’s not breathtaking, but I added the heat map generated from the reported HER2 signature genes.
Aaron Lun (13:34:14) (in thread): > You’re missing a full stop after the first Brahman citation.
Aaron Lun (13:39:49) (in thread): > And I don’t think you need to keep mentioning the full citation, just say Rahmanet al.in the following text.
Kevin Rue-Albrecht (13:40:19) (in thread): > yup, i wasn’t decided how to deal with that aspect, so I kept copy pasting
Aaron Lun (13:41:21) (in thread): > And I thought we decided to use colour for all references to UI elements. You’ve gotData parameters- why italics?
Kevin Rue-Albrecht (13:41:46) (in thread): > ah ok, I thought the color was only for panel names
Kevin Rue-Albrecht (13:42:10) (in thread): > I mean the ‘panel types’
Kevin Rue-Albrecht (13:42:23) (in thread): > but i can apply that to the collapsible ones too
Aaron Lun (13:43:02) (in thread): > yes, all UI elements, I think that’s what the wiki says.
Aaron Lun (13:43:16) (in thread): > Also missing a full stop at “Here, we note that a subset”…
Kevin Rue-Albrecht (13:43:48) (in thread): > thanks, found it
Aaron Lun (13:45:04) (in thread): > Also, the heatmap doesn’t look like it’s centered properly. Why are we getting all-purple or all-yellow rows?
Kevin Rue-Albrecht (13:46:15) (in thread): > Hm. Haven’t had a look into the heat map code much
Aaron Lun (13:46:35) (in thread): > Ah, I think I know why.
Aaron Lun (13:46:52) (in thread): > Because the centering is done globally, and this is subsetted upon selection to restriction.
Kevin Rue-Albrecht (13:47:18) (in thread): > ouch
Aaron Lun (13:47:29) (in thread): > anyway, your step 30 points to empty space.
Kevin Rue-Albrecht (13:48:24) (in thread): > really? i thought i fixed them all - it’s annoying that some UI elements can’t be pointed at (the selectize typically have issues for me)
Aaron Lun (13:48:31) (in thread): > Also, can you put up an issue on the iSEE GH saying that centering, scaling and clsutering should be done on the subsetted matrix?
Kevin Rue-Albrecht (13:49:57) (in thread): > can do
Kevin Rue-Albrecht (13:50:36) (in thread): > lemme just chase down the area of code where the issue is, to save time later for fixing it
Aaron Lun (13:53:05) (in thread): > Keep@Charlotte Sonesonin the loop about it as well. Should be a case ofheatmap.Rand moving the selection commands above the scaling/centering commands.
Aaron Lun (13:54:01) (in thread): > Though this is where it gets messy, because selection depends onplot.data, which is only defined after the centering/scaling.
Aaron Lun (13:56:04) (in thread): > The easiest way to do that is to create a “pretend”plot.datacontaining no columns but just rownames, for the function to use for storing"SelectBy".
Kevin Rue-Albrecht (13:58:06) (in thread): > Point 30 fixed.
Charlotte Soneson (14:39:26) (in thread): > We don’t have to center and scale before melting either, it can be done with dplyr after subsetting.
Aaron Lun (14:52:18) (in thread): > See GH PR. Feel free to edit; play around and merge.
Aaron Lun (14:53:47) (in thread): > We should probably start clearing out some of these PRs. I’ve assigned one to everyone else, so merge them once you’re satisfied.
Charlotte Soneson (14:53:48) (in thread): > Ok, you were faster…
Aaron Lun (14:54:28) (in thread): > Going home now to shave my head.
2018-05-29
Federico Marini (03:51:47): > on my side: tested/run on a release configured R - but the changes were only cosmetics in the tour
Federico Marini (03:52:07): > -> looks and reads nice, merging in soon my assigned PR
Aaron Lun (04:10:10): > Cool
Federico Marini (04:39:16): > I just saw we are failing on devel - on all machines
Aaron Lun (04:39:25): > Everyone is failing.
Federico Marini (04:39:45): > not just phylosophically, this time:smile:
Aaron Lun (04:39:51): > Woah, weird, space invader emoji is different again. EU/UK difference perhaps.
Federico Marini (04:40:20): > I see differences also between slack app VS slack via herokuapp
Federico Marini (04:40:25): > now you’Re violet
Aaron Lun (09:26:40): > Further on this : F1000Res can get the apps into iframes by the end of next week, apparently. So let’s look forward to that.
Aaron Lun (13:07:11): > In some unrelated news, I now have a mohawk.
Kevin Rue-Albrecht (13:40:44): > The heat map update seems to have broken the TCGA tour. No one else having trouble with heat map? I get:Warning: Error in : Insufficient values in manual scale. 6 needed but only 2 provided.
Kevin Rue-Albrecht (13:40:57): > Breaks the legend and the heat map
Aaron Lun (13:48:12): > Not sure why that would be
Kevin Rue-Albrecht (13:51:07): > sounds like a color map issue. I need to leave now, but I’ll try to have a close look as soon as i can
Aaron Lun (13:54:54): > probably a symptom rather than an actual issue with the colormap. Just throw a bunch of print statements inheatmap.Rto see what ggplot is trying to plot. Something funky with the restriction, I’ll bet.
Kevin Rue-Albrecht (14:35:39): > Must be somehow related to the fact that the factor has initially 6 levels, while only two of them were selected for the heat map
Aaron Lun (14:47:00): > Possibly… yes.@Charlotte Sonesonany ideas?
Kevin Rue-Albrecht (14:50:22): > indeed, for debug purpose, I removed the ‘Restrict’ link, and the heat map displays well. Plus, given that the numbers match up (6 levels subsetted to 2), it’s definitely the cause. I just need to pinpoint where the magic happens
Aaron Lun (14:50:45): > Probably some ggplot shit.
Kevin Rue-Albrecht (14:52:02): > ggplot: “making painful things simple, and easy simple things painful, since 20xx”
Aaron Lun (14:52:38): > amen to that
Kevin Rue-Albrecht (14:57:09): > Got it. +1 to@Charlotte Sonesonfor using my.nlevels()function. And -1 to Charlotte for using my.nlevels()function.:wink:
Kevin Rue-Albrecht (14:58:14): > I wrote that thing to return the number of unique values when the factor is stored as character, instead of returning NULL
Kevin Rue-Albrecht (14:59:05): > but if it’s applied before and after subsetting, the number of unique values won’t match
Kevin Rue-Albrecht (14:59:50): > I may not be able to fix that this evening, if anyone is impatient, I spotted it at line 154 of heatmap.R
Aaron Lun (15:04:31): > No kill stealing, so it’s all yours.
Kevin Rue-Albrecht (15:05:16): > do i sense a reference to ‘mid-lane’ scenarios?
Aaron Lun (15:06:46): > I’m more of a Halo guy
Kevin Rue-Albrecht (16:43:51): > right. haven’t played that one in a lifetime
Kevin Rue-Albrecht (17:03:13): > I got it - there was onefactor()conversion too many, between the heatmap and the legend.
Kevin Rue-Albrecht (17:07:21): > Though.. are you guys sure the centering/scaling was fixed? I mean the code update looks fine, but I still see rows of yellow/purple
Kevin Rue-Albrecht (17:16:16): > Separately, I’ll comb through the recent history now: does anyone remember touching anything that could cause the error'range' not meaningful for factorsfor redDimPlots ? I got that when I pulled the latestmaster
Kevin Rue-Albrecht (17:33:18): > hm.. nevermind, 1) i don’t see any update to the redDimPlot code, 2) i checked out the current upstream/master and I still have the same bug. I must have messed up my local SCE object
Kevin Rue-Albrecht (17:33:54): > although the onlyrange()calls that I can see are on the TSNE coordinates. what the hell
Aaron Lun (17:46:32): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (18:22:50): > eh.. yeah.. after re-running data.R, then wiping my session, then reloading the newly (likely identical SCE).. it worked again
Kevin Rue-Albrecht (18:23:18): > anyway.. still my heat map basically has rows of purple or yellow. am i missing sthg?
Aaron Lun (19:10:35): > yes, you’re probably missing something.
Aaron Lun (19:10:54): > Does the reported code have theavemeancalls?
2018-05-30
Kevin Rue-Albrecht (03:32:41): > that was the thing: I couldn’t see any reported code about this
Kevin Rue-Albrecht (03:33:06): > i was wondering if the code wasn’t picked up by the tracker yet
Charlotte Soneson (03:34:23): > I updated the tracker, I think it should all be in there. I’m on my way to the office, will check again when I get there.
Kevin Rue-Albrecht (03:34:47): > No rush. Also, I can’t seeavemean, onlyave
Kevin Rue-Albrecht (03:34:54): > (in the code, not the tracker)
Charlotte Soneson (03:36:02): > I had the same “range” problem as you when I pulled the updated heatmap code into my running R session. But restarting and regenerating the SCE object I couldn’t reproduce it so I figured it was some clash somewhere. If it shows up again we should investigate…for me, it showed up in all the panels (except the heatmap if I remember correctly).
Kevin Rue-Albrecht (03:37:12): > Oh. Interesting. I thought I was going mad
Kevin Rue-Albrecht (03:37:22): > (not mutually exclusive)
Aaron Lun (03:41:12): > Probably want to make sure this is base::range, then.
Aaron Lun (03:41:28): > Though, I mean, really, you should re-install fresh and restart the session.
Kevin Rue-Albrecht (03:42:01): > Alright, I don’t know exactly how that turns out this way… we all see the heat map update when we click the ‘centre’ and ‘scale’ checkboxes… but I just decided to printparam_choices[[.heatMapCenterScale]]and check if either option is in there, and I consistently getFALSEfor both
Kevin Rue-Albrecht (03:42:20): > It seems to miss[[1]]afterparam_choices[[.heatMapCenterScale]]
Aaron Lun (03:43:41): > Quite possibly, yes…
Kevin Rue-Albrecht (03:53:06): > Lines 110 and 113 of heatmap.R, if someone can add and commit that extra[[1]], it then shows the code in the tracker for me, and both logical turn to TRUE when I tick the boxes.
Aaron Lun (03:54:12): > Sounds like you already added it.
Kevin Rue-Albrecht (03:54:12): > I usually wouldn’t mind doing it myself, but this week is a bit tense and I don’t want to give someone another reason to make a fuss about a commit in another project
Kevin Rue-Albrecht (03:56:15): > Oh you know what, what the hell, i’ll commit. Tired of hiding
Aaron Lun (03:57:31): > :+1:
Federico Marini (07:18:35) (in thread): > photo or didn’t happen?:smile:
Aaron Lun (07:40:15): > @Aaron Lunuploaded a file:DSC_0001.JPG - File (JPEG): DSC_0001.JPG
Aaron Lun (07:40:30) (in thread): > see main channel.
Federico Marini (07:48:52): > Well than it did happen:slightly_smiling_face:
Federico Marini (07:49:35): > makes you somehow more badass
Aaron Lun (07:51:06): > Don’t know if that was even possible
Charlotte Soneson (07:54:32): > Is it very cold in your office?
Federico Marini (07:56:56): > Seems so. We have quite a massive hitze-wave - same there in Züri?
Aaron Lun (07:57:55): > Freezing. Some maniac turned the aircon to max.
Aaron Lun (07:58:00): > Takes a while to undo.
Charlotte Soneson (07:58:38): > Here it’s super hot. And no air conditioning at all…
Federico Marini (07:58:49): > good you have aircon, I don’t have it here and am minimizing my movements
Federico Marini (07:58:58): > keyboard typing as well
Kevin Rue-Albrecht (17:13:36): > I’ve open a PR for the updated TCGA tour. Please remember to rundata.Ragain.
Aaron Lun (17:41:17): > Devel or release?
Kevin Rue-Albrecht (17:46:27): > hm? I’ve tried to remain compatible with release. Only thing on my mind is that I’ve preserved thenlevelsof cancer type < 24 to make sure we don’t have a ‘categorical overload’ issue
Kevin Rue-Albrecht (17:48:55): > give me a chance to test it on release then, before we push it live
Kevin Rue-Albrecht (17:49:14): > I was just playing with the faceting branch in the meantime
Kevin Rue-Albrecht (17:49:42): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-05-30 22.48.08.png - File (PNG): Screenshot 2018-05-30 22.48.08.png
Kevin Rue-Albrecht (17:50:15): > there’ll be a few things to fix up …:sweat_smile:
Kevin Rue-Albrecht (17:54:18): > (I brushed before faceting) > 1) brush should probably be cleared > 2) new brushes are restricted within a single panel, and thus shouldn’t be drawn on all > 3) size of the square plots is currently computed across all facets
Kevin Rue-Albrecht (17:59:40): > oh wait.. indeed, the TCGA tour requires devel at the moment, due to the fix the heatmap scaling
Kevin Rue-Albrecht (17:59:49): > argh
Kevin Rue-Albrecht (18:00:50): > shall we cherry pick that couple of commits to release then?
Kevin Rue-Albrecht (18:02:12): > .. or is it affected by those other recent updates to the heatmap? (I must admit that I haven’t followed the heatmap dev closely)
Aaron Lun (18:06:25): > I’ll try to get devel iSEE installed on the server.
Kevin Rue-Albrecht (18:08:34): > Thanks.
Kevin Rue-Albrecht (18:08:59): > I think for the next 6 months, R-release should be enough foruseDevel
Aaron Lun (18:09:10): > Yes.
Aaron Lun (18:09:27): > Also, have you beheld my mohawk?
Kevin Rue-Albrecht (18:13:20): > haha yeah I saw it, the little emoji reaction was mine:slightly_smiling_face:
Kevin Rue-Albrecht (18:15:10): > are you going to switch from ABBA to metal then during your weekend office singstar sessions?
Kevin Rue-Albrecht (18:15:59): > Fede nailed the caption, saying it was making you look more badass
Federico Marini (18:20:10): > As long as this is not where this is heading:
Federico Marini (18:20:11): > https://asset.dr.dk/ImageScaler/?server=www.dr.dk&file=%2Ftjenester%2Fnetdoks%2Fudland%2Fbilleder%2Fcartoons%2Fsang-a.jpg&w=1510&h=849&scaleAfter=ratio&quality=38&ratio=16-9 - Attachment: Attachment
Kevin Rue-Albrecht (18:21:55): > I think you still have margin before you get there:rolling_on_the_floor_laughing:
Aaron Lun (18:32:59): > Next step is to dye it.
2018-05-31
Aaron Lun (05:50:53): > God, that HER2 signature is crap.
Aaron Lun (05:51:09): > Even a DE analysis between HER2-high and low samples would probably give better results.
Aaron Lun (05:58:27): > Well, whatever. Just do the little fixes and I’ll merge it.
Kevin Rue-Albrecht (06:14:00): > Yeah. I know that heat map isn’t great. Pr baby the reason why they only show the heat maps of their own experiment
Kevin Rue-Albrecht (06:14:53): > From memory they even do say something about reproducibility issues between donors
Aaron Lun (11:05:05) (in thread): > These tasks… will not be straightforward. (3) will require a complete change to the generating code, and we will have to think about how to do this properly.
Kevin Rue-Albrecht (11:57:04) (in thread): > I’m not the one who said it should be “easy enough”:stuck_out_tongue_winking_eye:https://github.com/csoneson/iSEE/issues/181 - Attachment (GitHub): Faceting options for dot plots · Issue #181 · csoneson/iSEE > Pretty self-explanatory, and it should be easy enough - just add a facet option to the visual parameters, along with the x- and y-axis variables to facet by (these need to be categorical, presumabl…
Aaron Lun (11:58:45) (in thread): > I - uh - don’t recall.
Kevin Rue-Albrecht (11:59:31) (in thread): > Joke aside, I finally found last night how to add the selectize items to the UI, to select the faceting covariates. (hence the screenshot yesterday). However, I’ve only done it for the colDataPlot so far. I’ll clean up this weekend
Kevin Rue-Albrecht (12:00:30) (in thread): > To be fair, you opened that issue a month ago. I wasn’t in a rush to get to it, as I could already imagine how involved that feature would be. Plus, we had other more pressing stuff to fix first, for the tours.
Aaron Lun (14:12:53): > You did see my suggestions, right?
Kevin Rue-Albrecht (14:28:16): > on the PR? - yeah
Aaron Lun (14:31:56): > good
Kevin Rue-Albrecht (14:32:03): > I just have to finish preparing a “once in 6 months” type of meeting with a collaborator for tomorrow. I’ll see if I can fix up the few edits and chase down the missing samples tonight, but it might be only tomorrow.
Kevin Rue-Albrecht (14:32:17): > For the samples, I thinkNAcan make samples disappear
Aaron Lun (14:32:41): > Yes, they will disappear. And I thought of that too. Best to check.
Kevin Rue-Albrecht (14:33:31): > Note however, that the mightyggplothas options to retainNAas an extra level, for discrete (categorical) covariates
Kevin Rue-Albrecht (14:34:32): > i’d have to check again, although I thought we were using that feature
Aaron Lun (14:36:38): > NAs get explicitly removed.
Aaron Lun (14:36:42): > It’s the only safe way.
Kevin Rue-Albrecht (14:38:48): > Ah right - now that you say, it does ring a bell, seeing code about that. I must have confused with an older implementation
Kevin Rue-Albrecht (14:45:58): > Tomorrow’s meeting can wait 5 min. Here’s my homework, that confirms the numbers of samples between the 2 colDataPlots: > > dim(sce) > [1] 23368 8411 > > brca <- sce[,sce$CancerType == "BRCA"] > dim(brca) > [1] 23368 1195 > > brca_CNTL <- brca[,(brca$CNTL)] > dim(brca_CNTL) > [1] 23368 1195 > > brca_CNTL_her2 <- brca_CNTL[,(brca_CNTL$her2_status_by_ihc)] > dim(brca_CNTL_her2) > [1] 23368 994 >
Kevin Rue-Albrecht (14:46:14): > However, dinner can’t wait. Catch up later!
Aaron Lun (14:47:43): > racked up another 400 commit month
2018-06-01
Kevin Rue-Albrecht (11:08:10): > getting a Friday vibe here:slightly_smiling_face:
Kevin Rue-Albrecht (11:10:19): > what’s the status with f1000 these days btw? are we waiting on their iframes, or are they waiting on our updated tours?
Aaron Lun (11:12:28): > We’re waiting on them.
Aaron Lun (11:12:33): > Is the tour ready to go?
Kevin Rue-Albrecht (11:16:29): > I’ve fiddled with it last night, but haven’t pushed yet. I wanted to take it for another tour (oh-oh) before giving the green light
Kevin Rue-Albrecht (11:17:06): > This evening will be done.
Kevin Rue-Albrecht (11:17:34): > I’m just as irritated as you by the $#!t heatmap of HER2 signature
Kevin Rue-Albrecht (11:19:00): > Still. Thing is I’m giving a tour of the dataset as reported by them. It’s not about doing a new analysis, unfortunately.
Aaron Lun (11:26:49): > Yeah. Well, maybe mention something in the tour about it, otherwise people will think that we fucked up.
Kevin Rue-Albrecht (11:33:11): > True. It crossed my mind, but I’m still working on a politically correct phrasing.
Kevin Rue-Albrecht (11:33:51): > “The heat map looks shit. Please contact the authors for more information.” still needs a bit of work, I think.
Kevin Rue-Albrecht (12:01:13): > what the hell is this PR?
Kevin Rue-Albrecht (12:03:18): > Ah ok, he didn’t actually offer any new code.
Kevin Rue-Albrecht (12:06:14): > It’s actually not a bad idea to allow shape for data points. I’ve been so focused on the applications for single-cell data that I didn’t think of scenarios where there are few enough data points that their shape would actually become visible enough to be worth controlling it
Kevin Rue-Albrecht (12:07:33): > On the + side, I think all the aesthetic infrastructure is there already. It should be as easy as adding the UI controls
Aaron Lun (12:15:37): > Yes, but we have bigger fish to fry.
Kevin Rue-Albrecht (12:40:19): > Carp diem … sort of
Aaron Lun (12:40:58): > sighjust fix the damn tour
Kevin Rue-Albrecht (12:41:29): > I know. I’m heading back home now. I’ll push tonight
Aaron Lun (17:45:30): > :+1:
Kevin Rue-Albrecht (17:51:21): > Cool. I was about to drop the exclamation mark (in the last step), but you already merged. Plus I’m sick of fixing this damn heat map.
Aaron Lun (18:13:04): > Are you going to work on the faceting this WE?
Aaron Lun (18:13:53): > There’s probably some mods to make it easier - namely spinning out all the square/violin plot set-up code into separate exported functions.
Kevin Rue-Albrecht (18:17:27): > As a matter of fact, I was just adding unit tests right now.
Kevin Rue-Albrecht (18:21:57): > I’ve stashed the unit tests til tomorrow. However, faceting is effectively implemented now, pending integration with the brushes and stuff
Kevin Rue-Albrecht (18:22:08): > feel free to take the feature for a ride
Kevin Rue-Albrecht (18:24:27): > word of advice, don’t try to facet on continuous covariates. I would like to say that bad stuff happens… but essentially nothing happens as the app freezes, probably trying to create one facet by value
Kevin Rue-Albrecht (18:25:36): > @Kevin Rue-Albrechtuploaded a file:spoiler alert - File (PNG): spoiler alert
Kevin Rue-Albrecht (18:28:26): > @Kevin Rue-Albrechtuploaded a file:one more for good luck - File (PNG): one more for good luck
Aaron Lun (19:27:27): > Looks decent.
Aaron Lun (19:27:37): > I will be working onmnnCorrect2on the WE.
Aaron Lun (19:27:54): > So won’t get to doing iSEE-related stuff unless I’m bored.
Aaron Lun (19:28:01): > Which might happen sooner rather than later, but let’s see.
Aaron Lun (19:29:47): > Now, here’s an interesting question - are the faceting options visual parameters or data parameters?
Aaron Lun (19:30:01): > It wouldprobablybe more natural to have them as visual parameters.
2018-06-02
Kevin Rue-Albrecht (05:23:33): > I put them in visual, with the other tick boxes
Aaron Lun (06:45:59): > Makes sense.
Aaron Lun (07:06:06): > <!channel>Can all authors check their tours on the marioni lab server, as it’s just been updated to use the Github version of iSEE?
Aaron Lun (07:19:00) (in thread): > I would suggest limiting the choices toonlycategorical variables during UI generation. At app start-up, identify all categorical metadata, store that information in theint_metadataof the SCE, and then retrieve it for use in.panel_generation()when setting up theselectInputs for faceting. This ensures that the app canneverbe invalid. Similarly, turn off faceting choices if there are no available metadata fields.
Aaron Lun (07:19:54) (in thread): > Then we don’t have to worry about catching invalid faceting choices downstream.
Aaron Lun (07:22:32) (in thread): > Violin and square plot recalculations should probably be done inviolin_setupandsquare_setup, though this will be messy… the easiest way to do it would be topasteexisting groups with the faceting options, but that’s not entirely safe and it is rather inefficient… hm…
Kevin Rue-Albrecht (13:33:16): > alright, just went through my tour step by step and can’t find anything wrong with it (aside from the crappy heatmap, obviously)
Kevin Rue-Albrecht (13:45:15): > probably useless announcement: please no one touches unit tests: I’ve globally reindented plotting.R to 4-space, which ‘could’ cause a hell of conflicts
Kevin Rue-Albrecht (14:02:31): > Actually, I can trim down the new faceting code by nearly half: there are virtually no difference between column and row data plots, aside from the choice of covariates, which is already handled by.create_visual_box_for_row_plots
Aaron Lun (14:11:56): > Good.
Kevin Rue-Albrecht (14:12:45): > thank unit tests for that - copy pasting helps notice redundancy
Kevin Rue-Albrecht (14:13:18): > on a separate note, can anyone think of anything else than colData and rowData, that users might want to facet on?
Aaron Lun (14:14:53): > No, as these are point-level attributes.
Kevin Rue-Albrecht (14:15:07): > ok - so i didn’t miss anything obvious then
Kevin Rue-Albrecht (15:24:54): > question about heat maps: do we have any plan to allowcolumnclustering (@Charlotte Soneson). I can imagine lots of extra requests/issues coming from it, but I think even a simple implementation of it could help visualisation. Thoughts welcome of course
Charlotte Soneson (15:26:57): > Yeah…guess we are moving into analysis in that case. The problem would be the large number of cells I guess
Kevin Rue-Albrecht (15:27:44): > One ‘advanced’ implementation that I could think of would be clusteringwithineach ‘group’ of samples (if we want to preserve at least the grouping of samples). > But you’re right, probably too much into analysis already
Charlotte Soneson (15:28:10): > We are limiting the number of genes that we allow, but there could be many more cells than that (and will be in most single-cell data sets)
Charlotte Soneson (15:28:24): > People can cluster before and include the order in the colData:slightly_smiling_face:
Kevin Rue-Albrecht (15:28:38): > very true, that last one
Charlotte Soneson (15:29:31): > btw, Marioni server CyTOF tour also works@Aaron Lun
2018-06-03
Aaron Lun (06:20:22): > good
Kevin Rue-Albrecht (06:22:47): > anyone wanna speak now against merge themore_discretePR, or forever hold your peace?
Aaron Lun (06:27:34): > Fine with me.
Kevin Rue-Albrecht (06:31:26): > Cool, i’ll do now, before the branch gets outdated by another commit
Federico Marini (15:03:25): > @Aaron Lun: Allen tour fine for me!
2018-06-05
Kevin Rue-Albrecht (16:54:55): > Just to say that I haven’t dug deep, but I found that:https://stackoverflow.com/questions/40102096/r-shiny-brush-with-facet-grid-doesnt-select-correct-pointswhich doesn’t seem fixed yet, as faceting turns my valid selections into ‘0 points selected’ - Attachment (Stack Overflow): R Shiny brush with facet_grid doesn’t select correct points > Please run the reproducible code here first. Here are two box plots created. The upper one is simple. The bottom one is a box plot with facet_grid. The idea is to link them together. Brushing po…
Kevin Rue-Albrecht (16:55:57): > After the lasso selection, we might be about to fix another shiny awaited features/fixes
Kevin Rue-Albrecht (17:32:08): > I’m not sure what the catch is, butbrushedPoints()is supposed to handle faceting.
Kevin Rue-Albrecht (17:35:08): > As a matter of fact, I just spotted something strange: > -in the app, brushing on a faceted plot does not select the point (as indicated by the ‘0 points selected’ messageandthe fact that a linked plot does not highlight the selected point) > - however, copy-pasting the reported code in an R shelldoeseverything as expected (i.e. the call tobrushedPointsreturns a non-empty data frame, and the receiving plot does highlight the selected points) > Can anyone confirm?
Kevin Rue-Albrecht (17:36:21): > PS: I’ve tested that using thefacetbranch in which I’ve mergedmasterearlier this evening, and just pushed now
Kevin Rue-Albrecht (17:44:55): > Ok. I (partially) know why.all_coordinates- which is used to apply brushes - seems to only store X/Y coordinates > that obviously causes problems when the brushes needs faceting information to know within which panel to subset (see the source code ofbrushedPoints)
Kevin Rue-Albrecht (17:48:20): > Now, looking at the code reported,all_coordinatesis supposed to be updated with theplot.datathat contains the faceting data columns at that point. > … which suggests to me that there may be somelocalissue somewhere, whereall_coordinatesis updated within a local scope, but doesn’t remain updated for subsequent plots. > I recommend addingprint(lapply(all_coordinates, "head"))at line 409 ofplotting.Rfor debugging
Kevin Rue-Albrecht (17:59:38): > Actually, I think I got it. > Each panel is currently evaluated in its own new environment (eval_env <- new.env()at line 406 of plotting.R) > Which differs from the user experience of running all the reported code in the same session, as I did above. > I assume this somehow explains thatall_coordinatesonly ever knows about the XY coordinates, probably set during initialisation, but never remembers the optional additional covariates such as facets or colors (I just tested the latter now)
Kevin Rue-Albrecht (18:01:39): > Nevermind, got it,pObjects$coordinates[[plot_name]] <- p.out$xy[,c("X", "Y")]at line 1245 of iSEE-main.R
Kevin Rue-Albrecht (19:03:11): > Alright, selection through Shiny brushes fixed for faceted plots: > - lassos will require extra attributes to work@Aaron Lun. Shiny brushes stored the faceting aesthetics (names of faceting covariates) as well as the level of the faceting variables corresponding to the panel brushed. At the moment, a lasso drawn in one panel selects across all panels (at least it works in a way!) > - the persistent brushes, both Shiny and lasso, are drawn on all panels. I’m curious to look into this myself next time I get the chance, as it should be a matter of simply adding an aesthetic to the ggplot call
2018-06-06
Aaron Lun (06:10:24): > We will need to spin this out into a new function, e.g.,lassoPoints.
Kevin Rue-Albrecht (06:10:42): > absolutely
Kevin Rue-Albrecht (06:11:15): > btw, I nearly solved the “show the persistent Shiny brush only in the panel that was brushed” last night
Kevin Rue-Albrecht (06:14:48): > there is just a catch whether faceting is doneeitherin a single direction (i.e. row or column) on onboth(rowandcolumn) > with the latter behaving likeflip=TRUEeven when the plot doesn’t callcoord_flipanyway, I’ll let you know when it’s fixed and pushed
Kevin Rue-Albrecht (15:06:29): > Alright. I think I’ve completed the update to handle Shiny brushes in faceted plots. Branchfacet, for those who’d like to try and break it.
Kevin Rue-Albrecht (15:08:13): > As per earlier messages, Note that lassos are not supported in faceted plots yet. That’ll require extra attributes/functions to store the faceting and panel information in the lasso brush.
Aaron Lun (17:58:20): > I may or may not get to this tomorrow, depending on my load.
Kevin Rue-Albrecht (18:09:42): > No rush. I’ve got code to clean up for the rest of the week. Spreading the good R package development practice around me:wink:
Aaron Lun (18:12:37): > The to-do list is getting quite large here, so perhaps we should start a Project to coordinate things. First order of business would be the lasso handling.
Aaron Lun (18:13:40): > Then there are issues with moving things out of code-as-strings and into iSEE exported functions, where they can be tested more rigorously.
Aaron Lun (18:14:06): > e.g., violin and square plot definitions with faceting are no longer trivial.
Kevin Rue-Albrecht (18:24:13): > i just googled GitHub project (https://help.github.com/articles/about-project-boards/), as I’ve never used them before. I assume that’s what you mean? - Attachment (help.github.com): About project boards - User Documentation > Project boards on GitHub help you organize and prioritize your work. You can create project boards for specific feature work, comprehensive roadmaps, or even release checklists. With project boards, you have the flexibility to create customized workflows that suit your needs. …
Kevin Rue-Albrecht (18:24:28): > Nevermind
Kevin Rue-Albrecht (18:24:59): > Just saw the “Projects” tab at the top of the GH repo staring right back at me
Aaron Lun (18:34:38): > seegithub.com/LTLA/csawfor an example of an epic project.
2018-06-07
Kevin Rue-Albrecht (05:04:50): > I like the ‘epicity’ of csaw. Might bother you about it when I get some data processed in the next few weeks
Kevin Rue-Albrecht (05:05:33): > Anyway. Happy to organise Projects, I can look at it this weekend
Kevin Rue-Albrecht (10:49:05): > Do you guys know if anything is broken in SCE or SE packages? Locally, I can’t compile the iSEE vignette because theshowmethod for SCE crashes with error: > > > sce > class: SingleCellExperiment > dim: 20908 379 > metadata(3): SuppInfo which_qc log.exprs.offset > assays(6): tophat_counts cufflinks_fpkm ... counts logcounts > rownames(20908): 0610007P14Rik 0610009B22Rik ... Zzef1 Zzz3 > Error in .local(x, ...) : unused argument (use.names = FALSE) >
Aaron Lun (10:52:47): > Pull from GH.
Kevin Rue-Albrecht (11:13:23): > nice. all good now
2018-06-08
Kevin Rue-Albrecht (05:08:11): > @Aaron Lunwill be happy to hear that the ggplot updates include “scale_*_gradientn()now allowscoloursorcolors(#1290)“:wink:
Aaron Lun (05:12:34): > YES
2018-06-09
Aaron Lun (08:52:56): > Line 1883: why is it being compared to a dot?
Aaron Lun (08:53:05): > we have a.noSelectionfor something that hasn’t been selected.
Aaron Lun (08:54:06): > same for line 1802
Aaron Lun (08:54:15): > and 1768
Aaron Lun (08:54:24): > ofplotting.R.
Aaron Lun (08:55:18): > though really, if nothing was selected, we should go withNA, as all strings are possible covariates.
Aaron Lun (08:56:13): > Also, you should restrict the UI to only allow selection of discrete variables.
Aaron Lun (08:56:54): > likecolumn_discrete_covariatesandrow_discrete_covariates.
Kevin Rue-Albrecht (08:59:45): > the.is because that’s the way to tell ggplot “don’t facet”
Aaron Lun (09:00:03): > Well, that’s going to be confusing.
Aaron Lun (09:00:15): > How will the faceting be turned off?
Aaron Lun (09:00:37): > If there’s a.in the possible column names
Kevin Rue-Albrecht (09:00:59): > hm.. what if there’s a---
Aaron Lun (09:01:08): > Yeah, that’s what I was saying withNA.
Aaron Lun (09:01:19): > Though the safest option is to have a check mark saying “faceting on”
Aaron Lun (09:01:25): > and a conditional select with the possible choices.
Aaron Lun (09:02:16): > Let’s go with that.
Aaron Lun (09:02:30): > A bit heavier, but that’s the price of security.
Aaron Lun (09:04:41): > Now, how do we protect users against selecting continuous covariates? We could maintain its validity bycutting it into a certain number of user-specified groups. That would allow us to avoid having to actively protect users against it.
Aaron Lun (09:04:58): > Well, while you think about that, I’m going grocery shopping.
Kevin Rue-Albrecht (09:11:13): > yep sorry, we’ve got our timing off, was helping around the house
Kevin Rue-Albrecht (09:20:38): > the idea of a “facet on/off” toggle did cross my mind, as it could have allowed users to select their categorical variablesbeforerisking toggling the faceting on. At the same time, I just went the quickest way to get a prototype running and I didn’t imagine a.field in colData or rowData, so I thought it would be safe enough. > Conclusion, I think we should go a bit heavier and have that faceting toggle
Kevin Rue-Albrecht (09:30:04): > Cutting continuous variables sounds a power user feature, which I think doesn’t address things like character fields (e.g. “sample_id”).
Kevin Rue-Albrecht (09:31:51): > I suppose we could simply force users tofactor()everything that is categorical, and we’d coerce everything else to numeric, especially character values
Kevin Rue-Albrecht (09:32:59): > because, as you rightly pointed out a while back,integeris not enough to know whether we’re dealing with categorical or continuous.
Kevin Rue-Albrecht (09:41:24): > Here, I’ve cleaned up a bit and pushed the ‘grouping’ functions into a separate file. > I know it’s not ideal, but so far it struck a compromise between the app respecting user-defined factors, and protecting them against untyped variables (mostly character fields, as integer fields not typed as factors were just handled as numeric)
Kevin Rue-Albrecht (11:17:33): > alright, groceries done here as well.
Kevin Rue-Albrecht (11:18:42): > od we have a dropdown with NA anywhere yet? I’m not sure whether NAs show up in choices
Aaron Lun (11:21:42): > No, that’s a bad idea.
Aaron Lun (11:21:45): > having thought about it
Aaron Lun (11:21:50): > shiny will just spit the dummy
Kevin Rue-Albrecht (11:25:37): > also, having thought about the toggle, I’m not so sure it’s a good idea anymore, if we don’t have a ‘null’ value, then we’d need separate toggles for row and column faceting
Kevin Rue-Albrecht (11:30:10) (in thread): > see commithttps://github.com/csoneson/iSEE/commit/7e40f269ac975081c3bc4f9ff488ad07494f485c, I was working on this idea
Kevin Rue-Albrecht (11:31:49) (in thread): > just haven’t finished it yet. And also, we need some fallback plan whennoneof the variables are groupable
Aaron Lun (11:32:47): > Yes, that’s fine.
Aaron Lun (11:33:48) (in thread): > I think it would be a lot easier if we just forced continuous variables to be groupable.
Aaron Lun (11:34:46) (in thread): > This frees us from having to change the UI, and allows us to directly plug into the rest of the UI protection - e.g., faceting can be completely turned off when there are no covariates, just like how other options get disabled.
Aaron Lun (11:35:05): > It’s fine to have separate toggles.
Kevin Rue-Albrecht (11:35:36) (in thread): > I can see the issue in that. First off, I’ll say that a covariate named.is stupid. However, it would still be technically feasible to support it for faceting, as I store the faceting covariates under the namesFacetRowandFacetColumninplot.data. > Obviously, this won’t work as long as I use.as the “don’t facet” option in the dropdown
Kevin Rue-Albrecht (11:35:59): > ah. ok. seemed heavy, but I can see the point in it
Kevin Rue-Albrecht (11:37:36) (in thread): > force continuous to be groupable? but how would users do a scatter plot then, if they wanted to?
Kevin Rue-Albrecht (11:39:51) (in thread): > how about we just say that anything that’s not a factor is not groupable?
Kevin Rue-Albrecht (11:40:49) (in thread): > That’ll train people to just clearly state what is groupable. Then we’d simply coerce anything thing that’s not a factor to numeric. No more guesswork.
Aaron Lun (11:42:10) (in thread): > No, this is to protect ourselves.
Aaron Lun (11:42:28) (in thread): > For the purpose of faceting, if the user specifies a continuous variable, wecutit and make it discrete.
Aaron Lun (11:42:43) (in thread): > If you don’t want to do that, you will have to restrict the choices in the UI to be discrete only.
Aaron Lun (11:43:25) (in thread): > This would require a run through thecolDataat every UI regeneration to check for groupability.
Kevin Rue-Albrecht (11:43:26) (in thread): > I was thinking about the latter, when
Aaron Lun (11:43:48) (in thread): > You will also have to sanitize the memory in case someone provides a non-groupable input as the initial setting for the app.
Aaron Lun (11:44:30) (in thread): > The most self-contained solution, therefore, is to just find a way to maintain validity at all user choices.
Kevin Rue-Albrecht (11:44:45) (in thread): > what’s the impact on memory?
Aaron Lun (11:45:05) (in thread): > Well, say I specifiedfacetColumnas some non-groupable covariate column.
Aaron Lun (11:45:34) (in thread): > You would have to check all arguments for sanity during the start of the app.
Kevin Rue-Albrecht (11:45:57) (in thread): > Oh god, i get it.
Aaron Lun (11:45:57) (in thread): > Restricted choices in the UI only provides protection during the lifetime of the app.
Kevin Rue-Albrecht (11:46:39) (in thread): > i.e. even if the dropdown offers only valid choices, users may initialise the app in an invalid choice state… right?
Aaron Lun (11:47:04) (in thread): > Yes.
Kevin Rue-Albrecht (11:47:58) (in thread): > damn that weird, can a selectize be initialised with a selected value that’s not in choices? I thought not
Aaron Lun (11:50:58) (in thread): > It will be set to empty, I believe. But that still leaves a small interval between the app start and the UI setup where the memory is invalid. This has a few implications, not least due to the fact that there are some run-throughs of the plotting functions to seed the coordinates for linking.
Kevin Rue-Albrecht (11:53:50) (in thread): > yeah ok, i can’t picture all the implications, but I can imagine that it’s not ideal.
Aaron Lun (11:57:52) (in thread): > Basically: if you allow all covariates, then you can piggy-back off thecovariatesargument in the various internal UI functions. If you only allow discrete covariates, you will have to either check for discreteness in each internal function (which is sometimes not even possible), or pass another argument specifying covariate discreteness to all internal functions.
Kevin Rue-Albrecht (11:59:42) (in thread): > yeah.. in the current setting, all I could think of so far was to pass the full list of covariatesandthe subset of discrete covariates
Kevin Rue-Albrecht (12:00:53) (in thread): > grep forcolumn_groupableindynamicUI.Rin my latest commit to see what I mean
Kevin Rue-Albrecht (12:01:34) (in thread): > it doesn’t do anything yet, so it’s harmless, but I just wanted to show what I had in mind
Kevin Rue-Albrecht (12:02:26) (in thread): > still, it’s probably not optimal either, as you pointed out: this would get repeated at every UI regeneration
Aaron Lun (12:03:06) (in thread): > Well, that’s not the main problem.
Aaron Lun (12:03:15) (in thread): > The main problem is that the internals are getting messier.
Kevin Rue-Albrecht (12:04:35) (in thread): > True. > My hesitation is just where to start the cleanup
Aaron Lun (12:05:21) (in thread): > We could probably store a lot of information in the SCEint_metadata.
Aaron Lun (12:05:40) (in thread): > And just keep on passing the SCE object, and extract out information from the container as required.
Kevin Rue-Albrecht (12:05:56): > Speaking of “shiny will just spit the dummy”.. I just tested, and Shiny turns a realNAvalue into a"NA"character in the dropdown, which screws up the logic even more
Aaron Lun (12:06:52) (in thread): > e.g., do all of the processing for UI-related choices once, and just archive it in theint_metadata.
Kevin Rue-Albrecht (12:08:12): > in other words, it comes back to your example of users initialising with invalid values, even though I initialise withNA, the selectize stores"NA"inparam_choices, so I’d end up havign to test=="NA", which is even worse than"."
Kevin Rue-Albrecht (12:10:06) (in thread): > .. like identifying the groupable variables upon initialisation/sanitisation, store data types inint_metadata, and refer to that one to populate the widgets.. correct?
Aaron Lun (12:11:01) (in thread): > Yes.
Kevin Rue-Albrecht (12:14:33) (in thread): > I’ll add that to the “faceting” Project then. > Have you had a chance to see the mess I’ve done of it already? > Notes started looking more like issues, but I also didn’t want to open issues for those things until we discussed them
Aaron Lun (12:15:19) (in thread): > ¯*(ツ)*/¯
Aaron Lun (12:15:47) (in thread): > I think we need to consolidate what we have before we tackle lassos.
Kevin Rue-Albrecht (12:21:06) (in thread): > Right.. “what we have” including the recent work onfacetbranch, right? I’ll continue there rather branch offmasteragain
Aaron Lun (13:08:49) (in thread): > Yes.
Aaron Lun (13:09:03) (in thread): > I will start looking at this once I eat my dinner.
Kevin Rue-Albrecht (14:22:31) (in thread): > I’m just starting to cook now, but using code in my latest push earlier today, I suppose what you expect would be something likesce@int_metadata$groupable_colData <- names(.which_groupable(colData(sce)))
Kevin Rue-Albrecht (14:22:55) (in thread): > and equivalent forrowData
Aaron Lun (14:29:04) (in thread): > uh.
Aaron Lun (14:29:07) (in thread): > Hm.
Aaron Lun (14:29:18) (in thread): > Come back after dinner and see where it’s at.
Kevin Rue-Albrecht (14:30:41) (in thread): > Ok, no point in duplicating effort, if you’re onto an implementation already
Aaron Lun (16:37:29): > My changes are done for today. CHECK is crashing but I don’t have the will to figure out why. Add faceting sanity checks to.setup_memory, I have indicated where they should go. Specifically, the “facet by row/column” check boxes (that are yet to be added) should be turned toFALSEwhen there are no discrete covariates.
Kevin Rue-Albrecht (16:39:11): > ok, will checkk. Thanks
Kevin Rue-Albrecht (18:29:52): > @Aaron LunI have a feeling that everyall_argsreference from l. 233 onwards in.setup_memoryshould be referring tomemory. Took me a while to realise it (i didn’t look above that line enough), but I can’t see a reason to edit something that you’re not returning anymore.
Kevin Rue-Albrecht (19:36:19): > I’ll probably push and go to bed now, I’ve got a a set of 2 “grouped checkboxes” for row/column faceting, as for heatmap center/scale, whichalmostworks, except that it needs the same# Saving list-based values.treatment
Kevin Rue-Albrecht (19:46:54): > So yeah, logging off. > Aaron, I’ll need your double-expertise on ‘# Saving list-based values.’ (see heatmap section) and ‘protected’ fields that “# Defining fundamental parameters that destroy brushes/lassos upon being changed.”
Kevin Rue-Albrecht (19:47:45): > Right now, only the last box ticked takes effect. And unticking both boxes leaves the last one active. (I suppose we haven’t enabled NULL to trigger replot)
2018-06-10
Kevin Rue-Albrecht (05:06:12): > « From an intern in the Zurich office »… hmmm..:thinking_face:Google AI Blog: Realtime tSNE Visualizations with TensorFlow.jshttp://ai.googleblog.com/2018/06/realtime-tsne-visualizations-with.html - Attachment (Google AI Blog): Realtime tSNE Visualizations with TensorFlow.js > Posted by Nicola Pezzotti, Software Engineering Intern, Google Zürich In recent years, the t-distributed Stochastic Neighbor Embedding (…
Kevin Rue-Albrecht (05:06:53): > https://twitter.com/googleai/status/1004802557166829569?s=12 - Attachment (twitter): Attachment > Some new research from an intern in our Zürich office shows an approach to tSNE that allows real-time interactive visualization of large, high-dimensional datasets by leveraging GPU capabilities through WebGL. Oh, and it’s open source too! Check it out ↓ http://goo.gl/oxPgh4
Aaron Lun (06:06:18) (in thread): > Yes, that’s probably true.
Aaron Lun (06:07:16) (in thread): > I would go with one separate check box for each, rather than grouped. Personally I think the grouping was a bad idea in the visual parameters.
Aaron Lun (06:08:04): > Oh, we knew webGL would be a thing,@Charlotte Sonesonand I saw it at the HCA meet
Aaron Lun (06:09:09): > It felt like something that would be more centrally deployed.
Aaron Lun (06:09:21): > Rather than distributed likeiSEEcan be .
Kevin Rue-Albrecht (08:13:07) (in thread): > i should have confirmed, it did fixed that part of it
Kevin Rue-Albrecht (08:13:36) (in thread): > ok, i also thought that individual checkboxes would always have TRUE/FALSE, without the issue of NULL
Kevin Rue-Albrecht (08:15:13) (in thread): > i’ll update that now then
Kevin Rue-Albrecht (08:56:57) (in thread): > Alright. One thing to point out: I’ve put both the toggle checkboxesandthe dropdown in protected params, which causesoneundesirable behaviour: the panel gets replotted if the dropdown is changed when the checkbox is unticked
Kevin Rue-Albrecht (08:57:31) (in thread): > suggestions?
Kevin Rue-Albrecht (08:58:39) (in thread): > Ah, and actually, point 2) individual checkboxes show up on separate lines (one benefit of grouped checkboxes wasinline=TRUE
Kevin Rue-Albrecht (09:01:03) (in thread): > For 2) I am vaguely tempted by a radio button facetby = “None | Row | Column | Both”, but I don’t like the idea of spelling out the combinations, it’s a waste of space
Kevin Rue-Albrecht (09:25:09) (in thread): > @Aaron LunI’m heading out for a bit. I’ve locally started dealing with the ‘shapeBy’ issue (#205) while waiting on feedback for the state of faceting.
Aaron Lun (09:25:32) (in thread): > Don’t bother dealingw ith that yet.
Aaron Lun (09:25:45) (in thread): > Need to fix square and violin plot scatter for the facets
Aaron Lun (10:00:34) (in thread): > You’ll notice that the violin x-jitter is computed globally but that gets applied to each facet; this is incorrect.
Aaron Lun (10:03:46): > Going for a walk.
Kevin Rue-Albrecht (10:17:01) (in thread): > Ah yeah I forgot tabout that
Kevin Rue-Albrecht (10:49:27) (in thread): > To solve 1), I think that we should hide the corresponding facetBy selectize when the corresponding facet row/column checkbox is unticked
Aaron Lun (11:13:52): > Having considered it in more depth, perhaps we shouldn’t worry about sanitizing the input parameters.
Aaron Lun (11:14:24): > Just tell people that invalid parameters will be used as-is.
Aaron Lun (11:14:45): > We’re only obliged to protect the user
Aaron Lun (11:14:57): > People who are launching the apps with custom settings should be able to deal with it.
Kevin Rue-Albrecht (11:15:10): > what’s your thinking? now that it’s there, I don’t mind having it, although obviously ‘blame the user’ is a decent strategy when the user provides crap inputs
Aaron Lun (11:15:18): > it’s Unsustainable
Aaron Lun (11:15:34): > I’ve realized that we have to do this sanitization for every argument.
Aaron Lun (11:15:45): > That’s… a lot.
Kevin Rue-Albrecht (11:16:04): > Ok, i agree. Less is more in many cases, and it applies here I think.
Aaron Lun (11:16:15): > I will need to think more about this.
Aaron Lun (11:16:31): > Mostly Shiny will protect us anyway
Aaron Lun (11:16:47): > because the UI will initialize and force selections to a sensible value
Kevin Rue-Albrecht (11:16:52): > I mean, so long asweinitialise the app to a valid state, it’s the user’s responsibility to initialise it with valid parameters if they customise it.
Kevin Rue-Albrecht (11:17:27): > Although in that case, it’sourresponsibility to document as best we can how they may customize startup params
Aaron Lun (11:18:19): > The correct approach would be to have another suite of functions for validating start-up params.
Aaron Lun (11:18:31): > One per*Defaults()function.
Kevin Rue-Albrecht (11:18:59): > I think it make sense
Aaron Lun (11:19:27): > That would be a real pain in the arse.
Aaron Lun (11:19:35): > Need to think about how to do it smoothly.
Aaron Lun (11:20:02): > … and I know how.
Aaron Lun (11:20:15): > But it’s a big-ish undertaking, so let’s put it to the side.
Aaron Lun (11:21:02): > Basically, for every parameter we define a triple of (parameter_name, default_value, validity_function).
Aaron Lun (11:21:18): > This means we have to define this triplet once - and exactly once - every time we add a new argument.
Aaron Lun (11:21:49): > We then go through this definition to create theDefaults()andValid()functions.
Kevin Rue-Albrecht (11:21:51): > hm.. I think I can see your idea. The little brother to class validity functions
Aaron Lun (11:22:23): > This ensures that any new parameter must have some way to specify whether it is valid or not.
Aaron Lun (11:22:57): > With that in mind, I’ll delete the current parameter checks.
Kevin Rue-Albrecht (11:25:34): > Ok. A bit something like the.helperfunction you currently have I think, with the validity function ‘knowing’ for each parameter, the set of valid values (e.g. for assay choices, either an integer withseq_along(assays(...))or a character withinassayNames(...)
Aaron Lun (11:28:10): > Yes, basically.
Kevin Rue-Albrecht (12:31:51): > how’s the big cleanup coming along? or are you onto something else?
Aaron Lun (12:35:53): > Just finished.
Aaron Lun (12:36:03): > Well, it’s not the cleanup of the vaidity
Aaron Lun (12:36:11): > it’s ensuring that the jitter is facet-aware
Aaron Lun (12:36:14): > Which it now is.
Aaron Lun (12:36:29): > The remaining issue is how to draw the rectangles within each facet.
Aaron Lun (12:36:54): > You need to modify the square plotting code to do so.
Kevin Rue-Albrecht (12:38:40): > Oh god. Indeed.
Kevin Rue-Albrecht (12:40:05): > Although (without looking at the code directly), I suspect that ando.call("rbind", lapply(...))could build up a data.frame just fine for the square plotting code
Kevin Rue-Albrecht (12:40:37): > (with minor modifications, of course, but still, not that bad)
Kevin Rue-Albrecht (12:42:53): > if you look at.self_brush_box, you’ll see that all that’s needed are an extra two columns in the lasso plotting data.frame, namedFacetRowandFacetColumn, that declare in which panel each lasso waypoint should be drawn
Kevin Rue-Albrecht (12:45:36): > oh god wait.. it’s not so much theplottingaspect than the lasso ‘resetting’ that’s going to be a pain: given that a lasso should only ever be drawn within a single panel (like the Shiny brush), you’ll want to reset the lasso if the user creates a new waypoint in a new panel
Aaron Lun (12:46:14): > Yes.
Aaron Lun (12:46:20): > Just put that to the side for now.
Kevin Rue-Albrecht (12:46:35): > sure, what’s on your mind for now, then?
Kevin Rue-Albrecht (12:47:55): > @Kevin Rue-Albrechtuploaded a file:Getting there - File (PNG): Getting there
Aaron Lun (12:48:58): > Storing faceting info during calculation of the rectangular boxes is… not easy.
Kevin Rue-Albrecht (12:52:30): > hm.. I forgot (or never really knew) how you computed them the first time. Isn’t there some dplyr magic that could use the faceting info to stratify the computation?
Aaron Lun (12:53:36): > Well the problem is that it has to (i) adapt to either, both or neither FacetRow and FacetColumn being available, (ii) return FacetRow and FacetColumn in the frequency summaries.
Kevin Rue-Albrecht (12:54:48): > (i) sounds the most painful part [especially for NSE like dplyr], but i thought functions likeaggregatecould address (ii)
Kevin Rue-Albrecht (12:56:05): > that said, I still haven’t read as much as I’d like about the dplyr and co.
Aaron Lun (12:56:07): > well, it’s not just that because the code still needs to behave for violin plots
Kevin Rue-Albrecht (13:02:19): > Sure. I don’t mean to say that it’s all easy. I spent the best part of the afternoon adding a ‘simple’ shapeBy feature
Kevin Rue-Albrecht (13:16:10): > thought I was done: still thescale_shape_*instruction to add:sob:
Kevin Rue-Albrecht (13:18:42): > uh-oh… “scale_shape maps discrete variables to six easily discernible shapes. If you have more than six levels, you will get a warning message, and the seventh and subsequence levels will not appear on the plot.”
Kevin Rue-Albrecht (13:18:59): > meh.. “implement first, fix later”
Aaron Lun (13:28:23): > Setup is done. You’ll need to figure out how to usesummary.datato facet the rectangles; but the information should be available in the data.frame. Also,facetByRowColDatais a pretty weird name. Suggest usefacetRowsByColDataandfacetColumnsByColDataandfacetRowsbyRowDataandfacetColumnsByColData.
Kevin Rue-Albrecht (13:29:07): > indeed, current naming createdfacetByRowRowData, too
Aaron Lun (13:30:30) (in thread): > Yes. I was just going to tell you to do that.
Aaron Lun (13:30:45) (in thread): > 2) is not an issue.
Kevin Rue-Albrecht (13:31:53) (in thread): > I spotted some JS of yours doing that kind of hiding magic. Just thought I’d make sure everyone’s happy with the current layout (I think it’s my 3rd implementation of UI controlling faceting)
Kevin Rue-Albrecht (13:33:03): > Last thing I need to do for the shapeBy is to set theguidesname. Tracking it down now
Kevin Rue-Albrecht (14:07:49): > Alright, I’m done with the shapeBy feature (branchshape, slightly out of date, will fix soon)
Kevin Rue-Albrecht (14:08:44): > Moving on to the renaming we just discussed, to standardise the faceting structure
Kevin Rue-Albrecht (14:17:50): > Renaming done.
Kevin Rue-Albrecht (14:30:19): > @Aaron Lun: are you sure I’ve gotanythingto do withsummary.data? the jitter and boxes look fine to me, e.g. for allen: (Yaxis = primary.type, Xaxis = passes_qc_checks, facetRow = core.type, facetColumn = driver_1_s)
Kevin Rue-Albrecht (14:30:50): > @Kevin Rue-Albrechtuploaded a file:e.g. - File (PNG): e.g.
Kevin Rue-Albrecht (14:32:16): > Meanwhile, I just noticed that the axis ticks are somehow messed up upon zooming on faceted square plots (faceted violin plots are fine)
Kevin Rue-Albrecht (14:32:37): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-06-10 19.32.27.png - File (PNG): Screenshot 2018-06-10 19.32.27.png
Kevin Rue-Albrecht (14:52:44): > fixing up the Shiny brush on faceted plots, i had the X/Y aesthetics messed up for either/both facets
Kevin Rue-Albrecht (15:01:41): > I’ve mergedfacetintoshape, as the latter depended on changes in the former anyway. > I suggest we work directly onshape, as I tested it and it looks stable
Aaron Lun (15:18:51) (in thread): > Yes.
Aaron Lun (15:19:22) (in thread): > One of the faceting directions works fine and the other doesn’t.
Aaron Lun (15:26:11) (in thread): > Allen data, Column data plot, Y-axis = core.type, X-axis = driver_1_s, Turn on both facet by row and column with driver_1_s in the rows and columns.
Kevin Rue-Albrecht (16:07:39): > Alright, pausing it here
Kevin Rue-Albrecht (16:09:50) (in thread): > Mind giving it another try? I think I fixed that as part of the last few commits
Kevin Rue-Albrecht (16:29:24) (in thread): > (as far as I can see it’s fine)
Aaron Lun (16:47:29) (in thread): > Nope. Are you sure we’re looking at the same thing here?
Aaron Lun (16:48:12): > @Aaron Lunuploaded a file:image.png - File (PNG): image.png
Unknown User (16:49:20): > A file was commented on
Kevin Rue-Albrecht (16:54:31): > ahhh i’m blind sorry
Unknown User (17:08:45): > A file was commented on
Unknown User (17:10:49): > A file was commented on
Kevin Rue-Albrecht (18:25:15): > But I can’t get my head around the jittering functions to fix it
2018-06-11
Aaron Lun (04:27:10): > Fixed. Seems like the faceting now works as-is.
Kevin Rue-Albrecht (04:50:10): > Yeah - ggplot2 is magic like that. The single facet command applies to all layers that have the aesthetics in their plotting data frame . Thanks!
Kevin Rue-Albrecht (04:50:45): > Which branch did you push too?
Kevin Rue-Albrecht (04:51:55): > Despite what I said I kept committing facet related stuff to facet, and then re merged it to shape, which is the « derived » branch
Kevin Rue-Albrecht (04:52:51): > I.e shape « contains » facet
Aaron Lun (04:53:02): > Well, too late for that.
Aaron Lun (04:53:18): > I just heard you say that we should be working onshape.
Kevin Rue-Albrecht (04:53:41): > No problem. That works
Kevin Rue-Albrecht (04:54:44): > It’s the most up to date one and it is stable, so let’s do that for good now
Aaron Lun (06:30:25): > Where is the faceting information stored in the click object?
Kevin Rue-Albrecht (06:31:04): > Good question, I can’t remember the last time I handled a Shiny click object
Kevin Rue-Albrecht (06:31:37): > I could only have told you for the Shiny brushes
Kevin Rue-Albrecht (06:40:33): > actually without running/checking any code, this post suggests that it’s the same as for a Shiny brushhttps://stackoverflow.com/questions/41654801/r-shiny-plot-click-with-geom-bar-and-facetsi.e.,input$plot1_click$panelvar1 - Attachment (Stack Overflow): R Shiny Plot Click with geom bar and facets > I would like to create an interactive plot with shiny and ggplot2. I have been able to do this successfully with geom_point and other geoms with an obvious x and y axis. When using something like a
Kevin Rue-Albrecht (06:41:47): > I’ll accompany that with a word of warning, that I already discretly mentioned on the project board: > 1) when faceting is done in a single direction, that faceting information goes inpanelvar1no matter whether it’s row of column faceting
Kevin Rue-Albrecht (06:43:06): > 2) when faceting is done in both directions (rowandcolumn),panelvar1stores thecolumnfaceting (andpanelvar2the row faceting)
Kevin Rue-Albrecht (06:43:41): > see my hack here:https://github.com/csoneson/iSEE/blob/facet/R/plotting.R#L1830 - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Aaron Lun (06:50:37): > It’s actually easier - just retrieve the identity of the panelvar frommapping.
Kevin Rue-Albrecht (06:54:43): > well, that’s for the covariate name, the level also needs to be retrieved
Aaron Lun (06:56:00): > Yes, so you can get them both from the shiny object.
Aaron Lun (06:56:09): > The lasso will also store both levels of information.
Kevin Rue-Albrecht (06:58:27): > Cool. So long as the logic mimics a Shiny brush (with our attribute system rather than their list of values), it should be straightforward to update the lasso drawing code similarly to what I’ve done for the brush
Aaron Lun (06:58:49): > Some of the names will change, but yes.
Kevin Rue-Albrecht (07:00:00): > No problem. Note that I might only get to it properly on Wednesday, got slides to prepare first
Aaron Lun (08:38:30): > Lasso infrastructure is done, just make it pretty.
Aaron Lun (08:38:41): > Note that, with faceting, the tolerance for the lasso close is brutal
Kevin Rue-Albrecht (08:38:42): > Ok cool.
Aaron Lun (08:38:56): > Probably need to think in absolute terms
Aaron Lun (08:38:58): > i.e., 1 mm
Aaron Lun (08:39:05): > but I’m not sure how to do that.
Kevin Rue-Albrecht (08:40:08): > I was just going to ask you: can you send a screenshot of the faceting on square plot that you mentioned as an example? I’m pulling and re-building right now to check, but earlier this morning still looked odd to me (some boxes looked still too large for their point content)
Aaron Lun (08:40:34): > huh?
Kevin Rue-Albrecht (08:40:35): > This one - Attachment: Attachment > Allen data, Column data plot, Y-axis = core.type, X-axis = driver_1_s, Turn on both facet by row and column with driver_1_s in the rows and columns.
Aaron Lun (08:40:58): > See my image from before, 9:48 yesterday
Kevin Rue-Albrecht (08:41:43): > yep, that’s when it was still broken
Kevin Rue-Albrecht (08:41:50): > hang on, let me try now agian
Kevin Rue-Albrecht (08:42:47): > ok nevermind that example works now
Kevin Rue-Albrecht (08:44:03): > @Kevin Rue-Albrechtuploaded a file:facet column onprimary.type - File (PNG): facet column on primary.type
Aaron Lun (08:44:32): > Remember boxes are always relative within each facet.
Kevin Rue-Albrecht (08:44:42): > ah. ok.
Kevin Rue-Albrecht (08:45:16): > i suppose absolute would make everything look fairly small
Aaron Lun (08:45:32): > Yes, that was the idea.
Aaron Lun (08:45:37): > And with violin plots we don’t have a choice.
Aaron Lun (08:46:41): > Because ggplot decides what is good for us.
Kevin Rue-Albrecht (08:48:32): > better to be consistent with ggplot, indeed
Kevin Rue-Albrecht (08:49:37): > plus, now that i’m reminded of the ‘within facet’ behaviour, it does make a lot of sense like this
2018-06-12
Kevin Rue-Albrecht (04:06:13): > @Kevin Rue-Albrechtuploaded a file:our package is ‘bomb’ ;) - File (PNG): our package is ‘bomb’ ;)
Kevin Rue-Albrecht (04:10:03): > let’s keep blowing minds:exploding_head:^^
Aaron Lun (04:16:00): > lol
Federico Marini (04:22:07): > iSEE_boom
Aaron Lun (04:37:50): > @Federico MariniI’d like some pretty toggles to replace some of the visual parameter checkboxes we have now.
Aaron Lun (04:38:06): > Also can you think of a better colour scheme for our panels?
Aaron Lun (04:38:20): > Our first four panel types dominate the primary colours and we’ve got nothing left.
Federico Marini (04:39:47): > We’re not stuck anymore with the colours from shiny, right?
Federico Marini (04:40:04): > you had the custom_col in brownish if I recall correctly
Aaron Lun (04:40:37): > Yes, and it’s a crap colour.
Federico Marini (04:40:41): > I’ll look around in any case
Federico Marini (04:40:44): > literally:stuck_out_tongue:
Aaron Lun (04:40:52): > So we need to find more evenly distributed colours for a variable number of plot types
Aaron Lun (04:41:05): > currently we have 7 panel types?
Aaron Lun (04:41:09): > Maybe 8
Aaron Lun (04:41:13): > if we get epivizr working
Federico Marini (04:41:35): > out of the blue, I have in mind thepastel1palette (I think fromscales)
Aaron Lun (04:41:45): > Also, I’d like to change the shinydashboard colour from blue to something a bit more subdued.
Aaron Lun (04:42:03): > Because the blue in the sidebar and header runs up against the blue in the reduced dim panel.
Federico Marini (04:43:45): > anything from this one?
Federico Marini (04:43:48): > https://github.com/federicomarini/haR - Attachment (GitHub): federicomarini/haR > haR - A Palette Generator Inspired by Haribo Gummy Candies
Aaron Lun (04:45:23): > god you have too much time on your hands
Federico Marini (04:45:36): > :stuck_out_tongue:
Federico Marini (04:45:43): > jokes apart
Federico Marini (04:45:48): > this is pretty comprehensive
Aaron Lun (04:45:50): > I guess I like baerchenpaerchen
Federico Marini (04:45:51): > https://github.com/EmilHvitfeldt/r-color-palettes
Aaron Lun (04:45:57): > Um.
Federico Marini (04:46:10): > wrong copypaste
Aaron Lun (04:46:17): > Well I’ll leave the final choice to your discretion.
Aaron Lun (04:46:30): > Just fiddle with the colours inpanel_colours.Rand see what looks nice.
Federico Marini (04:46:57): > at best some discrete ones, I guess
Federico Marini (04:47:18): > just let me know if there are some plots which you connect to a particular colour
Aaron Lun (04:47:39): > Not really; the key is to avoid blue if we can’t change the shinydashboard defaults.
Federico Marini (04:48:33): > got it
Aaron Lun (09:39:27): > Does anyone have a link to that discussion about linking warnings in the docs?
Kevin Rue-Albrecht (09:40:51): > what discussion?
Kevin Rue-Albrecht (09:41:33): > wrong channel?
Aaron Lun (09:42:49): > On the mailing list.
Aaron Lun (09:43:05): > Remember something about insanity when we have to link to the documentation files in\link[]
Aaron Lun (09:43:25): > I remember talking about it here last, but I can’t find the link anymore.
Kevin Rue-Albrecht (09:44:06): > hm don’t remember the discussion, but are you referring to the usage of```/
Aaron Lun (09:44:32): > yes
Aaron Lun (09:44:53): > Something to do with the windows machines not being happy.
Aaron Lun (09:45:16): > duringCHECK
Kevin Rue-Albrecht (09:45:33): > found the bioc-devel email, I think, I’ll forward to you
Kevin Rue-Albrecht (09:45:49): > wehi address, right?
Aaron Lun (09:46:38): > just put it here
Aaron Lun (09:46:47): > or, as in, just give me the title
Aaron Lun (09:46:52): > and I’ll look up the thread online
Kevin Rue-Albrecht (09:47:58): > https://stat.ethz.ch/pipermail/bioc-devel/2018-April/013279.html
Aaron Lun (09:50:10): > thx
Kevin Rue-Albrecht (09:50:44): > No worries. Taught me how to use the online archive again ^^
Kevin Rue-Albrecht (16:06:20): > <!channel>here goes: I think my job with lasso display update is done. Test it break it, let me know. > I tried to update some of the docs where applicable, but once we open the PR we might use the associated git diff to check out areas of code that require new doc or unit tests before merging
2018-06-13
Aaron Lun (04:08:34): > > Listening on[http://127.0.0.1:1234](http://127.0.0.1:1234)***** stack smashing detected *****: /home/cri.camres.org/lun01/Software/R/devel/bin/exec/R terminated > Aborted (core dumped) >
Aaron Lun (04:08:36): > Lol
Aaron Lun (04:12:10): > Probably a firefox thing. Tried to open youtube and iSEE at the same time.
Aaron Lun (04:13:25): > @Kevin Rue-Albrechtlasso works for me.
Aaron Lun (04:13:31): > just a minor bug fix for violin faceting.
Kevin Rue-Albrecht (04:40:45): > ISEE_hulk: smaaaaaah:right-facing_fist:
Kevin Rue-Albrecht (04:41:46): > (Despite all the political correctness, they still don’t offer a green fist..)
Kevin Rue-Albrecht (04:42:40): > What about the faceted violins?
Kevin Rue-Albrecht (04:43:19): > I may not have tried those
Aaron Lun (05:14:46): > Just a bug fix to make the jittering facet-aware.
Aaron Lun (05:14:58): > Same bug as for the square plots earlier, I fixed it then but didn’t propagate it to the violins.
Kevin Rue-Albrecht (05:16:42): > Right. I pulled and tested. Seems to work fine. Again, I’ll admit that I didnt spot the issue before you mentionned it (faceting allen by core.type and driver_1_s, with driver on x axis) maybe just a combination of params that didn’t really highlight the issue. Anyway, thanks for fixing.
Kevin Rue-Albrecht (05:17:07): > We should be able to progressively close issues and project board at this point.
Aaron Lun (05:46:12): > Do it.
Aaron Lun (05:46:15): > FINISH HIM
Aaron Lun (05:46:20): > :-1:
Kevin Rue-Albrecht (05:46:56): > good old omnislash
Federico Marini (05:47:42): > Kev please do the fatality:stuck_out_tongue:
Federico Marini (05:48:40): > @Aaron Lunand everyone: I am busy with other things but still thinking about the potential new color palette.
Federico Marini (05:48:56): > How important would it be to somehow stick to the bioc colors?
Federico Marini (05:49:28): > (or at least, try to pick two in the palette)
Aaron Lun (05:49:44): > Not at all.
Aaron Lun (05:50:22): > If we are having Bioc colours somewhere, they would go on the header/sidebar, but I don’t feel any particular need for that.
Federico Marini (05:50:55): > I’ve heard that someone is working on an adminLTE-based version of the dashboard
Federico Marini (05:51:07): > whether that could give us some more flexibility, dunno yet
Kevin Rue-Albrecht (05:51:48): > yeah - not sure if I picture Fede’s idea well, but I’d see Bioc as a ‘background’ app theme, while the panel colors should stand out of that background, not ‘disappear’ in it
Kevin Rue-Albrecht (05:52:37): > holly molly, that kind of thing?https://adminlte.io/themes/AdminLTE/index2.html
Aaron Lun (05:58:44): > It’s okay, I guess.
Kevin Rue-Albrecht (05:59:13): > Bit overwhelming, if you ask me. But again, everything is until one gets used to it.
Federico Marini (06:37:12): > overwhelming because overfull
Federico Marini (06:37:27): > still, it seems the colours are not so super-customizable
Federico Marini (06:37:42): > I’ll check further
2018-06-14
Charlotte Soneson (03:39:31): > I was looking at some custom shiny apps the other day and it got me thinking-we don’t really have a way of plotting, e.g., two samples against each other in terms of their expression values (right?). Maybe not so useful for single cell, but might be for bulk, to check correlation between replicates etc. In other words, we have row data, column data and row assay plots, but no column assay plot. One could move the assay values to the rowData, but that does not seem like the best solution…:slightly_smiling_face:
Federico Marini (03:40:39): > We don’t have it, you’re right
Kevin Rue-Albrecht (04:30:57): > Yep. Shouldn’t be difficult either, as it’s going to behave very much like a rowDataPlot, but with X/Y axes linked to sample names, and just an assay to select as a general data param.
Charlotte Soneson (04:35:55): > Yes, I guess it should be pretty straightforward to mix the rowDataPlot and featureAssayPlot. If we also have a custom row data plot, it would maybe also allow people to plot more complex associations between (groups of) samples
Kevin Rue-Albrecht (04:37:20): > It’s probably over-the-top, but I’ll mention alsoGGally::ggpairsthat I’ve used in the past:https://www.bioconductor.org/packages/release/bioc/vignettes/TVTB/inst/doc/Introduction.html#8_pairwise_comparison_of_info_data_between_phenotype_levels
Kevin Rue-Albrecht (04:38:27): > That could allow a multiple selection of samples. > However, I’m just realizing the mess that could result of facets and brushes in there
Kevin Rue-Albrecht (04:41:27): > Anyway, we could open up an issue to centralize the discussion about it. > First things first, I need to get through a journal club and give my lab meeting today. Then, I’ll clean up thefacetanddockerbranches tonight, to clear the air. That’ll give us a better viewpoint to plan future developments
Kevin Rue-Albrecht (05:08:56): > @Kevin Rue-Albrechtuploaded a file:ta-da! - File (PNG): ta-da!
Kevin Rue-Albrecht (05:09:46): > awesome work all! > and thanks@Aaron Lunfor handling the submission and chasing down F1000
Charlotte Soneson (05:11:38): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-06-14 at 11.11.03.png - File (PNG): Screen Shot 2018-06-14 at 11.11.03.png
Charlotte Soneson (05:11:51): > Even on the slow wifi in our institute retreat it works!
Kevin Rue-Albrecht (05:12:41): > hahaha.. I’ll be honest: when I just tried to open the first one for the first time right now, I was welcome by a “Disconnected from server” message xD
Kevin Rue-Albrecht (05:14:34): > Dammit, I always forget to tag you and Fede on my iSEE tweets. I’m really more of a reader than a tweeter:sweat_smile:
Charlotte Soneson (05:14:56): > :slightly_smiling_face:
Kevin Rue-Albrecht (05:20:06): > I hope the Marioni group is getting that Alienware sometime soon, with the traffic that’s coming their way:stuck_out_tongue:
Federico Marini (05:28:17): > FYI, the links are not correct:stuck_out_tongue:
Federico Marini (05:28:30): > plate -based links to TCGA
Kevin Rue-Albrecht (05:30:58): > ..because TCGA’s the best:innocent:
Kevin Rue-Albrecht (05:31:53): > … probably the worst of the 4, considering that I couldn’t really illustrate the point they were making in the paper ^^
Kevin Rue-Albrecht (05:32:20): > ah.. and TCGA points to emptyDrops
Kevin Rue-Albrecht (05:34:30): > OK, so summary: > * plate-based –> TCGA > * emptyDrops [OK] > * TCGA –> emptyDrops > * Mass Cyto [OK]
Kevin Rue-Albrecht (05:36:06): > We need our corresponding author@Aaron Lun(although I’m sure anyone could contact F1000 to poke them, I’m sure we gave them the correct links).
Aaron Lun (06:02:31): > on it
Aaron Lun (10:45:27): > did it get fixed
Aaron Lun (10:45:28): > ?
Kevin Rue-Albrecht (11:40:39): > having a look now
Kevin Rue-Albrecht (11:44:13): > plate-based –> allen [OK] > droplet –> emptyDrops [OK] > TCGA –> TCGA [OK] > MS -> MS [OK]
Aaron Lun (11:52:36): > cool
Kevin Rue-Albrecht (11:53:07): > any idea how many concurrent connections your Shiny server can handle? just curious
Aaron Lun (11:54:40): > ¯*(ツ)*/¯
Aaron Lun (11:54:45): > We’ll find out, won’t we?
Kevin Rue-Albrecht (11:57:56): > Only if you somehow track connections (?) but I don’t think we need to go that far ^^
Kevin Rue-Albrecht (14:24:10): > Found the source of the recent TravisCI failed builds: my.which_groupablefunction didn’t know how to deal with 0-column DataFrames. Fixing now
Kevin Rue-Albrecht (14:24:34): > woohoo.. we have a vignette again
Kevin Rue-Albrecht (14:27:51): > Ow. passing the vignette again unveils a few documentation warnings. I’ll send that in tonight. I’d like to close the docker and shape branches asap
Aaron Lun (14:35:27): > @Charlotte Sonesonis the vignette cytof branch still active? Seems like overkill now that we have it on the article as well.
Kevin Rue-Albrecht (14:39:54): > I’m runningcheckagain now, but I think the only things left to fix are unit tests, one of which related to jitter@Aaron Lun. Others probably due to the new params (facet, shape)
Kevin Rue-Albrecht (14:43:32): > I’ve just pushed, dinner and I’ll be back later for the rest.
Federico Marini (15:11:47): > https://twitter.com/MMMarksman/status/1007255686403842048 - Attachment (twitter): Attachment > I always get a thrill from seeing authors go the extra mile and create really cool interactive figures in thier work. I hope this trend continues to grow. If you want to see if you could use one in your next article, then ping me! https://twitter.com/F1000Research/status/1007252315332415489 - Attachment (twitter): Attachment > Take a tour of the iSEE: Interactive SummarizedExperiment Explorer package complete with interactive figures! https://f1000research.com/articles/7-741/v1 > @KevinRUE67 @CSoneson @FedeBioinfo
> #bioconductor
Federico Marini (15:12:05): > honey all over the face:stuck_out_tongue:
Federico Marini (15:12:34): > Cheers to “not fitting in the BAN format”, peeps!
Kevin Rue-Albrecht (15:54:17): > absolutely, the BAN format would have killed the article in the end!
Kevin Rue-Albrecht (15:55:18): > i’m starting to look at the unit tests failed.
Kevin Rue-Albrecht (17:24:27): > Alright, i’ve done all the ones that I was confident about.@Aaron Lunif you could look at the last 4 please, then we can open the PR: > > ══ testthat results ═════════════════════════════════════════════════════════ > OK: 703 SKIPPED: 0 FAILED: 4 > 1. Error: (unknown) (@test_custom.R#13) > 2. Error: memory setup works correctly (@test_misc.R#97) > 3. Error: initialization of active panels works correctly (@test_misc.R#236) > 4. Error: .report_memory generates code that evaluates to the provided object (@test_report_memory.R#33) >
2018-06-15
Federico Marini (04:29:59): > > Additional functionalities > > Row statistics tables can be augmented with dynamic annotation based on the selected row, linking to online resources such as Ensembl (Zerbino et al., 2018) or Entrez (Coordinators, 2017). For large data sets, >
Federico Marini (04:30:00): > Is there a way to refer to the full name “NCBI Resource Coordinators”
Federico Marini (04:38:33): > tiny tiny thing on the published article:
Kevin Rue-Albrecht (04:42:28): > Isn’t that just about fiddling with the .bib file?
Federico Marini (04:49:34): > probably
Federico Marini (04:49:54): > maybe they can fix it quickly
Kevin Rue-Albrecht (04:50:48): > I’m sure they can, but I guess we want our iSEE2018 repo to match the published article
Kevin Rue-Albrecht (05:33:58): > Btw, TravisCI is currently failing, but it seems to be for a timeout issue related to downloading R-devel, nothing to do with the iSEE code itself: > > $ curl -fLo /tmp/R-devel-$(lsb_release -cs).xz[https://s3.amazonaws.com/rstudio-travis/R-devel-$](https://s3.amazonaws.com/rstudio-travis/R-devel-$)(lsb_release -cs).xz > No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself. > Check the details on how to adjust your build configuration on:[https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received](https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received)The build has been terminated >
Aaron Lun (06:35:25): > @Aaron Lunuploaded a file:image.pngand commented: Do you guys have to pay money every time you make an empty new line or something? - File (PNG): image.png
Aaron Lun (06:38:04): > Also, extractingcolumn_groupablefrom the internal fields is not safe, there is no guarantee that this is available when this function gets called by the user (indirectly via the exportedDefaults()functions). The internal fields should only be used in functions that are internal to theiSEE()function.
Aaron Lun (06:39:09): > You will have to recompute this instead.
Kevin Rue-Albrecht (06:39:21): > :cry:
Aaron Lun (06:40:06): > that’s why it’s a pain to have to distinguish between groupable and non-groupable fields.
Aaron Lun (06:42:12): > Also replace yoursapplycalls withvapply, as recommended by BiocCheck. I’ve already dealt with one bug in.which_groupablethat way.
Aaron Lun (06:55:19) (in thread): > Or check againstNULLfirst.
Kevin Rue-Albrecht (07:07:49): > woops. I likevapplytoo, I just have some residual muscle memory from my early days
Aaron Lun (08:20:17): > Mostly passes CHECK. Addshape_labreferences to internal docs. Fix.add_shape_to_row_plotlinks.
Aaron Lun (08:42:26): > Also, whenever you add new parameters, don’t forget to update the documentation indefaults.R.
Kevin Rue-Albrecht (08:47:13): > Ok. Thanks. I’ll wrap that up asap then
Aaron Lun (08:47:45): > It’s done.
Kevin Rue-Albrecht (08:48:06): > Ah, right. Another thanks then!
Aaron Lun (08:52:31): > Haven’t added theshape_labstuff though, you’ll have to do that.
Kevin Rue-Albrecht (08:55:15): > Ok. Easy enough.
Aaron Lun (08:59:06): > Also, your choices of user-visible facet variable names leaves something to be desired.
Aaron Lun (08:59:14): > “RowFacetRowData” is not interpretable
Aaron Lun (08:59:24): > You should at least have “RowFacetByRowData”
Aaron Lun (09:00:23): > And yourtest_ui.Ris crazy. What happens if shiny changes how they report the tags?
Kevin Rue-Albrecht (09:01:32): > yeah, I wasn’t sure what to test there, I think I can simply make the shiny UI manually in the test and test for identical
Aaron Lun (09:02:20): > I would just check that it runs. Full stop.
Aaron Lun (09:02:40): > Repeating the UI call is silly, we might as well spend more time checking the original call.
Aaron Lun (09:03:58): > Regardless of right or wrong, your faceting calls hasNREADSas the variable
Aaron Lun (09:04:08): > which is clearly forbidden now that we’re only dealling with categoricals.
Kevin Rue-Albrecht (09:04:13): > well, i’ve got a whole lot of plotting tests that basically only run the code, but don’t check the result for sanity. I thought I’d start to be stricter for the UI
Kevin Rue-Albrecht (09:04:22): > woops
Aaron Lun (09:04:33): > Well, it breaks anyway so pull down and check it.
Aaron Lun (09:04:46): > I fixed the other internal info issue.
Kevin Rue-Albrecht (09:05:05): > it breaks? i had it fixed last night. Sure you pulled everything again?
Aaron Lun (09:05:23): > It probably breaks because of the internal info check for categorical variables.
Kevin Rue-Albrecht (09:05:30): > ow ok
Kevin Rue-Albrecht (09:06:21): > anyway, just keep pinging me the things as you spot them, and I’ll pass through when you’re done
Aaron Lun (09:17:15): > I’m effectively done. Note the change to the user-visible faceting field name.
Aaron Lun (09:18:33): > I’ll come back later tonight to start migrating all of the plotting code to use the new command managers.
Kevin Rue-Albrecht (09:31:03): > Ok. Head down in some debugging over here. So i’ll need to have a look only later as well.
Aaron Lun (14:39:12): > I’VE HAD ENOUGH.
Aaron Lun (14:39:18): > Dumped a shit load of commits.
Aaron Lun (15:55:50): > @Jayaram Kancherla@hcorradaApologies for the very late response - currently staring at your shiny vignettes. Does the epiviz interface have any capacity to pass information back to the shiny server? It would be nice (magical, perhaps) to be able to select a gene on the genomic track, and have this information relayed to other iSEE panels.
Aaron Lun (15:56:56): > Though if not, that’s okay too, we’ll just use the epiviz tracks as an “end-point” panel that responds to other selections.
hcorrada (21:54:24): > I think@Jayaram Kancherlaimplemented that very recently. He should be able to help with that.
2018-06-16
Kevin Rue-Albrecht (08:16:40): > Hi@Aaron Lun. Thanks for all the cleaning work yesterday. > There’s only one unit test left:.make_customColPlot responds to downsampling (@test_custom.R#159)and therunargument of.evaluate_commands <- function(obj, envir, run=TRUE)is not documented, but doesn’t seem to be used (yet). i.e. it needs documentation or removal
Aaron Lun (08:41:30): > Done.
Kevin Rue-Albrecht (08:41:44): > Awesome. Thanks!
Kevin Rue-Albrecht (08:42:37): > That should restoreR CMD checkin the green, and let codecov give us an update on coverage
Kevin Rue-Albrecht (08:44:35): > I’m thinking about adding a couple of unit test fordynamicUI.R,.panel_generationshould help get a nice boost
Aaron Lun (08:45:59): > Yes. Just execute it with all possibilities, don’t bother checking the output.
Kevin Rue-Albrecht (08:46:44): > Exactly,for (i in seq_len(nrow(active_panels)))will take care of going through all theifs
Kevin Rue-Albrecht (08:47:32): > Anyway, just going to do the groceries and get some air first. Catch up later
Aaron Lun (09:29:08): > I got some leftover wine from last night’s institute social. Getting sloshed NOW.
Aaron Lun (09:29:34): > Can one write unit tests while drunk? The answer is an emphatic YES.
Aaron Lun (09:31:41): > Hm. Tastes like vinegar.
Aaron Lun (09:55:00): > Woah. I can really feel it kicking in.
Aaron Lun (09:55:11): > Two glasses down, two to go.
Aaron Lun (10:00:15): > Okay, I’ll finish the other two tomorrow, I can’t walk straight anymore.
Kevin Rue-Albrecht (10:36:36): > there’s only one way to write unit tests without pain: having little blood in one’s alcohol:stuck_out_tongue:
Aaron Lun (10:43:26): > Man, coding while slightly drunk is crazy
Aaron Lun (10:43:29): > my head feels so warm
Kevin Rue-Albrecht (10:51:25): > working late at night, the lack of sleep can have a similar effect… without the nice taste of wine ^^
Aaron Lun (10:54:37): > PR
Aaron Lun (10:54:38): > PR
Aaron Lun (10:54:40): > PR
Aaron Lun (10:54:40): > PR
Kevin Rue-Albrecht (10:58:28): > you have no idea how long it takes to write a meaningful PR message between making crepes (~1min cooking each side)
Kevin Rue-Albrecht (10:58:32): > incoming
Kevin Rue-Albrecht (11:01:37): > speaking of PR (public relation), Federico and I were wondering yesterday whether we should wait to see if the RNAseq blog picks up iSEE by themselves, or if our corresponding author should contact them to suggest the story (https://www.rna-seqblog.com/contact/) - Attachment (rna-seqblog.com): Contact > Send us an email at admin@rna-seqblog. com
Aaron Lun (11:03:43): > Haven’t heard of it. ¯*(ツ)*/¯
Kevin Rue-Albrecht (11:05:35): > they’re doing a decent job of tracking nice stories, including scRNA-seq:https://www.rna-seqblog.com/transcriptional-benchmarking-of-in-vitro-cells-to-in-vivo-with-single-cell-rna-seq/ - Attachment (rna-seqblog.com): Transcriptional benchmarking of in vitro cells to in vivo with single-cell rna-seq > Single-cell genomic methods now provide unprecedented resolution for characterizing the component cell types and states of tissues such as the epithelial subsets of the gastrointestinal tract. Nevertheless, functional studies of these subsets at scale require faithful in vitro models of identified i
Aaron Lun (12:25:35): > A bit more sober now, so I’ll do my reviewing.
Kevin Rue-Albrecht (12:26:28): > TravisCI has some random failures that seem to be related to the download of TexLive. Interferes with the PR CI check:confused:
Aaron Lun (12:27:06): > Hm. There was a pass a few commits ago.
Kevin Rue-Albrecht (12:27:27): > Speaking of review: I’ve copied chunks of ‘setup code’ fromtest_table_linksfortest_ui. We might want to refactor that more centrally
Aaron Lun (12:28:01): > Probably not.
Kevin Rue-Albrecht (12:28:06): > You had comments “# Do NOT move to setup; re-defined here to keep tests self-contained.“.
Aaron Lun (12:28:11): > Yes.
Aaron Lun (12:28:32): > Any internal structures should be defined locally.
Aaron Lun (12:28:45): > setup_sce … sets up the SCE.
Aaron Lun (12:29:03): > Otherwise it makes it difficult to define the memory for specific tests.
Aaron Lun (12:30:18): > because you’d have to thnk about its effects on other tests
Kevin Rue-Albrecht (12:30:28): > Absolutely. I just wanted to mention the redundancy before moving on and forgetting about those tests (until the day they fail)
Aaron Lun (12:34:01): > I just realized - for the paper I’m currently reviewing, I also reviewed the paper of their competitors. I wonder why they think I’m a suitable reviewer for this subfield.
Aaron Lun (12:34:11): > It’s not even computational!
Kevin Rue-Albrecht (12:36:25): > the ransom of success I guess
Aaron Lun (12:38:24): > I’m having a tough time finding flaws in this paper.
Kevin Rue-Albrecht (12:38:25): > which reminds me: I’m genuinely curious which “key revisions are required to address specific details and make the paper fully scientifically sound” in our case
Aaron Lun (12:38:28): > Either it’s good or I’m still drunk.
Aaron Lun (12:38:37): > Huh?
Aaron Lun (12:38:51): > Did a review come back already?
Kevin Rue-Albrecht (12:38:57): > no no
Kevin Rue-Albrecht (12:40:25): > I was trying to keep the message short: what i mean is that I looked at the possibilities of responses, and the only reasons why we wouldn’t get the tick is the quote that I copy pasted above
Kevin Rue-Albrecht (12:42:24): > (the possibilities are detailed under the article, clicking on the question mark beside “Current Referee Status: AWAITING PEER REVIEW”)
Kevin Rue-Albrecht (12:43:18): > That said, it’s be fun to see the article reviewed similarly to our package submission to BioC ^^
Aaron Lun (12:45:05): > Reviewers be reviewers.
Aaron Lun (12:45:38): > I think I chose a fairly reasonable bunch, though who knows. One of them might be all like, “you should have faceting”
Aaron Lun (12:45:44): > and we’d be like, “we done that, bitch”
Aaron Lun (12:46:57): > Or same for shape
Aaron Lun (12:47:06): > or same for subset PCs
Aaron Lun (12:47:19): > If they want genomic tracks, we got that too.
Aaron Lun (12:47:37): > or will be getting that.
Kevin Rue-Albrecht (12:47:45): > lol I thought exactly that: the only “key revision” I can imagine would be feature requests
Aaron Lun (12:48:52): > unless someone is like, ” I don’t believe in t-SNE plots.”
Aaron Lun (12:49:04): > Oh, BTW, you should turn down the transparency in the TCGA plot
Aaron Lun (12:49:16): > or specifically turn up the transparency, it’s hard to distinguish between selected and non-selected points.
Kevin Rue-Albrecht (12:49:29): > right, I see what you mean
Kevin Rue-Albrecht (12:50:29): > I think that’s also why I had turned ‘up’ the downsampling to 100: the overplotting was defeating the transparency
Kevin Rue-Albrecht (12:56:52): > PR is all green. Any last word? > We could keep throwing in unit tests, updating the NEWS, or perhaps more importantly: bump up the version number and release on devel
Aaron Lun (12:58:13): > The last thing
Aaron Lun (12:58:15): > DO IT
Aaron Lun (12:58:18): > FINISH HIM
Aaron Lun (12:58:21): > :-1:
Aaron Lun (12:58:46): > Oh wait - diversity training kicking in
Aaron Lun (12:58:48): > FINISH HER
Kevin Rue-Albrecht (13:02:29): > 15min drumrolls while Travis decides whether bumping up the version number may blow up R CMD check …:sleepy:
Aaron Lun (13:05:21): > While we wait; you used WGCNA before?
Kevin Rue-Albrecht (13:06:18): > yeah, a long time ago, with moderate success at the time, but it’s got some useful features
Aaron Lun (13:06:43): > Did you use it on DE genes, or on everything?
Kevin Rue-Albrecht (13:07:08): > it was everything, but i have to point out that it was metabolomics data
Aaron Lun (13:07:23): > hm.
Kevin Rue-Albrecht (13:07:38): > obviously, everythingdetectable(above a decent threshold)
Kevin Rue-Albrecht (13:08:49): > i don’t remember it as the friendliest programming API /documentation
Kevin Rue-Albrecht (13:09:05): > but the tutorial gives the bases
Kevin Rue-Albrecht (13:17:04): > god those CGAT pipelines take me more time to debug than to write new ones -_-
Aaron Lun (13:17:50): > Yeah mike was also mentioning that.
Aaron Lun (13:18:01): > Probably someone’s fulltime job
Kevin Rue-Albrecht (13:19:02): > more like a team, but yeah, I’m only aware of one full time guy maintaining them (Sebastian Luna-Valero, with others joining in as they find the time
Kevin Rue-Albrecht (13:28:36): > we have just officially merged +3,436 −1,866 lines of diff tomaster
Aaron Lun (13:28:48): > Beast
Kevin Rue-Albrecht (13:29:28): > Last time I checkedcat ... | wc -l, the R directory was ~10k lines
Aaron Lun (13:29:45): > That’s getting pretty big.
Kevin Rue-Albrecht (13:29:51): > > $ cat R/* | wc -l > 9838 >
Kevin Rue-Albrecht (13:30:48): > (although that includes probable half of comments and docstrings)
Aaron Lun (13:31:07): > Well, that’s code too.
Kevin Rue-Albrecht (13:32:13): > still… I’m fairly sure that at this point there must be a bunch of helper functions that could spin out as an auxilliary package. Remember that templating package at EuroBioc2017?
Aaron Lun (13:32:23): > nope
Kevin Rue-Albrecht (13:33:01): > it wasn’t particularly mindblowing - I remember thinking it was a nice exercise in style that I would never use
Aaron Lun (13:33:16): > I don’t remember it.
Kevin Rue-Albrecht (13:33:17): > I think it was two women representing some company
Aaron Lun (13:33:43): > nup.
Aaron Lun (13:33:52): > probably not a good thing for an organizer to say
Kevin Rue-Albrecht (13:35:21): > anyway that’s not the point - I’m just thinking that iSEE has two main facets: a bunch of functions that generate plotting code, and the Shiny UI/server that evaluates those chunks of code
Kevin Rue-Albrecht (13:36:01): > kind of an iWrite/iSEE duality
Kevin Rue-Albrecht (13:36:56): > the iWrite doesn’t necessarily have to feed uniquely into the Shiny app - the chunks of code could be written to an Rmarkdown script
Kevin Rue-Albrecht (13:37:36): > chain that to a pdflatex call, and you have a PDF ready to submit:stuck_out_tongue:
Kevin Rue-Albrecht (13:38:50): > hell you can add a button to Shiny app “Export as Rmd/PDF”
Kevin Rue-Albrecht (13:41:08): > again I’m drifting, but still, point is if iSEE itself is getting too large, there may be ways to extract some functionalities
Aaron Lun (13:45:48): > ¯*(ツ)*/¯
Aaron Lun (13:45:52): > Don’t think that’s really necessary here.
Aaron Lun (13:46:03): > I mean, there are real monster packages out there.
Aaron Lun (13:46:24): > stringi is a beast, for example.
Aaron Lun (13:46:42): > okay, off to have a dinner
Aaron Lun (13:46:51): > A COLD LONELY DINNER FROM THE CO-OP’s MEAL DEAL.
Kevin Rue-Albrecht (13:47:33): > can’t be the scientific advisory board restaurant every day ^^
Kevin Rue-Albrecht (13:48:02): > I dug up some notes from EuroBioc: > Kirsten Van Hoorde, OpenAnalytics > R ‘template’ package > GitHub: openanalytics/useR2017_templatePackageExample
Kevin Rue-Albrecht (13:48:50): > but again, I wouldn’t spend much time looking that up again. I’ve got iSEE to write my code now:stuck_out_tongue:
Aaron Lun (15:18:03): > listening to whitney’s greatest hits
Kevin Rue-Albrecht (17:19:11): > Just watched the butterfly effect (2004), damn it’s a pretty well made storyline, I liked the twists
Jayaram Kancherla (17:49:08) (in thread): > @Aaron Lunthat is possible. I made this change on a different branch but will push this to the github branch soon.
2018-06-17
Aaron Lun (06:31:52): > Are we sync’d with devel?
Kevin Rue-Albrecht (06:32:06): > Yup, last night.
Kevin Rue-Albrecht (06:32:49): > I can still never get my head around what time (local, UK) is the official deadline to make it in the next build
Aaron Lun (06:33:46): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (06:47:36): > well, latest build > > Snapshot Date: 2018-06-14 16:46:08 -0400 (Thu, 14 Jun 2018) > This page was generated on 2018-06-15 10:44:18 -0400 (Fri, 15 Jun 2018). > > And today, my laptop terminal tells me: > > $ date -R > Sun, 17 Jun 2018 11:45:07 +0100 > > So basically, aside from the absence of build updates since Friday, I’m counting a 5h time difference with a snapshot at ~17h BioC time, which means ~22h UK time
Kevin Rue-Albrecht (06:49:18): > I pushed upstream way before 22h yesterday, so if the build system is up and running we should see something happening by tomorrow at the latest
Charlotte Soneson (15:26:23): > Just pushed first attempt of sample assay plots to thecolumnassaybranch, if anyone wants to try it out.
Aaron Lun (15:27:27): > I’m pretty drunk right now, so I will do so later.
Charlotte Soneson (15:27:42): > Two glasses today as well?
Aaron Lun (15:28:36): > yep.
Aaron Lun (15:28:38): > It’s finally over
Aaron Lun (15:28:52): > I got through my drunken review
Charlotte Soneson (15:28:54): > It’s a hard job
Charlotte Soneson (15:29:27): > Also, I was wondering: is there a reason for the swap betweenXandYin the selectize updators (iniSEE-main.R): > > for (axis in c("xaxis", "yaxis")) { > if (axis=="xaxis") { > axis_name_choice <- .featAssayYAxisFeatName > } else { > axis_name_choice <- .featAssayXAxisFeatName > } >
Charlotte Soneson (15:29:43): > I mean,featAssayYAxisFeatNameifaxis=="xaxis"
Aaron Lun (15:30:00): > where is this?
Aaron Lun (15:30:20): > oh right
Aaron Lun (15:30:22): > hm.
Aaron Lun (15:31:09): > seems like it’s the wrong way around - but it doesn’t matter, asaxisnever gets used for anything anyway.
Charlotte Soneson (15:31:28): > Yeah, it doesn’t make a difference to swap them, I tried:slightly_smiling_face:
Aaron Lun (15:31:29): > you could replace that withaxis in c(.featAssayYAxisFeatName, .featAssayXAxisFeatName)
Aaron Lun (15:31:34): > might as well.
Aaron Lun (15:31:47): > must have been a vestige of a bigger thing that was progressively removed.
Aaron Lun (15:32:15): > If you can do that, I’m staggering home to cook some dinner
Charlotte Soneson (15:33:02): > Ok
2018-06-19
Federico Marini (03:49:46): > Could it be too much for us?https://dragular.zstat.pl/ - Attachment (dragular.zstat.pl): Drag and Drop Elements in ‘Shiny’ using ‘Dragula Javascript Library’ > Move elements between containers in ‘Shiny’ without explicitly using ‘JavaScript’. It can be used to build custom inputs or to change the positions of user interface elements like plots or tables.
Aaron Lun (04:06:45): > Could be useful for the panel organization sidebar.
Aaron Lun (04:06:53): > As long as we get a record of its new position.
Aaron Lun (04:07:21): > Anyway, can someone update the package CITATION?
Charlotte Soneson (05:39:46): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-06-19 at 11.39.03.pngand commented: Seems we have our first review:slightly_smiling_face: - File (PNG): Screen Shot 2018-06-19 at 11.39.03.png
Federico Marini (06:25:30): > I’ve always been a fan of james:smile:
Federico Marini (06:25:49): > but hey, it is a very fair and nice review
Kevin Rue-Albrecht (06:59:37): > Oh I like James! Awesome annotations workshop at the Boston BioC 2017
Kevin Rue-Albrecht (06:59:54): > I’ll read the review now
Kevin Rue-Albrecht (07:08:38): > Alright, really good review. > Still, that may be the moment to draw the ace up our sleeve and demonstrate howdockercan avoid the need for “R, iSEE, and all its dependencies installed,” or even “that you have a server running all necessary packages that you can point the colleague to”.
Kevin Rue-Albrecht (07:10:18): > e.g. > > docker pull kevinrue/isee_allen > docker run --rm -i -p 1234:1234 kevinrue/isee_allen >
Kevin Rue-Albrecht (07:47:59): > Sure it moves the requirement to havingdockerinstalled on the computer of the collaborator, but that becomes theonlyrequirement
Federico Marini (07:55:30): > once the conda recipe would be available, this is also another option
Kevin Rue-Albrecht (07:57:25): > well conda moves thecodearound, or can it also package a data set?
Aaron Lun (07:57:30): > Do what you like, as long as you remember to add irrelevant crap to .Rbuildignore.
Aaron Lun (07:57:47): > I hope that the build scripts are not too large.
Federico Marini (08:01:19): > the conda recipe is completely separated
Federico Marini (08:01:24): > so no worry abt that
Kevin Rue-Albrecht (08:02:05): > I’ll push an example build scripts on a side branch of iSEE2018 this evening if you like. > But basically, the Dockerfile is tiny, and the R script that launches the Shiny app is 99% identical to our currentapp.R, with the only different ofrunApp(..., host = "0.0.0.0")to make the port visible for the host machine
Federico Marini (08:02:46): > If it is not too much, we could also have a small script to setup the shiny server
Aaron Lun (08:03:48): > Does it really require any set-up? If you must, write a vignette about it.
Federico Marini (08:05:37): > … or that
Aaron Lun (10:20:08): > Do we need to raise any issues for James’ review?
Kevin Rue-Albrecht (10:28:55): > The only thing that I gathered from our discussion would be more documentation about ways of distributing ‘packaged’ instances of iSEE (i.e., that include a preconfigured data set and/or tour) to collaborators
Aaron Lun (10:29:16): > I haven’t actually read the review yet, been busy with other things
Aaron Lun (10:29:23): > But basically we don’t have to do anything?
Kevin Rue-Albrecht (10:29:48): > Nope.
Kevin Rue-Albrecht (10:30:41): > Aside from all the pats in the back that he gave us, his review is pretty much “The only negative for this package is that, being based on the Shiny framework, to allow a colleague to explore the data requires that the colleague either have R, iSEE, and all its dependencies installed, or that you have a server running all necessary packages that you can point the colleague to. This limits sharing with people who are not R savvy, but is a function of how Shiny works, rather than the iSEE package.”
Aaron Lun (10:30:41): > good.
Aaron Lun (10:31:00): > Well, the docker and conda things should address that.
Kevin Rue-Albrecht (10:31:24): > Exactly. Our discussion above probably makes more sense to you in light of this comment ^^
Aaron Lun (10:33:12): > Though I wonder whether docker is any easier to install than R.
Federico Marini (10:35:53): > On windows there was this kitematic which made it pretty easy to install also the images
Kevin Rue-Albrecht (10:35:56): > on mac it was pretty much drag and drop
Aaron Lun (10:36:09): > okay.
Federico Marini (10:36:18): > the advantage i see about that would be that it is guaranteed to work
Kevin Rue-Albrecht (10:36:28): > the only trickery I had to dig in the docs for was to run the TCGA tour
Federico Marini (10:36:31): > no worry of whatevs other existing installation giving troubles
Kevin Rue-Albrecht (10:36:47): > the default memory allocated to docker by default is 2gb
Kevin Rue-Albrecht (10:37:32): > pretty much the same situation as our experineceshinyapps.io(instant crash)
Kevin Rue-Albrecht (10:40:03): > for the record on the subject, the docs are a bit misleading: some places suggest that memory can be increased at the command linedocker run ... --memory=10gbut that didn’t work for me. > Instaed I had to go in “Docker > Preferences > Advanced” and set thesliderInputto ~4-6 GiB
Kevin Rue-Albrecht (10:41:23): > apart from that it was a straight shot with the dockerpullandruncommands that I gave above
Kevin Rue-Albrecht (14:46:12): > it’s ugly, but I’ve pushed my current ‘docker strategy’ to share a preconfigured app
Aaron Lun (14:46:44): > Is this to iSEE2018?
Kevin Rue-Albrecht (14:46:56): > yes, bad idea?
Aaron Lun (14:47:00): > no, it’s fine.
Kevin Rue-Albrecht (14:47:10): > as I wrote earlier, it’s basically redundant ofapp.R, withshiny::runApp(app, port = 1234, host = "0.0.0.0")that’s different
Kevin Rue-Albrecht (14:48:17): > there may be a smarter way that I haven’t taken the time to discover yet. > I just googled now:https://docs.docker.com/v17.09/engine/userguide/networking/default_network/binding/“You can see that Docker has exposed these container ports on 0.0.0.0, the wildcard IP address that will match any possible incoming port on the host machine.” - Attachment (Docker Documentation): Bind container ports to the host > The information in this section explains binding container ports within the Docker default bridge. This is a bridge network named bridge created automatically when you install Docker. Note: The Docker…
Aaron Lun (14:48:26): > These are just words to me.
Kevin Rue-Albrecht (14:48:36): > Anyway, gotta go. Dinner’s on the table and I have to finish other stuff tonight
Kevin Rue-Albrecht (14:49:55): > @Davis McCarthymay have some suggestions, he’s the one who suggested docker alongtime ago (see his PR still open, I just explored the idea and came up with a lighter implementation)
2018-06-20
Charlotte Soneson (05:09:01): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-06-20 at 11.08.27.pngand commented::+1: - File (PNG): Screen Shot 2018-06-20 at 11.08.27.png
Kevin Rue-Albrecht (05:36:56): > Reviewer 2: “I think it would be nice to make available a docker image with all the requirements to run iSEE installed. It would promote the use of the tool a lot among bioinformaticians working with non-computational researchers.” - check:ok_hand:
Aaron Lun (05:37:03): > lol
Aaron Lun (05:37:06): > Well, that was easy.
Kevin Rue-Albrecht (05:37:34): > “For the features mentioned like code tracking and additional functionality, it would be nice to have a link to the vignette in the paper so the user can jump into how to get it done.” Not sure what ‘how to get it done’ means here
Kevin Rue-Albrecht (05:38:26): > I mean the code tracking is ‘done’ , and the button is there. > I don’t know if additional functionalities refers to custom plots or extending the app itself
Aaron Lun (05:39:15): > Sure, some of the features could be better documented.
Aaron Lun (05:39:21): > We should have an “advanced vignette” as well.
Aaron Lun (05:39:26): > For serious hardcore shit.
Aaron Lun (05:40:03): > Or perhaps alternative tours - that would also be useful, rather than one giant mega tour.
Kevin Rue-Albrecht (06:38:15): > Alright I’m back - improvised lab meeting in the office ^^
Kevin Rue-Albrecht (06:39:12): > About the last point ‘loading an RDA/RDS file’, that might be possible for small data sets, but I’m not sure whether it’s a desirable/sustainable feature overall
Kevin Rue-Albrecht (06:40:49): > Not sure how shiny servers would cope with multiple users loading data sets of various sizes and shapes
Kevin Rue-Albrecht (06:42:25): > I think the RData files should be a no-no, as those things can contain any number of objects, which can potentially cause some bad interactions with the shiny app (although that would be a ‘blame-the-user’ kind of scenario)
Kevin Rue-Albrecht (06:45:44): > Anyway, might be worth toying with the idea on a branch. Although even if it works, I’d rather keep that feature on a branch, and never merge it tomaster- I think we should always maintain a version of the app thatdoesn’tinclude this ‘load RDS’ button, especially for tours, where the data setmustbe fixed for the session
Aaron Lun (06:46:44): > I think it’s madness. We’ve been over this.
Kevin Rue-Albrecht (06:48:08): > True. Might be worth keeping record of the discussion on a closed issue in the GH though. For ourselves and to redirect users when they ask.
Kevin Rue-Albrecht (07:04:54): > Btw. Two reviews were enough to get us indexed. but we’re expecting more of them, right?
Charlotte Soneson (07:05:39): > I guess we don’t know how many reviewers accepted the invitation (or even how many were invited in the first place)
Aaron Lun (07:29:23): > No, we don’t.
Kevin Rue-Albrecht (07:37:15): > well, package accepted in a day, paper accepted and indexed in less than a week.. I think we’re doing well ^^
Federico Marini (09:37:57): > WE WANT MOAR
Aaron Lun (09:56:24): > Just got some feedback from a course i was involved in: > > The practical by Aaron Lun - this was the most purposeful and hands-on of the practicals, was delivered with flair, and distilled some complicated techniques in a way that was challenging but manageable.
Aaron Lun (09:56:40): > WOAH
Kevin Rue-Albrecht (09:57:11): > When can I attend?:nerd_face:
Aaron Lun (09:57:29): > I usually don’t look at these because of my sensitive heart
Kevin Rue-Albrecht (09:57:54): > jokes aside: congrats!
Aaron Lun (09:58:30): > On another note, did you put up an issue about the Rdata files?
Kevin Rue-Albrecht (09:59:43): > not yet, I may take some time in the evening, to consider more carefully each of the comments. I think there are a few of them that deserve their own issue/why-we-probably-won’t-do-it-discussion.
Kevin Rue-Albrecht (10:01:00): > Also, I’m off to a family wedding on Friday, so I’ve got a few things to wrap up over here as well the next couple of days
Aaron Lun (10:07:47): > WHAT
Aaron Lun (10:07:55): > Not your own, presumably.
Kevin Rue-Albrecht (10:08:19): > a familymember’swedding, I should have written
Kevin Rue-Albrecht (10:10:40): > Got invited to 3 weddings scattered over 4 months this summer.. I guess that’s the not-so-subtle reminder that I’m headed that way too next year ^^
Federico Marini (10:20:08): > @Aaron Lunput it in the CV: “I deliver with flair”:slightly_smiling_face:
Federico Marini (10:20:26): > but it would be also my wish to get something like this
Aaron Lun (10:20:30): > A wedding?
Federico Marini (10:20:37): > a feedback
Aaron Lun (10:20:37): > I thought you were already married
Federico Marini (10:20:55): > no I’m not - little jon snows:stuck_out_tongue:
Federico Marini (10:21:06): > and jana snow, well
Federico Marini (10:21:39): > BTW
Aaron Lun (10:21:50): > :shocked_face_with_exploding_head:
Federico Marini (10:21:58): > I saw a couple of jaws nicely dropping when demoing:rocket:
Aaron Lun (10:22:09): > I presume we’re talking about the app now.
Federico Marini (10:22:28): > yes sir:wink:
Federico Marini (10:22:49): > timing oh timing. was so good, review 1 and 2, bang
Aaron Lun (10:22:54): > Who were you demoing it to?
Federico Marini (10:23:15): > I could enter delirium soon - the room has barely oxygen left
Federico Marini (10:23:46): > local symposium, with both academia peeps and couple of company
Aaron Lun (10:24:32): > hm.
Aaron Lun (10:24:34): > Okay, good.
Federico Marini (10:26:28): > the server was good in responding:slightly_smiling_face:
Kevin Rue-Albrecht (10:27:00): > .. still waiting for that Alienware:stuck_out_tongue:
Kevin Rue-Albrecht (12:52:12): > Did anyone notice the BiocManager email on bioc-devel?
Kevin Rue-Albrecht (12:53:27): > it’s fairly concise and will likely go unnoticed by half the developers, but it looks like it will impact a lot of packages ^^
Kevin Rue-Albrecht (12:53:48): > (including us)
Aaron Lun (12:59:48): > Not really.
Aaron Lun (12:59:54): > I mean, it just changes how we download things.
Aaron Lun (13:00:17): > I don’t even put isntallation instructions in my own vignettes
Aaron Lun (13:00:30): > I mean, there’s instructions on the landing page. so I’m not going to repeat them.
Aaron Lun (13:00:44): > And if they’re looking at the vignette directly, then they’ve probably installed it already.
Kevin Rue-Albrecht (13:02:29): > True. Anyway, I just looked and we did include the installation statement in the vignette. Maybe we can simply remove it and avoid this kind of issue in the future, if they change it again.
Kevin Rue-Albrecht (13:04:13): > Btw, I just updated and pushed the docker containers forallenandtcgaon my dockerhub account, if anyone wants to try. Instructions forallenare in theREADMEon thedockerbranch ofiSEE2018
Kevin Rue-Albrecht (19:22:15): > quick note: I’ve built all 4 tours on containers on my laptop this evening. But the connection is so depressing terrible at home that I can’t even push the Allen data set to DockerHub. I’ll push tomorrow from the university: it takes about 5min for the TCGA…
2018-06-21
Kevin Rue-Albrecht (04:49:07): > Alright, the docker images are all up.https://hub.docker.com/u/kevinrue/If someone is patient enough todocker pull ...anddocker run -i --rm -p 1234:1234 ....them (see README ofhttps://github.com/LTLA/iSEE2018/tree/docker), we could then PR iSEE2018, and mention it in the docs or even a response to reviewers (I suppose we can reply even if their review accepted the package, right?) - Attachment (GitHub): LTLA/iSEE2018 > iSEE2018 - Code and manuscript files for the iSEE paper
Aaron Lun (04:56:55): > Yes, we can.
Aaron Lun (12:59:51): > I’m approaching my goal
Aaron Lun (12:59:54): > My DESTINY
Aaron Lun (12:59:57): > 500 commit month
Aaron Lun (13:00:00): > here i come
Kevin Rue-Albrecht (14:40:23): > I think the PR fordockeris ready oniSEE2018.
Kevin Rue-Albrecht (14:41:11): > Last chance for fixes is tonight, before I disappear until Sunday evening.
Aaron Lun (14:42:19): > uh - is your name on the docker pull command intentional?
Aaron Lun (14:43:21): > and can’t the docker.R just beapp <- source('app.R'); runApp(...)?
Kevin Rue-Albrecht (14:44:00): > well, my name is because I can only build the containers on my account
Aaron Lun (14:44:22): > Okay, so it’s like csoneson/iSEE.
Kevin Rue-Albrecht (14:44:28): > exactly
Kevin Rue-Albrecht (14:44:39): > andsourcews my first try too…
Kevin Rue-Albrecht (14:45:08): > oh wait.. I didn’t assign the result of source… is that even possible?
Aaron Lun (14:47:05): > Should be.
Kevin Rue-Albrecht (14:47:10): > what I tried wassource(app.R), but that does not maintain an interactive session. Instead the container shuts down as soon as the script is sourced. For some reason,runApp()doesn’t persist
Aaron Lun (14:48:52): > Just checked -sourcereturns back a list withvalue, which should be what you last evaluated.
Aaron Lun (14:49:09): > Hell, you could doeval(parse(file=))
Kevin Rue-Albrecht (14:49:12): > holy R
Aaron Lun (14:49:13): > and assign that to something.
Kevin Rue-Albrecht (14:50:53): > :sob:thanks
Kevin Rue-Albrecht (14:52:59): > @Kevin Rue-Albrechtuploaded a file:Another case of … - File (PNG): Another case of …
Aaron Lun (14:54:15): > Something should have started to smell when you started copying whole chunks.
Aaron Lun (14:54:38): > Did that the other day
Aaron Lun (14:54:41): > and stopped.
Aaron Lun (14:54:47): > Not before a few commits tho.
Kevin Rue-Albrecht (14:57:22): > One thing though - thebioconductor/devel_base2and by extensioncsoneson/iseecontainers don’t contains theorg.Hs.eg.dbpackage, which crashes theisee_pbmc4kcontainer if I don’t add an extrabiocLitebefore loading it
Kevin Rue-Albrecht (14:58:19): > But with our new strategy, I might just add that before sourcingapp.Rto produce the same effect. Nevermind
Kevin Rue-Albrecht (16:06:05): > Just to confirm thatisee_allenworks after the updated, i’ll try the other ones to make sure the PR is safe to merge
Kevin Rue-Albrecht (17:34:39): > I’ve tested all of them and they’re good to go. I’ve pushed 3 of them already, and the tcga is in progress but requires to push 1GB and transfer is slow from home. However, the push won’t changed the behaviour of the tcga tour, so the PR on iSEE2018 could be merged anytime now. I’ll leave it open for the weekend to give a chance for everyone to review it, as it’s kind of an important new aspect/feature.
Kevin Rue-Albrecht (18:15:36): > I’ve just added a Word file in our Dropbox with my own reply to each of Lorena’s comments. There are a couple of to-do’s (one for myself)
2018-06-22
Aaron Lun (04:06:27): > None of this word business. Google docs.
Federico Marini (04:43:15): > Naive thing, we can tweet out for the #ShinyAppreciation month abt our app. Anyone against it?
Federico Marini (04:46:01): > (https://twitter.com/hashtag/ShinyAppreciation?src=hash) - Attachment (twitter.com): #ShinyAppreciation hashtag on Twitter > See Tweets about #ShinyAppreciation on Twitter. See what people are saying and join the conversation.
Aaron Lun (04:51:06): > Sounds good to me.
Federico Marini (04:52:46): > Got a catchy compact text for the tweet?
Federico Marini (04:53:04): > I was thinking we can (re)use theiSEEing is believing
Federico Marini (04:58:42): > (plus we can retweet the f1000res one where the gif is already in there)
Aaron Lun (05:07:37): > I don’t think I can come up with anything catchier, so go for it.
Federico Marini (05:08:08): > k
Federico Marini (05:08:48): > BTW Aaron, Y U !(on twitter) Aaron?:stuck_out_tongue:
Aaron Lun (05:14:54): > No.
Federico Marini (05:19:00): > Anyway, tweet’s out ->https://twitter.com/FedeBioinfo/status/1010088781469421568 - Attachment (twitter): Attachment > Joining the party for the #ShinyAppreciation month… > We’ll show you that iSEEing is believing for genomics datasets! > Sparkled from a cooperation with @CSoneson @KevinRUE67 and Aaron Lun, here’s the package: http://bioconductor.org/packages/iSEE/ (manuscript with live apps in the retweet :wink:) https://twitter.com/F1000Research/status/1009103519721426944 - Attachment (twitter): Attachment > Have you taken a tour of the iSEE: Interactive SummarizedExperiment Explorer package yet? The first referee report is up! > https://f1000research.com/articles/7-741/v1 > @KevinRUE67 @CSoneson @FedeBioinfo
> #bioconductor
Aaron Lun (05:21:24): > :+1:
2018-06-23
Aaron Lun (10:11:57): > Behold the aaronverse:https://ltla.github.io/
Aaron Lun (12:26:26): > Dammit got sent two review requests from NAR. At the same time!
Aaron Lun (12:26:40): > What does the editor think I’m doing? Twiddling my thumbs and waiting for review requests?
Aaron Lun (12:27:10): > At least I get to rack up my stats onhttps://publons.com/author/1499079/aaron-t-l-lun#profile - Attachment (publons.com): Aaron T. L. Lun’s profile on Publons > Publons offers you a verified record of your peer review activity. See Aaron T. L. Lun’s profile with 26 reviews.
Aaron Lun (12:29:55): > In the top 9% of reviews - wort wort wort!
Aaron Lun (12:35:12): > Hello?
Aaron Lun (12:35:16): > Is anyone there?
Federico Marini (15:04:40): > Just dropping by, watching the game with ze germans sucking
Aaron Lun (15:10:17): > Yeah, that would be funny
Aaron Lun (15:10:19): > I’m going home now.
2018-06-24
Federico Marini (17:01:37): > http://celltypes.brain-map.org/rnaseq/mouse - Attachment (celltypes.brain-map.org): RNA-Seq Mouse Data Navigator :: Allen Brain Atlas: Cell Types > Allen Cell Types Database
Federico Marini (17:01:44): > &
Federico Marini (17:01:45): > http://celltypes.brain-map.org/rnaseq/human - Attachment (celltypes.brain-map.org): RNA-Seq Human Data Navigator :: Allen Brain Atlas: Cell Types > Allen Cell Types Database
Federico Marini (17:02:03): > open the contest:smile:
Kevin Rue-Albrecht (18:52:02): > Nice keeping an eye on other’s efforts@Federico MariniI just played with it for a few minutes now and I’m not convinced what I’d use it for. > Plus it seems to reset or simply forget the plot from each panel each time it’s left, and doesn’t really help link information between plots (not even mentioning dynamic links). > Catch up tomorrow!
2018-06-25
Federico Marini (03:49:35): > “keep your friends close, and other’s efforts closer:smile:”
Kevin Rue-Albrecht (04:04:21): > It all looks nice and shiny (pun intended) but my overall feeling was similar to most web hosted UI: not enough transparency about what we’re looking at, and not particularly interactive/flexible. Still, it made me think that ISEE apps do not have readme/methods/background pages or modals, that could be easily added through an optional markdown file, parsed and rendered within the app, a bit like the tours.
Federico Marini (04:05:31): > basic-er than basic, you mean, right?
Federico Marini (04:05:50): > kinda describing the plot types, probably
Federico Marini (04:06:14): > this would catch even the beginner-er-ers
Federico Marini (04:06:39): > (or said in a different way, this would give ’em no excuse)
Kevin Rue-Albrecht (04:07:26): > Hang on, just got to work, switching to keyboard and maybe screenshot to illustrate
Kevin Rue-Albrecht (04:13:45): > I mean fairly simple, like the 3-4 headers and paragraphs that are visible in the “Navigator Overview” panel of the two apps that you just shared. Could be something just as generic by default, just with the possibility to overwrite it with something potentially more bespoke to each data set (e.g., brief overview of background and methods to give users a better chance to grasp the nature/context of the data set)
Kevin Rue-Albrecht (04:17:02): > The idea would be to parse a text file in markdown format, and simply render its content either in a modal or a new tab (although we haven’t used the latter yet, and I think we can stay away from it unless modals can’t handle the content properly)
Kevin Rue-Albrecht (04:21:37): > At the moment, we’re kinda using the first few steps of each tour to describe each data set, which itches me for a few reasons: tooltips are fairly small in size (larger ones can sometimes require scrolling as it reaches out of screen), they encourage bite sized information, which isn’t really an optimal format to describe the process from experimental design to library prep method to preprocessing.
Federico Marini (04:23:25): > Overall: yes, we can do some work on that. it is quite “cheap” for us, with potential nice benefits. Plus, the modal popup “starter” can be also discrete, thanks to all nice extensions to shiny (or even just a small button)
Kevin Rue-Albrecht (04:25:47): > Yup. I just rediscovered the size of the first step tooltip for the iSEE allen data set tour (see f1000 interactive figure:wink:) > That’s something that would look a lot more elegant to me in a dedicated modal than a tool tip stretched across all the height of my screen
Aaron Lun (04:28:19): > I don’t know whether an app can be made with a modal pop-up to show up upon initialization.
Kevin Rue-Albrecht (04:28:55): > I didn’t expect the modal to open up on startup.
Aaron Lun (04:29:31): > I’d like it to.
Kevin Rue-Albrecht (04:30:15): > Ok. So it would beeitherthat modalorthe tour then.
Aaron Lun (04:32:01): > I mean, we can set up the model so that it triggers the tour.
Aaron Lun (04:32:19): > I thnk…
Kevin Rue-Albrecht (04:32:25): > I can see the point of opening it as an introduction, to encourage to get familiar with the context of the data set before messing and misinterpreting it. Meanwhile, I imagine it up there under the Documentation icon, so that users can open it again any time
Kevin Rue-Albrecht (04:33:37): > And yes, I’m pretty sure that if we can trigger the tour from a UI button, we can make the modal trigger it too. We’ll just need to make sure that the modal gets closed before it triggers the tour, otherwise users won’t be able to see much of it ^^
Kevin Rue-Albrecht (04:39:34): > Anyone else subscribed to the F1000 tracked article updates? This morning I got an email “Update: Passed peer review | New Referee Report”. But we’re still at two reviews. I suppose it’s a weekly update or something like that.
Aaron Lun (11:16:37): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (11:23:09): > On a separate note: all tours are now up-to-date on my Docker account.https://hub.docker.com/u/kevinrue/I’ll check and merge my PR on iSEE2018 in the evening. > I don’t think it’s worth setting up automated Docker builds that would trigger every time LTLA/iSEE2018 ‘master’ branch is updated. I’d rather build and test them manually when needed, before releasing updated images
Aaron Lun (12:05:01): > ¯*(ツ)*/¯
Aaron Lun (12:05:21): > No one commented on my 500 commit thing
Aaron Lun (12:05:24): > That was fucking hard
Federico Marini (12:06:11): > well I do envy sometimes your commitment
Federico Marini (12:06:22): > sorry for not putting it out in the channel:stuck_out_tongue:
Federico Marini (12:07:13): > (probably it is even making it harder that your sentence with the mohawk stimulated much more in terms of interest, but hey…)
Kevin Rue-Albrecht (12:07:23) (in thread): > pun intended on ‘commitment’ ?
Federico Marini (12:07:48) (in thread): > nowthat you say that, yes:stuck_out_tongue:
Federico Marini (12:07:57) (in thread): > committed to commit
Federico Marini (12:08:10) (in thread): > could be a blog title for OSS contributors
Kevin Rue-Albrecht (14:44:30): > I’m going to merge thedockerbranch from iSEE2018 anytime now. Any last words?
Kevin Rue-Albrecht (14:51:50): > (there are updates to the README file if that may prompt some feedbackhttps://github.com/LTLA/iSEE2018)
Federico Marini (16:42:59): > no red lights from my side, so please proceed - once the others are fine as well
Kevin Rue-Albrecht (16:48:07): > eh… toooo late:sweat_smile:but thanks for checking though. I gave myself a bit of time looking updating the iSEE package citation, but then I felt a signature “FINISH HIM” coming from our commit hero of the month, so I took on some responsibility and made a executive decision:wink:
2018-06-26
Aaron Lun (06:47:53): > lol
Kevin Rue-Albrecht (07:03:01): > @Kevin Rue-Albrechtuploaded a file:GitHubHero.pdf - File (PDF): GitHubHero.pdf
Kevin Rue-Albrecht (07:03:20): > Didn’t have time to do the colors.. sorry:wink:
Federico Marini (08:20:26): > Aaron you owe Kev one on this:smile:
Federico Marini (08:20:44): > A beer inthe O wordat least
Kevin Rue-Albrecht (08:22:14): > That said, I think we’re all waiting for the picture of@Aaron Lunrocking a 4-color keyboard (say, pull-checkout-commit-push) mohawk-style for his next profile picture:innocent:
Federico Marini (08:26:55): > we could make him an alias
Federico Marini (08:27:52): > git yolo
Federico Marini (08:28:08): > which does commit and push to all branches at once
Kevin Rue-Albrecht (08:30:13): > lol
Kevin Rue-Albrecht (08:30:32): > give me one more minute, i got something nice in preparation eheheh
Kevin Rue-Albrecht (08:33:20): > @Kevin Rue-Albrechtuploaded a file:GitHubHero2.pdf - File (PDF): GitHubHero2.pdf
Federico Marini (08:33:45): > BRANCH IT AARON
Kevin Rue-Albrecht (08:33:54): > I am so posting that back to the#randomchannel
Elana Fertig (08:37:30): > @Elana Fertig has joined the channel
Kevin Rue-Albrecht (08:47:23): > @Kevin Rue-Albrechtuploaded a file:GitHubHero3.pdfand commented: Alright, last one I promise. I just couldn’t resist putting the two together in a final version. - File (PDF): GitHubHero3.pdf
Kevin Rue-Albrecht (08:48:42): > (Welcome@Elana Fertig!)
Elana Fertig (08:50:04): > Thanks@Kevin Rue-Albrecht!
Federico Marini (09:52:13): > Hello iSEE-newcomer@Elana Fertig(or maybe just a fan:slightly_smiling_face:)
Elana Fertig (10:15:40): > a fan for now:slightly_smiling_face:
Federico Marini (10:24:32): > oh I am a fan of fans
Elana Fertig (10:32:15): > @Alex Hopkinsis working on a new summarized experiment for TCR-sequencing data — we’re trying to solicit community interest / feedback for his project — not sure if anyone here is interested in such a thing
Elana Fertig (10:32:32): > but we’d hate to develop a datastructure that’s just based on our use
Alex Hopkins (10:32:37): > @Alex Hopkins has joined the channel
Aaron Lun (12:28:24): > We should probably start working on the epiviz thing.
Kevin Rue-Albrecht (12:39:59): > Yup, On a completely different note, I’ve pushed acitationbranch to iSEE yesterday. The weird format of the F1000 citation forced me to use thetextVersion=argument ofbibentry(). > Feel free to fiddle with it before we open a PR
Kevin Rue-Albrecht (12:42:33): > Btw, not sure why Travis fails withError in loadNamespace(name) : there is no package called 'devtools'now. > Probably something that will fix itself
Kevin Rue-Albrecht (12:43:19): > I know Aaron: /shrug
Kevin Rue-Albrecht (12:43:50): > owwww it has to be a message on its own? poor thing
Aaron Lun (12:48:23): > I honestly wouldn’t worry about the crap that F1000 asks you to put in the citation. I never have.
2018-06-27
Charlotte Soneson (05:06:05): > @Charlotte Sonesonuploaded a file:there seems to be an issue with the TCGA app: - File (PNG): there seems to be an issue with the TCGA app:
Aaron Lun (05:16:30): > Does it occur locally?
Aaron Lun (05:16:42): > Might be just an oddity of the shiny server
Aaron Lun (05:16:45): > sometimes it happens
Charlotte Soneson (05:16:55): > Didn’t check yet
Charlotte Soneson (05:30:59): > running locally actually seems to work
Kevin Rue-Albrecht (05:33:26): > Did it happen only with the TCGA app so far?
Charlotte Soneson (05:33:50): > Think so. It was Lukas who told me, he had tried the one in the F1000 paper
Aaron Lun (05:33:51): > Maybe a server config thing - any thoughts@Federico Marini
Federico Marini (05:34:40): > logs are tricky to access in this, sometimes they are automatically gone
Federico Marini (05:35:03): > AFAIK there’S no much you can change re: such errors
Federico Marini (05:35:20): > does it happen again if you just relaunch it? (sorry for the dumb suggestion)
Kevin Rue-Albrecht (05:35:37): > https://github.com/rstudio/shiny-server/issues/267 - Attachment (GitHub): Shiny server does not work with my app, which is working in local · Issue #267 · rstudio/shiny-server > I have installed Shiny server on AWS Ubuntu. The default test page works both for rmarkdown and shiny server. However, if I upload a shiny app, which is working in local, it does not work on the sh…
Kevin Rue-Albrecht (05:36:36): > I suspect we’re probably going to find the same error in our log as posted there: ‘Job for shiny-server.service failed because a configured resource limit was exceeded. See “systemctl status shiny-server.service” and “journalctl -xe” for details.’
Aaron Lun (05:37:14): > The last log had: > > Error: cannot allocate vector of size 1.5 Gb > Execution halted >
Aaron Lun (05:37:48): > Not sure why this happens now, it was running fine before.
Charlotte Soneson (05:39:02): > @Federico Marini: Lukas had the problem, and I saw the same thing. Haven’t tried to run it directly from the CRUK server (only via the F1000 paper)
Kevin Rue-Albrecht (05:39:14): > Hm.. I’ve seen that before in a different context (velocyto.R using too large set of gene annotations). Maybe Luka picked a combination of covariates that creates a unusually large object during processing?
Kevin Rue-Albrecht (05:39:34): > Also:https://shiny.rstudio.com/articles/sanitize-errors.html
Kevin Rue-Albrecht (05:39:58): > ” To sanitize errors everywhere in your app, just addoptions(shiny.sanitize.errors = TRUE)somewhere in your app. Then, all error messages will be replaced with the following generic error message”Error: An error has occurred. Check your logs or contact the app author for clarification.
Charlotte Soneson (05:39:58): > It’s just the regular TCGA tour, from the paper
Aaron Lun (05:41:29): > I don’t even see how you can generate a 1.5 Gb vector at any point.
Aaron Lun (05:41:36): > Must be the heatmap.
Charlotte Soneson (05:42:18): > thesce.rdsobject is 1.15Gb
Charlotte Soneson (05:42:44): > at least when I generate it locally using thedata.Rscript
Kevin Rue-Albrecht (05:45:34): > I’m wondering if there were simply too many concurrent users, and that the system couldn’t allocate that memory to Luka’s session. How much memory is there on the node running the Shiny server?
Aaron Lun (05:47:46): > Yes, the object itself, but if the error’s occuring halfway through then the object should be loaded. There shouldn’t be a 1.5 GB vector creation at any point.
Aaron Lun (05:48:00): > node memory, I’ll have to ask. ¯*(ツ)*/¯
Kevin Rue-Albrecht (05:50:31): > The other thing that we should probably know by know is the approximate memory usage for a single instance of an app. And a rough prediction from the size of the SCE object. That’d help users and ourselves define the requirements for running an app.
Kevin Rue-Albrecht (05:51:32): > On top of the SCE object, the app stores the coordinates for each plot, I think even in duplicate, as they are also stored in the ggplot object.
Aaron Lun (05:52:27): > These should be super lite. 1 million points, 2 coordinates, 8 bytes each; that’s only 16 MB.
Aaron Lun (05:52:52): > Throw in other bits and pieces, it should still be less than 1 GB total.
Aaron Lun (05:52:56): > for all plots.
Kevin Rue-Albrecht (05:53:03): > From the experience of building Docker images for the tour, I can already tell you that the default 2 GB allowance for the Docker daemon were not enough to run the TCGA tour in a container.
Aaron Lun (05:53:26): > WEll shit
Aaron Lun (05:53:34): > then it’s probably ggplot’s fault
Kevin Rue-Albrecht (05:53:43): > I didn’t benchmark values to find the sweetspot, but it worked again when I gave 4 GB to the Docker daemon
Kevin Rue-Albrecht (05:56:05): > If anyone’s got $10k / year to spend, I believe that Shiny server Pro includes memory usage tracking:wink:
Kevin Rue-Albrecht (05:56:53): > Not sure how easy that is to usingps, if at all possible on a per-app-instance level
Charlotte Soneson (09:15:56): > btw, another question from Lukas was how difficult it would be to add contour lines on top of plots, indicating the density. I think it should be reasonably straightforward, but I don’t think I have tried it with ggplot2 before.
Aaron Lun (09:18:22): > It would only make sense for scatter plots.
Charlotte Soneson (09:18:28): > yes
Charlotte Soneson (09:18:41): > he was thinking about the “gating plots”
Aaron Lun (09:21:00): > Possible, as a visual parameter. Feel free to try implementing it if you like. Note that the UI itself does not currently respond to whether a variable is discrete/variable; this means that a contour on/off switch would do nothing but still trigger replotting for other plot types, as the same observers are being used.
Kevin Rue-Albrecht (09:21:08): > contours as isgeom_rug? or actual density lines?
Charlotte Soneson (09:21:43): > I think actual density lines
Charlotte Soneson (09:25:49): > I’m missing something - why would the main plot be smaller?
Charlotte Soneson (09:26:27): > @Charlotte Sonesonuploaded a file:Screen Shot 2018-06-27 at 15.22.08.pngand commented: I think he means something like this - File (PNG): Screen Shot 2018-06-27 at 15.22.08.png
Aaron Lun (09:52:25): > Hey, is there a point-and-click way for forks to pull from the source? Can’t be bothered explainingfetch.
Kevin Rue-Albrecht (17:31:10): > Nope - i’ve looked at that a few weeks ago. I remember there’s an issue open somewhere for the GH developers to consider it.
Kevin Rue-Albrecht (17:35:03): > I understand it’s for someone else. I suppose you’ve already come across those links that you could send them:https://help.github.com/articles/merging-an-upstream-repository-into-your-fork/https://help.github.com/articles/syncing-a-fork/Although it sounds like your situation is more aboutexplainingthem what those commands are actually doing (?) - Attachment (help.github.com): Merging an upstream repository into your fork - User Documentation > If you don’t have push (write) access to an upstream repository, then you can pull commits from that repository into your own fork. … - Attachment (help.github.com): Syncing a fork - User Documentation > Sync a fork of a repository to keep it up-to-date with the upstream repository. …
2018-06-28
Federico Marini (03:55:19): > new revieeeeeeew:slightly_smiling_face:
Federico Marini (04:06:17): > https://f1000research.com/articles/7-741/v1#referee-response-35044 - Attachment (f1000research.com): F1000Research Article: iSEE: Interactive SummarizedExperiment Explorer. > Read the latest article version by Kevin Rue-Albrecht, Federico Marini, Charlotte Soneson, Aaron T.L. Lun, at F1000Research.
Aaron Lun (04:22:08) (in thread): > That’s right. As if I would need a point and click thing.
Kevin Rue-Albrecht (04:49:50): > Yup. Slack not working last night but I meant to say that he’s suggesting several things we’ve decided to avoid
Kevin Rue-Albrecht (04:51:44): > (Not his fault, he wasn’t there when we discussed it ^^)
Kevin Rue-Albrecht (04:52:24): > Aaron i can’t wait to see « multiple selection in redDimPlot »:yum:
Kevin Rue-Albrecht (04:56:27): > His point 3, we’ve discussed extensively: we’re not going to identify markers and pathways for active selection on the fly.
Kevin Rue-Albrecht (04:56:55): > That said, It’d be fun to benchmark even the simplest of methods for DE on a branch
Aaron Lun (05:03:43): > Certainly I wouldn’t want to impose specific analyses.
Aaron Lun (05:04:04): > However, it may be possible to have a panel that, given a selection, reports a set of statistics via a user-supplied function.
Aaron Lun (05:04:27): > In a similar vein to our currentcustomColFun
Kevin Rue-Albrecht (05:04:37): > Yup. We could stop at markers DE stats in a first instance.
Kevin Rue-Albrecht (05:39:45): > Anyway, point is, we can consider adding a whole bunch of features form user/reviewer requests, but at the end of the day, what’s key is to prioritize and get them done properly one by one, rather than trying to do too much at the same time
Kevin Rue-Albrecht (05:40:39): > I’m saying that because I can imagine a lot more requests coming from enthusiast users, as iSEE gets adopted
Aaron Lun (05:41:24): > Sure.
Kevin Rue-Albrecht (05:44:56): > Speaking of which, I haven’t taken the time yet to test the columnAssayPlot that Charlotte started a couple of weeks back. Might want to wrap up that one before embarking on the epivizr journey.
Aaron Lun (05:47:56): > Yep
Kevin Rue-Albrecht (05:54:34): > Alright, I think I’m done writing my feedback on Lorena’s reviews, in the Google Docs.@Federico MariniI don’t have a GMail for you so I could only send you an invitation. > I’ll try to add the other two reviewers this evening.
Federico Marini (05:54:58): > fede.maro@gmail.com
Aaron Lun (05:58:05): > I am knee-deep in one of my collaborator’s crap, so will be inactive for the rest of the week.
Federico Marini (09:14:10): > (on the enemy front, there’s this as well -https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/bty534/5046258)
Federico Marini (09:14:34): > although I gave it a first look, and the learning curve on their UI is damn steeper as for iSEE
Federico Marini (09:14:55): > (and I don’t want to mention the lesser functionality:stuck_out_tongue:)
Kevin Rue-Albrecht (09:15:24): > meh.. they only got it intoBioinformatics
Aaron Lun (09:16:01): > I gave up on the “upload data” button.
Kevin Rue-Albrecht (09:16:18): > you tried it in the end?
Federico Marini (09:17:02): > briefly, without finding out in 10 minutes what the hell I could do with the 50 buttons and options
Federico Marini (09:17:40): > and I guess this is a strong barrier VS adoption
Federico Marini (09:17:57): > I loved Saskia’s tweet about iSEE where she said “it just damn works”
Aaron Lun (09:18:10): > fuckin’ A
Federico Marini (09:18:30): > huh?
Aaron Lun (09:18:48): > “I agree.”
Federico Marini (09:20:02): > ohyeah
Kevin Rue-Albrecht (09:20:17): > what the hell:http://137.132.97.109/VTBox/newExperimentWizard.jsp?txtnewexperiment=test×tamp=1530191986138#
Kevin Rue-Albrecht (09:20:30): > does that download the user manual for you?
Federico Marini (09:20:40): > We need to plan something meaningful for us to work on until next december:smile:
Kevin Rue-Albrecht (09:21:15): > @Kevin Rue-Albrechtuploaded a file:image.png - File (PNG): image.png
Kevin Rue-Albrecht (09:21:30): > Sorry, here goes:https://github.com/soumitag/SLIDE/raw/master/application/slide/SLIDE_Users_Manual.pdf
Federico Marini (09:21:41): > I actually have an idea. Small topic, big application usage. GOOD OLD DAMN VENN DIAGRAMS
Federico Marini (09:21:52): > with members of set shown on hovering
Federico Marini (09:21:57): > and downloadable if wished
Aaron Lun (09:22:03): > Dunno what you mean.
Aaron Lun (09:22:11): > Epivizr was going to be the next on the to-do list
Federico Marini (09:22:33): > so that I stop playing with the attributes field of the output fromgplots::venn
Kevin Rue-Albrecht (09:22:46): > the only thing I see in their manual page above is a bunch of blue arrows that look like a RNA hairpin
Federico Marini (09:22:49): > sorry Aaron, it was more like a side rant/side project
Federico Marini (09:23:20): > (although it can something meaningful, but nothin’ to do with iSEE)
Kevin Rue-Albrecht (09:25:52): > SLIDE, 123 commits, by a single developer, started 3 Aug 2017 > iSEE: 1414 commits by 4 developers.. started Dec 2017
Kevin Rue-Albrecht (09:26:53): > Tbh, I don’t think it’s going to be long lived, from looking at the pattern of commits
Federico Marini (09:27:52): > bursts and then nothing?
Kevin Rue-Albrecht (09:28:12): > 4-commit bursts, yeah
Kevin Rue-Albrecht (09:28:38): > Novice-level of GitHub Hero:wink:
Kevin Rue-Albrecht (09:29:29): > Everyone’s gotta start somewhere, but it’s also the first GH repo of the guy
Kevin Rue-Albrecht (09:29:56): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-06-28 14.29.44.png - File (PNG): Screenshot 2018-06-28 14.29.44.png
Federico Marini (09:30:39): > you can tell when the reviews came in:stuck_out_tongue:
Federico Marini (09:31:21): > and i did not check the commit messages:stuck_out_tongue:
Kevin Rue-Albrecht (09:32:13): > But seriously, there must be something aboutBioinformatics‘s scope that I don’t get. What’s about SLIDE that got it in?
Aaron Lun (09:45:49): > We never even tried, though.
Aaron Lun (09:46:04): > We tried GB and they knocked us back.
Kevin Rue-Albrecht (09:47:21): > True. Still, the BAN format is so restrictive. And I can’t imagine OB (O. Bioinformatics) going the extra mile setting up live demos for us
Kevin Rue-Albrecht (09:47:29): > (or for anyone, that said)
Federico Marini (09:47:51): > no way, OB would hjave never done it
Federico Marini (09:48:15): > the BAN format would have just penalized how we depict our work
Federico Marini (09:48:33): > what we can still do is to enhance the resonance, via the RNA-seq blog that Kevin mentioned
Aaron Lun (09:50:23): > If you write a blurb, I can look at it.
Kevin Rue-Albrecht (09:50:29): > About that. They don’t seem to have picked iSEE on their blogging radar yet, so we could email them indeed. Just be careful what you write them even in your first email, as they quoted me from it last time I contacted them:sweat_smile:
Aaron Lun (09:51:05): > Well, if you write a draft email I (or you) can send it to them.
Kevin Rue-Albrecht (09:52:15): > Sure. We can set up another Google Docs writing exercise
Aaron Lun (13:14:39): > I was thinking of saying in my email “iSEE is the alpha and the omega of interactive data visualization.”
Federico Marini (13:18:14): > well nothing forbids us to use more colourful language
Federico Marini (13:18:22): > :stuck_out_tongue:
Federico Marini (13:19:24): > they will count publication ante iSEE and post iSEE
Federico Marini (13:19:27): > muhuhahaha
Federico Marini (13:19:56): > god I hate coop partners and their shitty time management
Federico Marini (13:20:27): > how they handle deadlines before grant submission is~a mistery~an abomination
Federico Marini (13:21:08): > even better the ones that tell “i’ll handle this part” and then put a “Fede bitte ergänzen” as a placeholder
Federico Marini (13:21:14): > RAAAAAAANT
Kevin Rue-Albrecht (14:32:40): > https://www.rna-seqblog.com/exploring-the-single-cell-rna-seq-analysis-landscape-with-the-scrna-tools-database/ - Attachment (rna-seqblog.com): Exploring the single-cell RNA-seq analysis landscape with the scRNA-tools database > As single-cell RNA-sequencing (scRNA-seq) datasets have become more widespread the number of tools designed to analyse these data has dramatically…
Aaron Lun (14:35:57): > If we can write papers on that, I should write “A festschrift for Aaron Lun: exploring the AaronVerse, one package at a time”.
Aaron Lun (14:36:33): > For those of you who missed it,ltla.github.io
Kevin Rue-Albrecht (14:40:10): > Just to illustrate my earlier point:https://www.rna-seqblog.com/goexpress-visualize-rna-seq-and-microarray-data-using-gene-ontology-annotations/I can’t wait for“iSEE is the alpha and the omega of interactive data visualization.“, say Aaron Lun one of the tool’s developers.^^ - Attachment (rna-seqblog.com): GOexpress - visualize RNA-Seq and microarray data using gene ontology annotations > GOexpress accepts gene expression datasets obtained from both microarray and RNA-seq platforms formatted in the recommended Bioconductor “ExpressionSet” container, to evaluate the power of each feature expressed in the dataset to cluster biological samples according to known experimental factors. In
Kevin Rue-Albrecht (15:24:28): > Anyone remembers having trouble with Travis not findingdevtoolsat some point? > Last few builds on branchcitationfailed because of that. > I added- Rscript -e 'install.packages("devtools")'in.travis.ymlto get around it. Still: > 1) it’s weird that it suddenly shows up > 2) “The default rules roughly amount to: install: > - R -e ’devtools::install_deps(dep = T)’” (https://docs.travis-ci.com/user/languages/r/) > 3) there has been a small change in the central Travis build script for R projects, but it doesn’t seem relevant to us:https://github.com/travis-ci/travis-build/commits/master/lib/travis/build/script/r.rb4) we’ve been caching packages in our.travis.ymlconfig - Attachment (GitHub): travis-ci/travis-build > travis-build - .travis.yml => build.sh converter
Kevin Rue-Albrecht (16:09:34): > Alright, nevermind, sonething messed up about the package cache of our Travis builds. Now that I added the installation of devtools, even the older failed builds succeed again, thanks to the updated cache.
Kevin Rue-Albrecht (16:56:30): > Just looking back at that last PR (about a screenshot in the README). I was going to close it saying “please refer to Figure 1 of the article”, but then it also made me realise that our README file could do with an update. It’s not particularly sexy and fairly outdated, to be honest. I’ll sleep on it and think how it could look better
Kevin Rue-Albrecht (16:56:49): > (shortcut for those who don’t want to type:https://github.com/csoneson/iSEE)
Federico Marini (17:00:32): > can we link to the png/jpeg in the online version of the manuscript directly?
Federico Marini (17:00:46): > so that the other PR about the screenshot is gone:stuck_out_tongue:
Kevin Rue-Albrecht (17:01:28): > hehehe you read my mind.. I wanted to do that, but no: the figure only links to a PPTX file
Kevin Rue-Albrecht (17:03:04): > although hang on… I’ve got a nasty one up my sleeve.. curious to see how it looks on GH
Kevin Rue-Albrecht (17:08:25): > ta-daaaaa
Kevin Rue-Albrecht (17:10:42): > For whatever reason, the image doesn’t show in the modal (when you click it in the article). But I just used copied the Image Address from the thumbnail in the article, and it’s actually pretty high-res
Federico Marini (17:13:04): > eheh
Kevin Rue-Albrecht (17:28:56): > Alright. I’m done writing my thoughts on the reviewer comments for tonight. Next step is to update iSEE where applicable (e.g. “add an example on how to change the palette for all categorical”).
Kevin Rue-Albrecht (17:29:37): > Feel free to pick up the Google docs, add your own thoughts and smash my own
2018-06-29
Aaron Lun (08:59:28): > 600, baby!
Aaron Lun (08:59:51): > TO REACH…. THE UNREACHABLE… STAR….
Aaron Lun (09:04:20): > I am… so burnt out.
2018-06-30
Kevin Rue-Albrecht (08:55:00): > Did a few fixes here and there, see reply to reviewers for more info.@Aaron Lunlet me know what you think of the fix inhttps://github.com/csoneson/iSEE/pull/217/filesit addresses a corner case related to Lorena’s 3rd comment, which may still need a ‘friendlier’ error message when users try to access out of range dimensions in the redDim panels - Attachment (GitHub): Valid initialisation of redDim panel UI when first redDimType has only one dimension by kevinrue · Pull Request #217 · csoneson/iSEE
Kevin Rue-Albrecht (08:55:52): > I’m going to head out for the afternoon now
Kevin Rue-Albrecht (08:57:51): > I’m pretty happy with the typically ‘subscript out of bound’ error, that is typical of R when trying to access a column by an index greater than `ncol(x), to be honest
Kevin Rue-Albrecht (08:59:21): > Meanwhile, if users type some character value in the Dimension1/2 UI input, it’ll give them a weird “replacement has 0 rows, data has …“. Still, they just shouldn’t type character in a numeric input
Aaron Lun (08:59:22): > This should have been fixed in devel.
Kevin Rue-Albrecht (08:59:43): > What’s “this” ?
Aaron Lun (09:00:03): > Trying to select a dimension greater than the number available.
Aaron Lun (09:00:28): > updateNumericInputshould also have done the job when the user changes the type of plot.
Aaron Lun (09:00:53): > See lines 1310 onward.
Kevin Rue-Albrecht (09:03:15): > when they change the type of plot, it does indeed. My corner case was just about the initialization
Aaron Lun (09:03:33): > okay.
Aaron Lun (09:03:45): > They shouldn’t be even able to enter some character value.
Aaron Lun (09:03:49): > It’s aNumericInput!
Kevin Rue-Albrecht (09:03:52): > you’ll see in the PR, redDimDefault wasn’t checking the `se
Aaron Lun (09:04:00): > yeah yeah
Kevin Rue-Albrecht (09:04:12): > i know… but i typedein it anyway, and it lets me
Kevin Rue-Albrecht (09:04:38): > it seems like a ridiculous oversight from Shiny
Aaron Lun (09:04:54): > ¯*(ツ)*/¯
Aaron Lun (09:05:14): > The thing with error messages is that we can’t leave the app in a false state.
Kevin Rue-Albrecht (09:05:22): > right
Aaron Lun (09:05:27): > Any error message must be accompanied by a reset to a valid value.
2018-07-02
Charlotte Soneson (05:28:02): > FYI, I’m giving a talk at the “New Directions in Single Cell Analysis” in Heidelberg in a couple of months (http://www.sound-biomed.eu/2018/02/28/final-consortium-meeting/), and I was planning to spend part of it showingiSEE(under the header “reproducible exploration”)
Kevin Rue-Albrecht (05:29:09): > Sounds cool!
Kevin Rue-Albrecht (05:31:53): > Btw, this morning I got a “tracked article alert” about “Swimming downstream […]” on F1000. First: congrats! Then, I don’t remember subscribing. I suppose I must have subscribed and forgotten… I don’t see any iSEE mentioned in there that could explain an alert
Charlotte Soneson (05:35:58): > Thanks, this one is mostly the work of by Mike and Rob though:slightly_smiling_face:I don’t know why you would get an alert:thinking_face:maybe F1000 is starting a recommender system:stuck_out_tongue_winking_eye:or maybe you are tracking the Bioconductor channel?
Federico Marini (05:38:23): > zere must be some deep learning behind zat:smile:
Kevin Rue-Albrecht (05:44:15): > I already have a recommender system for articles to read: it’s called a PI:stuck_out_tongue:
Kevin Rue-Albrecht (05:44:35): > “Probably Interesting”
Federico Marini (05:47:41): > I miss that someone doing this for me:smile:
Federico Marini (05:48:04): > I get a lot of nice ones more for life-science-content
Federico Marini (05:48:23): > but the new techniques of the cool kids, I have to dig’em out
Kevin Rue-Albrecht (05:50:10): > Took me time, but I’m starting to like the ‘1 paper a week for journal club’. Helps keep a rhythm and forces me and colleagues to keep an eye out
Aaron Lun (09:54:42): > I’ve been doing one review a week, so I think that counts.
Federico Marini (10:25:52) (in thread): > Huh, was not in my radar
Federico Marini (10:26:11) (in thread): > but it is around the corner for me, plus it is even free, plus we could catch up
Federico Marini (10:26:16) (in thread): > I see lots of wins
Charlotte Soneson (10:26:29) (in thread): > Sounds great:slightly_smiling_face:
Federico Marini (10:26:53) (in thread): > (pun intended?)
Charlotte Soneson (10:27:00) (in thread): > Ehm…no:slightly_smiling_face:
Charlotte Soneson (10:27:05) (in thread): > I don’t think that far
Federico Marini (10:27:24) (in thread): > :stuck_out_tongue:
Federico Marini (10:31:37) (in thread): > registration simply via email i guess?
Federico Marini (10:32:11) (in thread): > (and if wolfgang organizes, it’s going to be simone in charge)
Kevin Rue-Albrecht (10:32:11): > counts double, imho
Charlotte Soneson (10:32:24) (in thread): > I have really no idea:slightly_smiling_face:they contacted me and asked if I wanted to come and speak…
Charlotte Soneson (10:32:48) (in thread): > But I have indeed been in contact with Simone for the organization.
Charlotte Soneson (10:33:18) (in thread): > There is a registration link just above the programme
Charlotte Soneson (10:33:33) (in thread): > Almost invisible, but “register” is clickable
Kevin Rue-Albrecht (10:34:13): > reading, understanding, spotting the issues (and good stuff) AND reporting it (non negligible last bit)
Aaron Lun (10:34:20): > love racking up those reviewer credits on publons
Aaron Lun (10:34:31): > Wrote a 3000 word magnum opus of a review on Saturday.
Kevin Rue-Albrecht (10:34:47): > can you spend them on journal subscription (those reviewer credits)? ^^
Federico Marini (10:35:20) (in thread): > ah
Federico Marini (10:35:24) (in thread): > good catch
Federico Marini (10:35:34) (in thread): > her email is anyway on the bottom of the page
Federico Marini (10:36:39) (in thread): > jeez. lowest contrast ever for a pair of colors:smile:
Charlotte Soneson (10:36:43) (in thread): > makes sense that she doesn’t want all the registrations individually scattered in her inbox though…
Charlotte Soneson (10:36:57) (in thread): > yeah, made by someone with good color vision
Federico Marini (10:39:33) (in thread): > even if i’m not into sound itself i think it could be interesting to attend to the consortium meeting
Federico Marini (10:40:03) (in thread): > need to see how this fits with the kinderbetreuung (plus, I should be in osnabrück the week after that- i feel fury coming)
Charlotte Soneson (10:40:05) (in thread): > yes, I think it can be interesting. I know other people outside of SOUND who are going too
Federico Marini (10:42:21) (in thread): > ok, I registered:slightly_smiling_face:
Charlotte Soneson (10:42:33) (in thread): > awesome! then I’ll see you there:slightly_smiling_face:
Federico Marini (10:42:39) (in thread): > lets see if it is not sold out
Federico Marini (10:43:03) (in thread): > .. and I’ll seeiSEE, muhuhahaha:smile:
Charlotte Soneson (10:43:14) (in thread): > ah beh, then they have to move it to a bigger auditorium and order some more food:nerd_face:
2018-07-03
Aaron Lun (08:42:31): > Editing the reply to reviewers NOW.
Aaron Lun (08:50:10): > Uh - what’s the role of the Dockerfile iniSEE, versus the images on Dockerhub?
Kevin Rue-Albrecht (08:50:57): > the Dockerfile defines the instructions thatdocker build ...applies to make the image
Aaron Lun (08:51:23): > So the images in iSEE2018 use the DockerFile in iSEE?
Kevin Rue-Albrecht (08:51:33): > I’ve added a (very brief) description in the README ofhttps://github.com/LTLA/iSEE2018 - Attachment (GitHub): LTLA/iSEE2018 > iSEE2018 - Code and manuscript files for the iSEE paper
Kevin Rue-Albrecht (08:51:55): > hang on
Kevin Rue-Albrecht (08:52:21): > the Dockerfile iniSEEis used to make thecsoneson/iseeimage (call it ‘base iSEE’)
Kevin Rue-Albrecht (08:53:04): > then each tour on iSEE2018 has its ownextraDockerfile, that builds a tour-specific imageon topof the ‘base iSEE’ image
Aaron Lun (08:53:06): > How does it even know to look ininst/exdata/Dockerfile?
Kevin Rue-Albrecht (08:53:55): > we’ve configured that with Charlotte, in the DockerHub build configuration of her account
Aaron Lun (08:54:09): > oh.
Kevin Rue-Albrecht (08:54:09): > Default is to have the Dockerfile at the root of the repository
Aaron Lun (08:54:33): > Probably want to put that in the wiki someplace, in case Charlotte disappears somewhere.
Aaron Lun (08:54:45): > gets hit by a bus, etc.
Aaron Lun (08:57:39): > It occurs to me that all of our academic projects have a bus factor of 1.
Aaron Lun (08:57:51): > i.e., if anyone gets hit by a bus, the project stops.
Kevin Rue-Albrecht (08:58:39): > Ahhh, took me a second. Yup.
Aaron Lun (09:15:43): > Can someone add the message to the start of each tour to address Lorena’s first comment?
Kevin Rue-Albrecht (09:29:21): > I’m adding items to a to do list that I’ll process in the evening, if no one gets to it before I do
Aaron Lun (09:40:22): > Should we switch to a drop-down for the dimension choices?
Aaron Lun (09:40:33): > Given that numeric input is not robust.
Kevin Rue-Albrecht (09:41:42): > Dropdown sounds fair enough to me.
Kevin Rue-Albrecht (09:42:01): > It’s discrete, after all. Dropdown makes probably even more sense than a numeric input
Aaron Lun (10:08:18): > Is everyone happy with the response to James? I will add that first.
Kevin Rue-Albrecht (10:11:34) (in thread): > I’m a little biased about the content, but I’ll say yes anyway:wink:
Aaron Lun (10:12:21): > BTW red text is stuff to-do.
Aaron Lun (10:12:52): > We should split up the vignette into smaller vignettes, one for intro, others for use cases, others for tour set-up.
Aaron Lun (10:12:55): > and so on.
Aaron Lun (10:14:47): > Can I put@Federico Marini@Charlotte Soneson@Kevin Rue-Albrechton the vignette split-up? I have to deal with some shiny observer related matters - DON’T TOUCHiSEE.R.
Charlotte Soneson (10:15:20) (in thread): > Looks fine to me too
Aaron Lun (10:16:00): > ordynamicUI.R.
Aaron Lun (10:20:47) (in thread): > @Federico Mariniget yo ass in here
Federico Marini (10:21:05): > in a meeting, brb:stuck_out_tongue:
Federico Marini (10:47:01) (in thread): > I was discussing recently with a colleague what that risk would be for Hadley Wickham:stuck_out_tongue:
Federico Marini (10:49:19) (in thread): > fine for me as well!
Federico Marini (10:51:04): > most of the stuff in reply to lorena is fine for me too
Aaron Lun (10:52:51) (in thread): > lol
Aaron Lun (11:17:33): > I am happy to say that i have CORRECTED everyone’s two-space indents.
Aaron Lun (11:50:55): > I have also moved some of theimportFromstatements out ofiSEE-pkg.Rand into their correct homes. Please keep this in mind when you’re addingimportFroms - add them to the function in which they are needed.
Federico Marini (11:55:07): > ok, thanks!
Federico Marini (11:55:37): > @Charlotte Soneson: how much you might be into football, I don’t know that much, but location + provenance makes you the perfect audience:stuck_out_tongue:
Charlotte Soneson (11:56:50): > Yeah, I’m on the train trying to make myself invisible:stuck_out_tongue_winking_eye:
Federico Marini (11:57:36): > do they smell you’re swedish?:smile:
Charlotte Soneson (11:58:23): > You never know:smiley:
Charlotte Soneson (11:58:35): > Can never be careful enough:wink:
Aaron Lun (11:58:53) (in thread): > A couple of these are related to your classes@Kevin Rue-Albrecht, which I’ll leave you to clean up.
Aaron Lun (12:02:38): > Response to James has been posted.
Federico Marini (12:03:07): > Cool, thanks mr CA:wink:
Aaron Lun (12:03:22): > That’s Dr CA to you.
Federico Marini (12:03:35): > True that!
Federico Marini (12:04:06): > I was experimenting a little also with circleCI for testing. Not that I have troubles with travis, but also out of curiosity
Aaron Lun (12:04:20): > I don’t know the difference between those things.
Federico Marini (12:04:26): > circleCI seems to be quite powerful, plus it would even do checks on Mac
Aaron Lun (12:05:10): > Don’t they all just run R CMD CHECK?
Kevin Rue-Albrecht (12:05:13) (in thread): > Oh right. So that means you left them iniSEE-pkg.R, right?
Kevin Rue-Albrecht (12:05:24) (in thread): > (just so that I know where to look for them now)
Federico Marini (12:05:32): > not just that
Federico Marini (12:05:46): > circleCI can give you back”artifacts”
Federico Marini (12:06:01): > plus you can ssh login for “debugging” in case an error pops up on that machine
Federico Marini (12:06:13): > but we write great code anyway:smile:
Aaron Lun (12:06:28): > Hmph.
Aaron Lun (12:06:32) (in thread): > Yes.
Kevin Rue-Albrecht (12:06:35) (in thread): > yeah ok - I expected@importFrom methods as newto be one of them.
Aaron Lun (12:06:53) (in thread): > Just operate offinfrastructureand merge once you’re done.
Federico Marini (12:07:16): > other thingO, related to iSEE
Federico Marini (12:07:33): > it might be hopefully soon possible to build up the conda package
Federico Marini (12:07:47): > once they make 3.5.1 available from conda forge
Kevin Rue-Albrecht (12:08:05) (in thread): > No worries. I won’t get on it right now. I got yet another institute presentation to prepare for Thursday.
Kevin Rue-Albrecht (12:09:49): > nice
Kevin Rue-Albrecht (12:10:45): > unrelated: did anyone get an email about ORCID update around 4pm? I usually got those when a new review came in, but I can’t see anything new on the F1000 page
Federico Marini (12:11:48): > today?
Federico Marini (12:11:51): > no email
Kevin Rue-Albrecht (12:15:46): > Boh. Nevermind. Must be Crossref just messing with me then:sweat_smile:
Federico Marini (12:34:04): > btw, are we to expect more reviews? we thought the 2 were good enough
Federico Marini (12:34:26): > or maybe they sent it out to more and we now enjoy the enrichment of positive reviews
Kevin Rue-Albrecht (12:34:51): > I was wondering that last week, I think the answer is that we don’t know how many reviewers accepted the invitation
Kevin Rue-Albrecht (12:35:08): > If I’m not mistaken I’ve seen up to 5 reviewers for some articles
Kevin Rue-Albrecht (12:36:02): > (e.g.https://f1000research.com/articles/5-2122/v2) - Attachment (f1000research.com): F1000Research Article: A step-by-step workflow for low-level analysis of single-cell RNA-seq data with Bioconductor. > Read the latest article version by Aaron T.L. Lun, Davis J. McCarthy, John C. Marioni, at F1000Research.
2018-07-04
Aaron Lun (08:59:04): > Don’t touch the vignettes right now.
Aaron Lun (10:01:50): > Okay,vignettorbranch is done on my end.
Aaron Lun (10:17:19): > @Kevin Rue-AlbrechtCHECK is not happy with the CITATION, keeps on raising a NOTE about broken connections.
Kevin Rue-Albrecht (10:17:31): > ow, that’s weird
Aaron Lun (10:17:51): > Wait for the latest PR to merge.
Kevin Rue-Albrecht (10:17:55): > Travis-only? or local too?
Aaron Lun (10:49:44): > Still waiting.
Aaron Lun (10:50:32): > importFroms have all been relocated to their correct homes (more or less).
Aaron Lun (10:51:30): > Urk. Travis shat itself.
Aaron Lun (12:53:57): > Anyone know how to set off a new Travis build, other than dummy commits?
Federico Marini (12:55:08): > restart build
Federico Marini (12:55:13): > there should be a button
Federico Marini (12:55:27): > alas, provided you own the repo, i guess
Kevin Rue-Albrecht (12:55:52): > I’m pretty sure Travis has the button, in the build history tab
Federico Marini (12:55:53): > lemme check
Kevin Rue-Albrecht (12:56:05): > I’ve done that before for iSEE
Federico Marini (12:56:07): > no, ok, I could restart it
Federico Marini (12:57:11): > so, it is running, and passing over that critical spot
Aaron Lun (12:57:46): > geez it does an awful lot of work with reinstalling everything.
Federico Marini (12:58:13): > afaik it happens everytime
Federico Marini (12:58:23): > probably it is not shown in detail by default
Federico Marini (12:58:30): > the cache got picked correctly
Aaron Lun (12:58:31): > yeah, I know
Aaron Lun (12:58:50): > but if it has to reinstall from a fresh image… what a waste of time
Aaron Lun (12:58:59): > I mean, just have an image preloaded with R and Bioc on it.
Aaron Lun (12:59:09): > I can only hope that’s what they’re actually doing
Aaron Lun (12:59:18): > and that the “installation” is more like “loading stuff that’s already there”
Federico Marini (12:59:34): > that is actually what circleCI does
Federico Marini (12:59:45): > it builds upon docker images
Federico Marini (12:59:53): > say, rocker/devel
Kevin Rue-Albrecht (12:59:59): > There are some builds worse than others when sets of pacakges get released (e.g. Bioc release), but there are also more random events (e.g. when I had to add the extra step to install devtools last week, even thought it should have been in our build cache for months now)
Kevin Rue-Albrecht (13:01:20): > I’m pretty sure we could remove that extra line (in .travis.yml), now that it should be back in the cache, but I doubt that’s slowing down the new builds
Federico Marini (13:01:37): > that’s really not a big deal
Federico Marini (13:01:58): > BTW
Federico Marini (13:02:20): > I could happily sell the idea of iSEE as a browser for a big dataset to be generated
Aaron Lun (13:02:47): > What’s the difference between “push” and “pr” for travis?
Federico Marini (13:02:52): > ~600 patients, rnaseq, bulk. yet, pretty cool how easy I got them excited
Federico Marini (13:03:12): > one would be for the commit itself, the other for the merge? not so sure tbh
Aaron Lun (13:03:13) (in thread): > This is all done.
Aaron Lun (13:03:39): > Anyway, everyone; make sure you read the PR message athttps://github.com/csoneson/iSEE/pull/219 - Attachment (GitHub): Split vignette into three sub-vignettes by LTLA · Pull Request #219 · csoneson/iSEE > The names and titles of the vignettes have been chosen carefully to respect the following conventions: Vignettes are executed in the alphanumeric order of their file names. Vignettes are listed on…
Kevin Rue-Albrecht (13:06:04): > lol forxtreme:slightly_smiling_face:
Aaron Lun (14:12:52): > prdidn’t rebuild
2018-07-05
Federico Marini (10:09:39): > Do the editors @f1000res check what we post as reply? i.e. is it correct our reply is not showing up yet?
Aaron Lun (10:23:23): > yes
Aaron Lun (10:23:25): > it is curated
Aaron Lun (10:23:47): > I haven’t posted lorena’s one, still waiting for you guys
Aaron Lun (10:23:54): > to do the stuff marked in red.
Federico Marini (11:38:08): > sorry, I am a bit swamped from things that need to be done with deadlines pending
Kevin Rue-Albrecht (11:39:19): > Same thing here, in a tight spot with pre-summer deadlines and other reviews that came in this weekend
Aaron Lun (11:40:06): > I haven’t been asked to review a paper this week. Feel sort of odd.
Aaron Lun (11:40:11): > Feel unwanted.
Aaron Lun (11:40:28): > https://www.youtube.com/watch?v=FxYw0XPEoKE - Attachment (YouTube): Whitney Houston - I Have Nothing (Official Video)
Federico Marini (11:41:42): > do you want to revise a DFG grant proposal:stuck_out_tongue:?
Aaron Lun (11:42:11): > Well, Icould. How seriously I do it depends on how much money I get.
Federico Marini (11:44:43): > the guy leading the whole thing is applying for about a million Euronen
Aaron Lun (11:47:58): > and you’re applying for … ?
Aaron Lun (11:48:10): > Anything less than 500K is chump change to me.
Aaron Lun (11:48:17): > GBP
Federico Marini (11:50:42): > we want a person
Federico Marini (11:50:51): > plus sachmittel a.k.a. storage & cores & RAM
Aaron Lun (11:53:00): > so about 200 K eu?
Aaron Lun (11:53:12): > don’t know how expensive people are over there
Aaron Lun (11:53:23): > I guess it would probably be about 100 GBP over here.
Aaron Lun (11:53:26): > K
Federico Marini (11:55:10): > could be?
Federico Marini (11:55:15): > are you guys that cheap in UK?
Federico Marini (11:55:51): > (these are the guys who want the browser/portal based on iSEE, btw)
Federico Marini (11:56:11): > it is so nice and evil to promise milestones that can happen with fingersnaps
Federico Marini (11:56:13): > :smile:
Aaron Lun (12:15:33): > well, depends on who you’re hiring
Aaron Lun (12:15:43): > I mean, if it’s coding jesus, then it might cost a bit more.
Aaron Lun (12:15:49): > Or less, he might do it for free, I suppose.
Federico Marini (14:24:13): > he could transmute perl scripts to r packages
2018-07-06
Kevin Rue-Albrecht (18:58:22): > FYI, I’ve filled in the wiki with information about the Docker build here:https://github.com/csoneson/iSEE/wiki/Docker-image - Attachment (GitHub): csoneson/iSEE > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
2018-07-07
Aaron Lun (10:56:42): > Good. Good.
Aaron Lun (11:08:58): > The Force is strong in this one.
Kevin Rue-Albrecht (17:52:29): > Just spotted this iSEE-esque tweet:https://twitter.com/ldoroud/status/1015645075903320064 - Attachment (twitter): Attachment > Daniel Hudson’s talk has definitely been the funniest talk at #ISMB18 so far! His software splittree5 even generates a paragraph or two for your paper’s method section! #ISMB18 https://pbs.twimg.com/media/DhhLzwzX4AA13g0.jpg
2018-07-08
Aaron Lun (09:31:27): > lol
Aaron Lun (09:59:35): > What’s next, the package writing the paper for us?
Aaron Lun (09:59:53): > Better be in Latex
Aaron Lun (09:59:58): > don’t want none of this endnote shit
Aaron Lun (10:00:02): > :face_vomiting:
2018-07-11
Aaron Lun (10:50:39): > I’m planning to talk about iSEE at the single cell genomics meeting in oct
Aaron Lun (10:50:46): > it would be nice to get the remaining features off our plate by then.
Kevin Rue-Albrecht (10:51:20): > Agreed. Busy this week, but I’m definitely hoping to find more time over the summer.
Kevin Rue-Albrecht (10:52:51): > I was getting worried about losing track of where we left off, almost a couple of weeks ago. And I remember spotting a typo or two in your PR that split the vignettes, but I just couldn’t get around to it for time reasons
Kevin Rue-Albrecht (10:53:59): > last one, I also want to push upstream reasonably soon, as I’ve genuinely lost track of how much new stuff we have on our GH compared to the latest release on Bioc-devel
Kevin Rue-Albrecht (10:55:15): > With increasing gaps in our contributions frequency, I think it’s increasingly important to stick something in the NEWS file with every (reasonable) PR, to avoid that issue in the future
Aaron Lun (10:59:17): > WHAT. A TYPO in MY VIGNETTES!
Aaron Lun (10:59:23): > Inconceivable
Aaron Lun (10:59:41): > Expect the column assay to have merge conflicts galore as I fixed most files’ indenting.
2018-07-12
Aaron Lun (08:11:02): > Okay, submitted the abstract.
Charlotte Soneson (10:44:35): > I just pushed thetouractionbranch, where I (1) added a line instructing people to follow all the actions in the tour, (2) added instructions for how to set up a new tour, (3) update the CyTOF section of thextremevignette to use theHDCytoDatapackage.
2018-07-13
Charlotte Soneson (07:06:45): > Does anyone know if there is a good way to figure out the name of the UI elements while running the app? I triedhttps://selectorgadget.com/, which kind of works, but it’s not always very precise.
Federico Marini (07:29:38): > I go via “developer tools”, which differ in the exact name by browser
Federico Marini (07:29:49): > I guess you ask for finding out the anchor points of the tours?
Charlotte Soneson (07:30:00): > Yes, exactly
Charlotte Soneson (07:30:47): > If you have a way that works, could you add a sentence to the last section in thextremevignette (in thetouractionbranch)?
Federico Marini (07:40:44): > Good point. I can try to wrap up a line
Federico Marini (07:41:06): > Got to finish a couple of things before leaving for two weeks of bella Italia
Federico Marini (07:41:12): > but I’ll squeeze that in
Federico Marini (07:41:48): > (although, I got to say, this is really a non-basic topic. But indeed it ends up in the xtreme path)
Charlotte Soneson (07:46:33): > Sure. But if anyone wants to write a tour, they’re going to have to know how to figure out the UI element IDs somehow
Kevin Rue-Albrecht (07:47:20): > (Wasn’t immediately obvious to me)
Aaron Lun (07:47:39): > On another note: remind me what’s the use case for column assay plots.
Charlotte Soneson (07:47:51): > e.g. plot one sample vs another
Aaron Lun (07:48:08): > Yeah, but why?
Charlotte Soneson (07:48:34): > well, at least for bulk RNA-seq people look at correlations between replicates
Aaron Lun (07:49:52): > Hm. Sounds a bit silly IMO.
Aaron Lun (07:50:40): > It’s not like you’re actually getting the R^2 either.
Charlotte Soneson (07:51:24): > No, but as a sort of qc I don’t think it hurts to have the possibility
Aaron Lun (07:51:30): > Hm.
Aaron Lun (07:52:59): > Well, fine.
Aaron Lun (07:53:29): > But there’s so many merge conflicts iniSEE-main.Rthat I’ll just re-implement it from scratch.
Charlotte Soneson (07:54:02): > As you wish
Aaron Lun (08:14:25): > listening to “9 hours of relaxing rain sounds” while coding… ah
Kevin Rue-Albrecht (08:14:40): > What happened to ABBA?
Federico Marini (08:14:59): > he swiched to the latest one after “it’s raining men”
Aaron Lun (08:16:42): > Now I need to go to the toilet, brb.
Federico Marini (08:17:12): > Hopefully not joke-induced:slightly_smiling_face:
Aaron Lun (09:02:55): > @Federico Mariniwhere’s my colors
Federico Marini (09:11:48): > I collected some ideas in sparse notes, i.e. notes and electronic notes. Will try to put up one single doc with the content, at best where examples can be shown
Federico Marini (09:12:21): > (I did not forget that one)
Aaron Lun (09:17:25): > @Charlotte SonesonSeesampassay. Currently using select’s to choose columns. However, this will fail with the cytof examples as the selectize would be forced to load 1000’s of options (not that it really makes sense to create sample assay plot for these anyway). Might be a good idea to turn off these plots by default.
Kevin Rue-Albrecht (09:42:48): > Can have a similar check as for ‘discrete’ colData: set a global optionsiSEE.maxSampleAssayand disable this type of panels if there are more samples than allowed. > A simpler alternative being to add this as an argument toiSEE(...):sweat_smile:
Aaron Lun (09:47:11): > Sort of ugly. The solution would be to set up server-side selectize updating, but that’s a pain.
Aaron Lun (10:09:15): > @Charlotte SonesonAnyway,sampassayis done from my end. If you can fix the tests, make a PR and merge, we can mark this off.
Aaron Lun (10:10:04): > Also, does anyone know why we have 2 skipped tests?
Federico Marini (10:12:08): > (does it make sense we let the manuscript also in the R channel/gateway on f1000res?)
Aaron Lun (10:12:46): > I don’t understand what you just said.
Federico Marini (10:13:35): > our ms is as of now only in the bioconductor gateway
Federico Marini (10:13:48): > we could have it added in the R gateway as well?
Aaron Lun (10:29:46): > Well, I guess we could have.
Aaron Lun (10:29:54): > … not really a big deal, IMO.
Aaron Lun (10:30:04): > Don’t really think these gateways add all that much.
Federico Marini (10:59:14) (in thread): > Although I got to say, this is not so bad
Federico Marini (10:59:33) (in thread): > My way did not force to add any custom addon or similar
Federico Marini (11:01:23) (in thread): > What people could find useful would be some screenshots but we don’t want to put too much stuff in the pkg right?
Charlotte Soneson (11:04:46) (in thread): > you mean screen shots of how to apply the selector gadget? Maybe we can refer to the CRAN vignette:https://cran.r-project.org/web/packages/rvest/vignettes/selectorgadget.html
Federico Marini (11:09:56) (in thread): > rather for the “undocumented” default tools
Federico Marini (11:10:14) (in thread): > lemme check, maybe mozilla could/should have something
Federico Marini (11:11:31): > @Aaron Lun, the basic vignette is storing the RDS file, is this intended?
Aaron Lun (11:12:00): > Yes, because the later vignettes need the SCE and I don’t want the time delay of generating it again.
Federico Marini (11:13:03): > k
Aaron Lun (11:13:08): > I’m not entirely clear on the R build system; I don’t know whether objects persist across vignettes. Maybe they do, in which case we don’t need to save the SCE.
Aaron Lun (11:13:32): > Though probably safe to do so anyway, just in casescegets overwritten somewhere in other vignettes.
Federico Marini (11:13:33): > I was trying to run them “interactively” and they could not find directly the object
Federico Marini (11:13:44): > we’ll find out
Federico Marini (11:23:24): > catching up but finding this error > > Error in rowData(object, use.names = FALSE) : > could not find symbol "use.names" in environment of the generic function >
Federico Marini (11:23:47): > known to any of you? I am updating all pkgs just to be sure
Kevin Rue-Albrecht (11:34:55): > I think i had to install SingleCellExperiment from GitHub to get past that one
Federico Marini (11:35:17): > thanks, could be
Federico Marini (11:35:23): > i’ll try it out
Kevin Rue-Albrecht (11:35:41): > my memory is blurry because that was a few weeks ago
Federico Marini (11:36:10): > noprobs. I confess I could not do that much on iSEE recently. Well, except evangelizing
Kevin Rue-Albrecht (11:40:16): > Had a progress report meeting to prepare for this morning, and last minute analyses for an abstract to finish for Tuesday (Genome Informatics 2018, feat. Mark Robinson)
Federico Marini (11:41:04): > - if using Microsoft Explorer, well you have bigger problems to deal withcan be fine?:stuck_out_tongue:
Kevin Rue-Albrecht (11:41:29): > it’s endless, but I haven’t given hope of doing my part for iSEE following up on the reviewer comments
Kevin Rue-Albrecht (11:41:37): > lol
Kevin Rue-Albrecht (11:42:12): > “Selectize widgets powered by Bing”
Federico Marini (11:51:17) (in thread): > I think I managed to describe it via words, so no need to link to dozens of pages
Federico Marini (11:54:08): > ok, main descriptions to select names via browser are in
Federico Marini (11:54:27): > working on the sametouractionbranch created by Charlotte
Federico Marini (11:55:17): > disclaimer: the names of the menus and so can change throughout versions of the browsers, and probably also across OSs
Federico Marini (11:55:47): > maybe Aaron you could check it on the Ubuntu machine of yours?
Charlotte Soneson (11:57:47) (in thread): > Sounds great
Charlotte Soneson (11:57:50) (in thread): > I’ll have a look
Federico Marini (12:00:29) (in thread): > there you gohttps://github.com/csoneson/iSEE/commit/7af2b0b967023badfc0312335c5a1d59fe37f4fa - Attachment (GitHub): added description on how to select UI elements via browser functionality · csoneson/iSEE@7af2b0b > iSEE - R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class
Charlotte Soneson (12:03:02) (in thread): > :+1:
Federico Marini (12:04:03) (in thread): > (btw, it gets feasible more n more that we meet in HD)
Charlotte Soneson (12:04:27) (in thread): > Great:slightly_smiling_face:Simone mentioned that you had written to her
Federico Marini (12:04:30) (in thread): > maybe I’ll carry with me a colleague or two of mine which soon will generate sc data
Charlotte Soneson (12:04:46) (in thread): > We’re in Brixen for the Bioconductor course this week
Federico Marini (12:04:56) (in thread): > Do you know what the target would be for the course part?
Charlotte Soneson (12:05:09) (in thread): > No idea
Federico Marini (12:05:09) (in thread): > whether some bench biologists would benefit, de facto
Federico Marini (12:05:45) (in thread): > well, I’d tell them better to travel along than to regret to have missed that
Charlotte Soneson (12:05:55) (in thread): > Haha, exactly!
Federico Marini (12:07:12) (in thread): > the worst case scenario is that we got a train ticket to HD
Charlotte Soneson (12:07:55) (in thread): > Could be worse
Charlotte Soneson (12:28:45): > Pulling thesampassaybranch and running the usual Allen example completely crashes my R session without any helpful messages…@Aaron LunI suppose you tested the app on your system before pushing?@Kevin Rue-Albrecht,@Federico Marini: before going deeper into this, would any of you have the possibility of testing if it works for you (i.e., if it’s an issue with my setup)? I just did > > require(scRNAseq), > require(scater) > data(allen) > > sce <- as(allen, "SingleCellExperiment") > counts(sce) <- assay(sce, "tophat_counts") > sce <- normalize(sce) > sce <- runPCA(sce) > sce <- runTSNE(sce) > > rowData(sce)$num_cells <- base::rowSums(counts(sce)>0) > rowData(sce)$mean_count <- base::rowMeans(counts(sce)) > iSEE(sce) >
Federico Marini (12:41:16): > as kev said, install the SCE package from github
Federico Marini (12:41:19): > that works
Charlotte Soneson (12:41:33): > oh, it’s the same problem?
Federico Marini (12:44:19): > guess so
Charlotte Soneson (12:44:44): > Well, that made things worse…now I can’t even loadiSEEwithout R crashing:grimacing:ifSingleCellExperimentis already loaded
Charlotte Soneson (12:46:21): > Guess there are some other things that need reinstalling too
Kevin Rue-Albrecht (13:20:21): > still.. as said, that story about having to reinstall SCE from GitHub bugged me a few weeks back, and if I’m not mistaken i got the fix from@Aaron Lun. Point is, is there a release of SCE that’s awaiting a push upstream?
Federico Marini (14:17:44): > seems so
Aaron Lun (15:35:03): > SCE should be up to date.
Aaron Lun (15:35:33): > Crashing is a segfault due to exceeding stack limits, not sure where that comes from but it goes away after updating and reinstalling iSEE.
Aaron Lun (15:36:18): > Otherwise it seems to work fine for me… so
Aaron Lun (15:36:32): > ¯*(ツ)*/¯
Aaron Lun (17:41:53): > Also, someone needs to get rid of BiocInstaller mentions. Don’t know why we even have that - we should just point people to the landing page.
Charlotte Soneson (18:15:15): > PR ofsampassayinitialized
2018-07-14
Charlotte Soneson (02:47:43): > Anything else going intotouractionbefore merging? It addresses the text in red in the reply to Lorena (exceptWe now refer readers of the paper to the package vignette for further details., but I guess that should go into the paper)
Kevin Rue-Albrecht (04:38:38): > We could update thebiocLitething to BiocManager, in the vignette and the iSEE_install.R script (used to make the Docker image) > > $ grep -n biocLite vignettes/* > vignettes/basic.Rmd:67:source("[http://bioconductor.org/biocLite.R](http://bioconductor.org/biocLite.R)") > vignettes/basic.Rmd:68:biocLite("iSEE") >
Kevin Rue-Albrecht (04:51:29): > I could have a look around lunch time maybe. I need to sit myself down to update plots following a(nother) review
Aaron Lun (05:52:46): > Probably should remove all references to installation, and just re-direct people to the landing page.
Kevin Rue-Albrecht (05:53:08): > indeed. i think we mentioned that possibility too ^^
Kevin Rue-Albrecht (05:53:41): > I’ve just pulled thetouractionbranch, and waschecking it on my system before touching anything yet
Kevin Rue-Albrecht (05:54:14): > but if you feel like nuking those bits and pieces of vignette, I won’t stand in your way ^^
Kevin Rue-Albrecht (05:59:07): > checkstill running, but I’m getting a weird: > > * checking installed package size ... NOTE > installed size is 7.2Mb > sub-directories of 1Mb or more: > doc 6.3Mb >
Kevin Rue-Albrecht (05:59:16): > any idea where that could come from?
Kevin Rue-Albrecht (06:01:22): > Alright, it’s the only otherNOTEthan our regularUnexported objects imported by ':::' calls
Aaron Lun (06:08:09): > ¯*(ツ)*/¯
Aaron Lun (06:08:48): > I wouldn’t worry about it.
Kevin Rue-Albrecht (06:08:51): > Another question: the vignette is full of hypertext links to the iSEE landing page… How would you phrase “Installation instructions are available at ….”
Aaron Lun (06:09:02): > “the landing page”.
Kevin Rue-Albrecht (06:09:07): > aside from “click the goddam link”
Aaron Lun (06:09:17): > or “here”, with a link.
Aaron Lun (06:09:43) (in thread): > You might have some HTMLs invignettes/that you didn’t clear out beforeBUILD?
Kevin Rue-Albrecht (06:09:54): > alright, I’ll stick in the “Package Short Url” then, as the link
Aaron Lun (07:52:31): > @Federico Marininew sentences = new lines in vignette.
Kevin Rue-Albrecht (08:04:51): > btw, I thought needless to say, but i pushed the biocLite update
Aaron Lun (08:05:26): > k
Aaron Lun (08:22:08): > FINISH HIM:-1:
Kevin Rue-Albrecht (08:46:32): > the minutes of thetouractionbranch are now counted
Kevin Rue-Albrecht (08:50:24): > @Kevin Rue-Albrechtuploaded a file:iSEE User’s Guideand commented: shall we give different names to the triplets? - File (PNG): iSEE User’s Guide
Kevin Rue-Albrecht (08:55:35): > Also, we may want to customise the keywords for each vignette. Currently, they all have%\VignetteKeywords{GeneExpression, RNASeq, Sequencing, Visualization, QualityControl, GUI}, undoubtedly due to some paralogous relationship between them.
Kevin Rue-Albrecht (08:57:05): > For the vignette index, I’ll humbly suggest: > - “iSEE User’s Guide” > - “The ExperimentColorMap class” > - “iSEE Advanced Features”
Aaron Lun (09:06:33): > Yes, they need to be different. Can’t remmeber whether they need to be alpha ordered tho
Kevin Rue-Albrecht (09:07:16): > @Kevin Rue-Albrechtuploaded a file:Screenshot 2018-07-14 14.07.00.png - File (PNG): Screenshot 2018-07-14 14.07.00.png
Kevin Rue-Albrecht (09:07:22): > don’t seem to be an issue
Kevin Rue-Albrecht (09:07:39): > i think it’s only the file name that counts
Aaron Lun (09:26:26): > Hm. On the BioC landing page it’s the vignette title that counts.
Kevin Rue-Albrecht (09:26:48): > ow. didn’t think about there
Kevin Rue-Albrecht (09:27:51): > Well i’ve pushed it now, but feel free to edit. Worse case we can prefix them with number. I think ComplexHeatmap did that
Aaron Lun (09:28:08): > seehttp://bioconductor.org/packages/devel/workflows/html/simpleSingleCell.html - Attachment (Bioconductor): simpleSingleCell (development version) > This workflow implements a low-level analysis pipeline for scRNA-seq data using scran, scater and other Bioconductor packages. It describes how to perform quality control on the libraries, normalization of cell-specific biases, basic data exploration and cell cycle phase identification. Procedures to detect highly variable genes, significantly correlated genes and subpopulation-specific marker genes are also shown. These analyses are demonstrated on a range of publicly available scRNA-seq data sets.
Kevin Rue-Albrecht (09:28:43): > ah. always good ideas that guy
Aaron Lun (09:31:14): > Okay, I’m going for a walk. I finished tearing strips off a certain imputation method, so I will now think about happy things.
Aaron Lun (09:56:15): > @Charlotte SonesonOn an aside, why doesBodenmiller_BCR_XL_SE()give a SE object with cells in the rows and markers in the columns? That’s going to confuse a lot of people.
Aaron Lun (09:56:39): > I’m surprised whoever was handling that submission let that through.
Aaron Lun (09:58:38): > @Kevin Rue-Albrechtalso push upstream once it’s tested through.
Kevin Rue-Albrecht (09:59:50): > Yep. I’ve been nervous about accumulating new code without pushing upstream, but conversely there was quite a lot going on and I didn’t want to push upstream without a good/thorough check
Kevin Rue-Albrecht (10:00:24): > Knee-deep in updating figures for a revision, but I’m around if anyone wants to give the go-ahead
Aaron Lun (10:03:40): > Im going for my walk now, so just chill
Charlotte Soneson (10:19:38) (in thread): > Yes, we talked about this, but Lukas felt that this was more in line with how the CyTOF community sees things (and the package only contains cytometry data).
Aaron Lun (11:04:57) (in thread): > Sounds like it would have made more sense tonotstore it in a SE object, then. With a raw matrix, you would avoid implicit assumptions about what the rows and columns mean.
Aaron Lun (11:14:44): > Back from my walk
Federico Marini (13:02:28) (in thread): > Sorry about that
Federico Marini (13:02:53) (in thread): > I had been writing in another style for the whole day and forgot to switch to our common one
Federico Marini (13:03:08) (in thread): > I see you already fixed that, right? -thanks!
Kevin Rue-Albrecht (15:49:13): > if anyone wants a brain teaser, I can’t figure how to turnusr <- sapply(usr, .substituteNullColorMap) return type is "closure"into avapply. (AllClasses.R, line 140). > Namely, the return type of.substituteNullColorMapis aclosure(i.e., function), which is not among the documented values ( logical < integer < double < complex)
Kevin Rue-Albrecht (18:23:55): > There are 3sapplyleft, all of which with the same problem as above. Happy to hear thoughts. I might message on bioc-devel otherwise
2018-07-15
Aaron Lun (06:49:41) (in thread): > Yes, now colour me.
Aaron Lun (06:50:17): > Probably should just gounlist(lapply())
Aaron Lun (09:20:25): > or hell, justlapply. A vector of functions would just be a list anyway.
2018-07-16
Aaron Lun (13:15:52): > You know, I’ve just noticed this, but in the TCGA tour,CNTLis a factor not a logical. WTF?
Aaron Lun (13:46:01): > Also - this is more of an annotation problem, but the ExperimentHub TCGA data is missing quite a few genes. Where did it all go?
Kevin Rue-Albrecht (14:07:57): > 1) I think I was in a zealous phase “discrete covariate should be factors”, 2) for some reason it seems that I wantedTRUEto be the first level
Kevin Rue-Albrecht (14:08:40): > No clue about the missing genes. I just picked up the first big data set available on the ExperimentHub. Didn’t ask questions at the time.
2018-07-17
Aaron Lun (06:01:44): > Hmph.
2018-07-18
Aaron Lun (14:53:52): > recount’s SE is probably more comprehensive but is also more of a monster.
2018-07-19
Aaron Lun (12:45:10): > You guys still alive?
Kevin Rue-Albrecht (12:45:24): > barely
Kevin Rue-Albrecht (12:45:56): > late for a progress report
Aaron Lun (12:47:54): > Jesus
Aaron Lun (12:48:04): > you have to write reports?
Kevin Rue-Albrecht (12:48:18): > Powerpoint this time
Aaron Lun (12:48:42): > you want a job that doesn’t treat you like a child?
Kevin Rue-Albrecht (12:49:53): > Don’t tempt me right now ^^
Kevin Rue-Albrecht (12:50:53): > I’m just going nuts running the same notebook for the n-th time, and updating figures accordingly
Kevin Rue-Albrecht (12:52:05): > that, dealing with two article revisions, and trying to prepare for being off a week, next week
Aaron Lun (13:11:50): > Well, just so you know.
Brendan Innes (14:18:07): > @Brendan Innes has joined the channel
2018-07-20
Kevin Rue-Albrecht (18:09:18): > Until the golden days when I get to freely look after iSEE and add tons of new features (looking at you, gene sets), I just had some fun adding some 15% of coverage to the ECM (2% overall), which led me to fix up a couple of redundant lines
Kevin Rue-Albrecht (18:09:56): > with that, I think we’ll be good to merge this annoyingly old PR (6 days:wink:)
Kevin Rue-Albrecht (18:10:35): > I just couldn’t accept that decrease of 0.3% coverage for the open PR
Aaron Lun (18:10:44): > Oh YEAH.
Aaron Lun (18:11:02): > epivizr is probably priority
Kevin Rue-Albrecht (18:12:04): > i know.. but it also feel like an intense brain commitment, to understand and get things in place, followed by the usual aftermath debug session
Kevin Rue-Albrecht (18:53:13): > Just one thing that I almost missed: with the renaming of vignette files, the “open vignette” button in the app was still pointing to the old name of the single vignette. I’ve just updated it to the new ‘basic’ vignette. This said, we should bump up and push upstream ASAP, so that GitHub/bioc-devel all point to the ‘basic.html’ vignette. > Although for a couple of months now, the link to ‘…/release/…/basic.html’ will be invalid
Kevin Rue-Albrecht (18:53:55): > Anyway. I’m happy with the version I just pushed. So if I get at least one confirmation from anyone here, I’ll bump up and push upstream.
Aaron Lun (19:37:46): > FINISH HIM
2018-07-21
Kevin Rue-Albrecht (04:37:53): > I’ll finish him as soon as Travis approves the version bump (lol). > Incidentally, I figured again what ‘typo’ it was that I found in the updated vignettes… ehm.. just the discrepancy between ‘colormap’ and ‘color map’ (we also used ‘colour map’ in the article).. > In any case, I moved everything to ‘colormap’ (to match the updated vignette), including the validation messages for the ECM class. It seems to be Google’s consensus out there in terms of search results. The only exception being viridis which does use the two-word form. > .. anyway.. I’ll stop my doctor of “philosophy” rant here ^^
Kevin Rue-Albrecht (04:45:59): > Done. Pushed upstream. Let’s figure out where to go next
Kevin Rue-Albrecht (04:49:09): > In the other minor things, there are still several R/* files that are not 4-space indented. > However, trying to do it with RStudio makes it look even worse (to me): a bunch of function calls where subsequent lines are aligned with the opening bracket (instead of a 4-space indent) get messed up, and I’d hate to mess with code that I haven’t written (codetrack, collapseBox). > Where do we stand on the 4-space vs alignment with opening bracket choice?
Aaron Lun (06:39:15): > four space.
Aaron Lun (06:39:17): > Why?
Aaron Lun (06:39:29): > Because my old vim editor preferred opening bracket, while my new vim editor prefers four space.
Aaron Lun (06:39:37): > ¯*(ツ)*/¯
Charlotte Soneson (11:45:58): > So - with the new changes, are we ready to fix the responses to the remaining reviewers?
Kevin Rue-Albrecht (11:46:23): > I think Lorena’s comments are addressed, at least
Kevin Rue-Albrecht (11:47:59): > I’d have to check more carefully though - today’s theme is “prepare revision for Nature comm’“. I’ll try to join in the iSEE fun wagon later later, if there’s anything left to do
Aaron Lun (11:57:03): > I still haven’t dealt with Alejandro’s comments.
Aaron Lun (11:57:12): > I was going to add some code to allow for arbitrary panels.
Kevin Rue-Albrecht (11:57:23): > well, he’s not an easy one to please, from the reviews ^^
Aaron Lun (11:57:31): > Then we don’t have to write analysis code, and people get their on-the-run-DE.
Kevin Rue-Albrecht (11:57:51): > The multiple selection point killed me.
Kevin Rue-Albrecht (12:00:49): > Don’t get me wrong - it’d be fun to have - but I think it’s pretty clear Shiny doesn’t support that and we’ve run out of keyboard shortcuts
Aaron Lun (12:01:43): > From memory, I think that F1000 holds onto responses until we submit version 2.
Aaron Lun (12:01:49): > To answer your question from a while ago.
Kevin Rue-Albrecht (12:02:33): > Ah right. Makes sense
Aaron Lun (13:42:55): > Just crunched through another review.
Aaron Lun (13:43:11): > 13th for the year!
Kevin Rue-Albrecht (15:17:39): > @Kevin Rue-Albrechtuploaded a file:“… and one other”and commented: no offense@Aaron Lun:rolling_on_the_floor_laughing: - File (PNG): “… and one other”
Kevin Rue-Albrecht (15:20:21): > I’m just going through Lorena’s comments one more time in our Google Docs, once again, so far I think we’ve addressed them all
Kevin Rue-Albrecht (15:56:35): > Only thing left that I can see is to add a statement to the article that refers readers to the vignettes. Trying to whip up something now
Kevin Rue-Albrecht (18:20:14): > Question for the crowd: should we take the opportunity to also add a mention to the newly added sampleAssayPlot ? > In the same vein: this makes me realise that the ‘basic’ vignette doesn’t mention it yet either
2018-07-22
Aaron Lun (06:07:08): > Probably should mention this in the vignette, and just add it to the list of possible panel options in the manuscript.
Aaron Lun (09:50:55): > argh so bored
2018-07-23
Kevin Rue-Albrecht (13:00:35): > When’s the last time you had a break?
Kevin Rue-Albrecht (13:01:26): > Sounds like you’ve been at it day in day out for a while
Aaron Lun (13:28:24): > Dunno. Must have been last year.
Aaron Lun (13:28:40): > As in, a no commit day?
Aaron Lun (13:28:47): > I guess this was in april at the HCA hackathon
Aaron Lun (13:28:51): > when I was sick and couldn’t get up.
Aaron Lun (13:28:57): > I guess that counts as a break?
Kevin Rue-Albrecht (13:33:58): > hahaha let’s say it does
Kevin Rue-Albrecht (13:35:30): > Point is, breaks are good. I’m not nearly as productive as you, and still friends occasionally warn me about burnout when they don’t hear from me in a while
2018-07-24
Aaron Lun (09:39:39): > Anyone in Toronto?
Charlotte Soneson (09:40:04): > yes, I’m here
Aaron Lun (09:49:40): > Give a shout out to iSEE if you can
Charlotte Soneson (09:51:20): > yup
2018-07-25
Charlotte Soneson (08:58:18): > I’ll do a 5 min lightning talk this afternoon on iSEE - will probably be just enough time to show the Allen tour:slightly_smiling_face:
Federico Marini (09:01:22): > wish I would be there
Federico Marini (09:01:43): > plan B is the beach on the adriatic riviera, so I can’t complain thaaaat much
Charlotte Soneson (09:01:57): > Doesn’t sound too bad:slightly_smiling_face:
Federico Marini (09:02:02): > is it going to be streamed?
Charlotte Soneson (09:02:17): > not today I think - don’t know for the rest
Federico Marini (09:03:05): > could have been a good answer to the beach-neighbours asking me “what do you do for a living”
Federico Marini (09:03:40): > and were not satisfied by the usual elevator pitch about data and bioinformatics and and and:stuck_out_tongue:
Aaron Lun (09:16:34): > Just say “I cure cancer”
Aaron Lun (09:16:36): > I do.
Aaron Lun (11:05:57): > :fast_parrot:
Aaron Lun (11:14:15): > lol
Aaron Lun (13:28:16): > Is it party central over there?
Charlotte Soneson (13:28:37): > Well, we’re in a chapel:slightly_smiling_face:
Aaron Lun (13:30:28): > For the conference, or for the afterparty?
Charlotte Soneson (13:31:32): > Ha - for the conference. I suppose we will not stay here at night (I’ll probably collapse around 7pm anyway:slightly_smiling_face:)
Aaron Lun (14:01:59): > how did the lightning go?
Charlotte Soneson (14:07:56): > Haven’t done it yet - should be in an hour or so
Aaron Lun (14:10:54): > oh yeah right. Time diff.
Aaron Lun (14:11:14): > Best to have theallenpage set up and running before you get on stage. Even better if you’re running it locally.
Aaron Lun (14:11:37): > Otherwise you’ll be standing there silently for 10 seconds.
Charlotte Soneson (14:16:45): > Yeah, I’ll have it locally. Internet connection seems to be coming and going a little bit as well.
Aaron Lun (14:25:55): > Alright, I’m bugging out. Good luck.:party_parrot:
Charlotte Soneson (14:26:31): > Thx
Davide Risso (15:24:42): > Well… I tried to take a picture to advertize iSee on twitter… #fail #iDidntSee #damnwindows - File (JPEG): Image from iOS
Charlotte Soneson (15:25:11): > Hahaha, thanks for trying!
Davide Risso (15:25:43): > No problem… nice talk!:+1:
Charlotte Soneson (15:25:58): > Thank you:slightly_smiling_face:
2018-07-26
Aaron Lun (14:14:42): > 5 days, 50 commits to go to hit 500.
Aaron Lun (14:14:45): > YES WE CAN.
2018-07-27
Federico Marini (15:36:08): > came for the sticker, stayed for the chapel
2018-07-28
Charlotte Soneson (15:27:34): > Contour plots can now be added to scatter plots - File (PNG): Screen Shot 2018-07-28 at 21.10.32.png
Aaron Lun (15:47:14): > Nice. Should have a conditional option for contour color. Also suggest “(scatter only)”
Charlotte Soneson (15:47:36): > Right, good point. I’ll add that
Aaron Lun (15:50:01): > Make sure you updatedefaults.Rdwith new options as well.
Charlotte Soneson (15:50:20): > Ah yes
Charlotte Soneson (16:47:21): > Ok, that should be fixed
Kevin Rue-Albrecht (17:12:54): > Nice one.
Kevin Rue-Albrecht (17:13:49): > Stuck in Nice for another day - EasyCancel.. sorry EasyJet cancelled our return flight. Flying back tomorrow.
Charlotte Soneson (17:14:29): > How kind of them…but Nice is not too bad if you have to be stuck somewhere.
Kevin Rue-Albrecht (17:16:56): > Exactly
Aaron Lun (19:42:12): > haha I get the joke.
Aaron Lun (19:42:21): > It’s Nice
Aaron Lun (19:42:51): > :fiesta_parrot:
2018-07-31
Aaron Lun (12:21:22): > @Charlotte SonesonThrew in some comments about the latest PR
Charlotte Soneson (12:22:26): > Thx. I completely missed the.conditional_on_check_solo()function:face_with_rolling_eyes:
Charlotte Soneson (12:22:57): > Not that it was particularly far away from the other one…
Charlotte Soneson (12:40:33): > Ok - fixed and pushed to branchcontourfix
Aaron Lun (12:41:45): > looks good:fast_parrot:
Federico Marini (13:34:51): > jeez what would we do without parrots
Aaron Lun (13:36:04): > :dealwithit-parrot:
2018-08-02
Aaron Lun (13:54:15): > Talk to me, people.
Aaron Lun (13:54:39): > I think I’ve done ~60 commits over the past two days.
Aaron Lun (13:54:56): > Actually, 78 forscater
Aaron Lun (13:55:05): > and about 20 forSingleCellExperiment
Aaron Lun (13:55:13): > close to 100 on these two repos alone.
Aaron Lun (13:55:53): > Am I tired? Do I smell like a dog?
Aaron Lun (13:55:58): > The answer to all those questions is YES.
Kevin Rue-Albrecht (14:22:50): > xD
Aaron Lun (14:25:06): > :fast_parrot:listening to some bonnie tyler right now.
Aaron Lun (14:25:09): > why yes, I do need a hero.
Kevin Rue-Albrecht (14:28:13): > Just looking at the reply to reviewers (Lorena’s section). Seems like the last item pending is the PR on LTLA/iSEE2018, that updates the manuscript to redirect users to the vignette for further details.
Kevin Rue-Albrecht (14:29:42): > If you’re looking for some easy commits, you can rephrase/polish the PR. I wasn’t particularly inspired how to phrase it
Aaron Lun (14:32:23): > I am so tired.
Kevin Rue-Albrecht (14:33:20): > Probably sign to take a break then:slightly_smiling_face:
Kevin Rue-Albrecht (14:33:44): > No summer school around to supply a couple of wine bottles?
Aaron Lun (14:33:57): > fraid not
Aaron Lun (14:34:09): > last three days have been 12 hour shifts
Aaron Lun (14:39:19): > Request moving the sentence in the PR to the end of the “Operation” section.
Kevin Rue-Albrecht (14:45:37): > Sentence moved.
Kevin Rue-Albrecht (14:46:36): > I’ll have a better look and think over the weekend. I’ve just seen the issue I’ve opened about the new panel type that’s not documented in the vignette nor the article
Aaron Lun (14:51:35): > You moved it to the wrong place.
Aaron Lun (14:52:19): > :dealwithit-parrot:
Kevin Rue-Albrecht (15:29:26): > argh
2018-08-03
Aaron Lun (08:37:28): > If anyone can getscater’s CI up and running again, I will buy them a drink when I see you.
Aaron Lun (08:37:49): > Well, I’llgetyou a drink, depending on whether anything can be purchased at a meeting place.
Aaron Lun (08:47:28): > I mean, if drinks are already free, there’s no point buying them.
Kevin Rue-Albrecht (09:11:47): > .. do you mean this onehttps://ci.appveyor.com/project/davismcc/scater(Failed a year ago in 4 min 50 sec) ?
Kevin Rue-Albrecht (09:12:40): > Because I only see warnings on the Bioc build, and I can’t see a reference to TravisCI onhttps://github.com/davismcc/archive-scater
Aaron Lun (09:14:59): > yes, pretty much.
Aaron Lun (09:15:09): > Note that onlyrefactorshould build correctly right now.
Kevin Rue-Albrecht (09:20:18): > ahh wait a second, I was looking at the archive-scater repo. I found the current one athttps://github.com/davismcc/scaterwhich has a TravisCI badge. > Anyway, I can see the latest error message “The job exceeded the maximum time limit for jobs, and has been terminated.“, which I usually go around by triggering new Travis builds manually (the “More options” dropdown at the top right of the Travis build page) until it passes
Kevin Rue-Albrecht (09:20:48): > That timeout is common when the package hasn’t been updated for a while, and a bunch of packages need to be updated
Aaron Lun (09:21:53): > dammit, only davis can do that.
Kevin Rue-Albrecht (09:22:26): > in that case, he owes us both a drink, I suppose
Aaron Lun (09:23:08): > Well, okay, let me change the bounty then.
Aaron Lun (09:23:31): > I will get a drink for anyone who can convertplotHeatmapinto a fullyggplot-compatible function.
Aaron Lun (09:23:46): > Though I suspect this isn’t really possible.
Kevin Rue-Albrecht (09:29:00): > I must admit that I don’t remember using it. And thefor (field in colour_columns_by) {confuses me a bit about how many extra annotations that function is suppose to support.
Kevin Rue-Albrecht (09:29:50): > I need to finish some other stuff now, but I’m curious if I can save some time later today
Aaron Lun (11:05:33): > I wonder whether we could launch a shiny app from within another shiny app. This would make it much easier to do custom row/column plots; simply subset the SCE and relaunch the app!
Aaron Lun (11:09:11): > I think I know how to do it (in effect - not actually re-running things).
Aaron Lun (11:23:31): > Actually, scratch that, I don’t know how to do it.
Aaron Lun (12:33:28): > Wait, maybe I do.
Aaron Lun (13:21:13): > No, no I don’t.
Kevin Rue-Albrecht (14:08:19): > I definitely don’t. Can’t remember seeing an example either:neutral_face:
Aaron Lun (14:10:19): > https://github.com/csoneson/iSEE/issues/225 - Attachment (GitHub): Considering support for custom plots and tables · Issue #225 · csoneson/iSEE > We currently support a custom column plot panel type. This panel type accepts a selection from a column-based plot, performs an arbitrary user-supplied operation to generate coordinates, and then p…
Kevin Rue-Albrecht (14:18:25): > ah. Great minds think alike. I actually just hinted to this in the response to reviewers last night ^^
Kevin Rue-Albrecht (14:19:08): > (second to last comment for Alejandro)
Aaron Lun (14:23:23): > Didn’t even see it, though I was thinking of Alejandro’s suggestion when I wrote this.
Aaron Lun (14:23:37): > :beret-parrot:
2018-08-04
Kevin Rue-Albrecht (08:58:57): > I just noticed that our.travis.ymlincludes > > bioc_packages: > - BiocInstaller > > … which will probably be deprecated at some point. Not sure whether we should drop it altogether or replace it withBiocManager
Kevin Rue-Albrecht (08:59:04): > https://github.com/csoneson/iSEE/blob/master/.travis.yml
Kevin Rue-Albrecht (09:00:40): > (bearing in mind that BiocManager is actually CRAN, so would go underr_packages:YAML section)
Aaron Lun (09:04:28): > ¯*(ツ)*/¯
Aaron Lun (09:25:37): > Alright, time to finish the fight.
Kevin Rue-Albrecht (09:26:33): > you’ll have to be more specific i’m afraid
Aaron Lun (09:34:11): > custom
Kevin Rue-Albrecht (09:36:17): > Oh dear, indeed. Which one is it then: making them end-point, or adding custom tables and rowData?
Aaron Lun (09:53:39): > Hey, you’re engaged right?
Aaron Lun (10:16:40): > Are we getting an invite?
Kevin Rue-Albrecht (10:50:23): > 1) yes 2) eh.. let me ask miss 1st author:wink:
Aaron Lun (10:52:20): > :party_parrot:
Aaron Lun (11:27:41): > customized row plots are now added. Some tests are probably broken though.
Aaron Lun (11:27:52): > I will go for a walk and come back and add customized tables.
Aaron Lun (12:23:12): > Had a better idea.
Kevin Rue-Albrecht (13:05:02): > that’s what walks are for:slightly_smiling_face:
Aaron Lun (15:00:54): > seealt_custom, and in particularexample("Custom iSEE plots")(the written documentation hasn’t caught up yet but the example should work). Try selecting some cells or genes, and see how the PCA plot changes.
2018-08-05
Aaron Lun (10:46:24): > BEHOLD
Aaron Lun (10:46:27): > custom.Rmd.
Aaron Lun (10:46:39): > :exploding_head:
Aaron Lun (10:47:12): > Can someone fix the tests to passcheck? I am exhausted. All the bugs were in observers so they took ages to track down.
Kevin Rue-Albrecht (10:47:41): > oh god, those observers have always blown my mind
Kevin Rue-Albrecht (10:48:08): > (hence theprintstatements that occasionally made it in my commits)
Aaron Lun (10:48:22): > Check out the new custom vignette, you’ll see how amazing it is.
Kevin Rue-Albrecht (10:49:01): > Alright. Go take a break though in the meantime:wink:
Kevin Rue-Albrecht (11:17:55): > HOLY CUSTOM PLOT!
Kevin Rue-Albrecht (11:18:36): > they receive frombothcolumnandrow data plots now?
Kevin Rue-Albrecht (11:24:21): > on a separate note, I just realised that the “Sample assay plot” does not allow the selection of which gene is plotted. I can select X, Y, and the assay name, but I can’t see what gene expression I’m actually looking at
Aaron Lun (11:50:37): > Not sure what you mean. all things are genes.
Kevin Rue-Albrecht (11:51:13): > yeah.. i thought I was missing something:face_palm:
Kevin Rue-Albrecht (11:51:36): > sorry, juggling a few things this afternoon
Aaron Lun (11:57:16): > If you can fix the existing tests and get it to pass check, I’ll add the remaining tests later.
Kevin Rue-Albrecht (11:58:31): > I’ve had a brief look, and I’m hoping to get to it in a couple of hours, at least some of them seem to be related to missingstat_fun_list, which I assume relates to the custom tables
Kevin Rue-Albrecht (11:59:17): > I’ll just need to warm up a bit on the recent updates, to do that cleanly
Aaron Lun (12:11:34): > :party_parrot:
Kevin Rue-Albrecht (13:52:02): > Just for the record, I’ve started touchingtest_custom.R. Getting familiar with the updated functions/arguments
Aaron Lun (13:52:59): > kay
Kevin Rue-Albrecht (13:53:02): > feeling a bit:parrotconga::reverse_conga_parrot:
Aaron Lun (13:54:59): > huh:face_with_open_mouth_vomiting:
Kevin Rue-Albrecht (14:11:25): > hurray i’m getting a test to pass
Aaron Lun (14:19:43): > :party_parrot:
Kevin Rue-Albrecht (14:20:14): - File (PNG): Screenshot 2018-08-05 19.19.51.png
Kevin Rue-Albrecht (14:20:42): > quick one. these are the commands when no selection is fed to a customDataPlot
Kevin Rue-Albrecht (14:20:53): > used to be NULL
Kevin Rue-Albrecht (14:21:09): > is it overkill to test_identical the two commands?
Aaron Lun (14:22:02): > Probably. Just check that the plot command has the selection.
Aaron Lun (14:22:11): > i.e., matches"PCA2"
Aaron Lun (14:22:33): > It’s no longerNULLto allow user-defined functions to do their own thing when no selection is supplied.
Kevin Rue-Albrecht (14:22:47): > yup, i saw that, neat
Aaron Lun (14:22:49): > Maybe it does something sensible. Or it just prints a smiley face.
Aaron Lun (14:22:54): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (14:23:03): > do we have code for parrots?
Aaron Lun (14:24:16): > there was code for a rotating double helix somewhere
Kevin Rue-Albrecht (14:26:08): > I personally like the datasaurus:http://blog.revolutionanalytics.com/downloads/DataSaurus%20Dozen.gif - Attachment: Attachment
Kevin Rue-Albrecht (14:39:17): > Uh. Best one for last intest_custom.R..test_that(".make_customDataPlot responds to downsampling", {I don’t think it does, does it?
Aaron Lun (14:40:57): > No, not anymore.
Aaron Lun (14:41:34): > I’m going home now.
Kevin Rue-Albrecht (17:57:14): > just FYI, I’ve added a dummy custom stat function intest_custom.R. However, I can’t see aiSEE:::.make_customStatTablefunction yet. Too tired to look around for one. Let’s catch up tomorrow
Aaron Lun (18:16:48): > There is none. Tables don’t get the privilege of having their own functions. They’re made on the spot iniSEE.
2018-08-06
Aaron Lun (13:12:49): > okay, back to this.
Aaron Lun (13:12:58): > Had enough of working onbeachmatfor the time being.
Aaron Lun (13:16:02): > Talk to me bro
Kevin Rue-Albrecht (13:27:29): > Walking home. Got about 20min, but no keyboard, so bear with me hehe
Kevin Rue-Albrecht (13:27:48): > I think I left it at 12 unit tests failing
Kevin Rue-Albrecht (13:28:10): > I stopped when it was time to figure out how customStatTables work
Aaron Lun (13:28:23): > I’m fixing the code tracker right now, but I won’t touch the tests.
Kevin Rue-Albrecht (13:28:36): > Ok
Kevin Rue-Albrecht (13:31:33): > I’ll have a proper look at the tables tonight for a bit .
Kevin Rue-Albrecht (13:32:42): > I naively made up a function that makes a single column data frame full of ones for the tests, but I need to figure out how to plug that back into the UI
Kevin Rue-Albrecht (13:36:48): > On a separate note, now I’m just blowing my mind wondering whether it’d be possible to generate UI elements from parsing the definition of a user defined function
Kevin Rue-Albrecht (13:37:01): > The text field bugs me a little
Aaron Lun (13:37:39): > You’d have to set up the observers…. which I guess is possible at function start.
Kevin Rue-Albrecht (13:44:32): > … and all the optional arguments in the memory… argh my head hurts:sweat_smile:
Kevin Rue-Albrecht (13:44:55): > Good as it is, on a first instance
Aaron Lun (13:45:38): > The real problem is that the UI changes when you change the function.
Aaron Lun (13:45:49): > As in, it would need to change when you change the function.
Aaron Lun (13:46:05): > This would be krraaazzzy
Aaron Lun (13:48:31): > You would get:face_with_open_mouth_vomiting:
Kevin Rue-Albrecht (13:49:45): > Sounds like a roller coaster. I like roller coasters:smile:
Kevin Rue-Albrecht (13:51:51): > An alternative being to add one type of custom panel by custom function
Kevin Rue-Albrecht (13:52:13): > In that case the observers of each custom panel wouldn’t have to change
Kevin Rue-Albrecht (13:52:24): > Anyway, just a thought exercise
Aaron Lun (13:56:28): > That’s possible, though it means that the end-users cannot create more custom panels.
Aaron Lun (13:56:38): > Because their corresponding function would have to be fixed at app creation.
Aaron Lun (14:03:14): > Okay, code tracking is fixed and works for custom plots, but that probably broke a bunch of tests somewhere - can I leave that to you? Sorry
Aaron Lun (14:13:04): > Getting back to custom panels, the only thing that really needs some thorough testing is.text2args, which needs to give sensible output no matter what crap the user puts in.
Aaron Lun (14:13:21): > We should probably strip whitespace at the front, for example.
Aaron Lun (14:13:42): > Not sure about whitespace at the back, as this could be valid, but it could also confuse the hell out of people.
Kevin Rue-Albrecht (17:48:02): > holy crap i almost messed the tests in custom
Kevin Rue-Albrecht (17:49:22): > speaking of which: is the cache going to be reintroduced in the custom panels? i got the nose so close to it that I can’t take a step back right now to think about it
Kevin Rue-Albrecht (17:53:13): > we gotta stop with those renamings: i was about to remove some checks because I didn’t realisememory$customColPlotwas also nowmemory$customDataPlot
Aaron Lun (18:05:05): > well, it’s not a column plot anymore, as you can plot anything.
Aaron Lun (18:05:29): > And the cache will not be reintroduced; read the second half of the custom vignette for a better approach.
Aaron Lun (18:06:20): > There is also currently a bug when you delete custom panels; the selection link updater doesn’t work properly. I’ll fix this tomorrow.
Kevin Rue-Albrecht (18:07:01): > ok thanks for the pointer
Kevin Rue-Albrecht (18:07:19): > I’ll keep going tomorrow as well. Tired, and I need my head tomorrow morning
Kevin Rue-Albrecht (18:07:36): > g’night
Aaron Lun (18:09:05): > au reviour
2018-08-07
Aaron Lun (05:04:37): > Fixed.
Aaron Lun (05:04:54): > Onto my bigbeachmattest refactoring. I’ll leave the rest here to you.
Kevin Rue-Albrecht (05:06:49): > Cool, thanks. I’ll get to it as soon as i can
Kevin Rue-Albrecht (17:11:15): > 9 unit tests to go
Kevin Rue-Albrecht (17:24:55): > remind me to take a 6-month sabbatical next time we need to update the unit tests:stuck_out_tongue:
Kevin Rue-Albrecht (18:14:50): > I like the smell of a successful build in the morning. Going to sleep now The last build running on Travis was fine on my laptop.
2018-08-08
Aaron Lun (04:37:56): > sweet
Aaron Lun (05:17:44): > reading a thesis.
Aaron Lun (05:17:47): > PLease distract me
Aaron Lun (05:17:48): > :fast_parrot:
Kevin Rue-Albrecht (05:23:02): > I was actually wondering. Any chancedownsampleMatrixcould take weights by genes (e.g. gene length)?
Aaron Lun (05:27:43): > Not sure I know how to implement weighted downsampling without replacement… so no.
Kevin Rue-Albrecht (05:30:15): > :sad-parrot:
Aaron Lun (05:58:25): > oh I didn’t know we had that
Aaron Lun (05:58:42): > anyway why do you even need that
Federico Marini (11:12:12): > chiming in after the holidays AND the first day of school for the twins
Federico Marini (11:12:21): > Sorry for missing out the latest rounds
Federico Marini (11:12:45): > We’ll have an almost plenary iSEE meeting at the end of the month in heidelberg, BTW
Federico Marini (11:13:05): > if Kev is also there for the SOUND consortium workshop, it’s all of us:slightly_smiling_face:
Charlotte Soneson (11:14:04): > :+1:btw, did you get some sort of confirmation when you registered for that? Someone asked me today since they hadn’t received any email.
Aaron Lun (11:19:44): > ¯*(ツ)*/¯
Aaron Lun (11:20:00): > Well, I’m giving a workshop and you’re giving a talk, so it should be pretty damn obvious.
Charlotte Soneson (11:22:06): > What should be obvious?
Charlotte Soneson (11:22:25): > I was just wondering whether Federico got a confirmation when he registered.
Aaron Lun (11:22:54): > oh
Aaron Lun (11:22:58): > I thought you were talking about us.
Aaron Lun (11:23:02): > I didn’t register.
Aaron Lun (11:23:05): > Was I meant to?
Charlotte Soneson (11:23:09): > Ah no. Me neither.
Aaron Lun (11:23:28): > Imagine turning up and then being told to go away
Aaron Lun (11:23:32): > Well, free trip to heidelberg
Aaron Lun (11:23:38): > never been to germany before
Charlotte Soneson (11:23:39): > But a former colleague to me did, and he didn’t get a confirmation, so he asked me if I knew whether he should receive one.
Charlotte Soneson (11:24:13): > Well, Heidelberg is nice, you could always take a sightseeing tour:slightly_smiling_face:
Aaron Lun (11:27:51): > I was watching an anime where an arc was set in heidelberg castle.
Aaron Lun (11:28:00): > Character got killed there.
Aaron Lun (11:28:05): > By a serial killer
Aaron Lun (11:28:17): > I don’t think I’ll take anything out of that.
Charlotte Soneson (11:29:36): > Ehm, I think you’ll be fine…never been to the castle though, so I can not promise anything.
Aaron Lun (11:34:58): > Anyway, people should check outcustom.Rmdin thealt_custombranch. Might be:exploding_head:
Aaron Lun (11:40:11): > @Kevin Rue-AlbrechtI like the righteous 4-space indenting.
Aaron Lun (12:02:10): > And@Federico Mariniwhere’s my colors
Kevin Rue-Albrecht (13:24:58) (in thread): > Coming back in the chat. Dammit y’all be at the same meet? shame I’m missing that
Kevin Rue-Albrecht (13:25:16) (in thread): > Curious about the location of EuroBioc2018 now
Kevin Rue-Albrecht (13:26:50): > I knew I was alone in the unit tests, so I took my chance. Only for the files where there were issues though. Haven’t bothered indenting all the tests
Aaron Lun (13:27:14): > I’m done with the tests on my end.
Aaron Lun (13:27:22): > There should be a decent coverage bump.
Aaron Lun (13:27:51): > There’s one more feature that should just involve adding some switches to the code tracker functions, but I’m out of juice for today.
Federico Marini (15:47:18) (in thread): > did not. not sure we do get any
Federico Marini (15:47:31) (in thread): > sounded somewhat liberal in the participation
Federico Marini (15:47:38) (in thread): > I guess some email will come sometime
Federico Marini (15:47:56) (in thread): > normally I need such a thingy to justify the reimbursements
Federico Marini (15:48:42): > about the castle, well, there’s not so much to see
Federico Marini (15:49:03): > you can always take a guided tour and that person will tell you things for 2+ hours
Federico Marini (15:49:12): > but it is quite compact
Federico Marini (15:49:19): > there’S the story of the Perkeo
Federico Marini (15:49:29): > the drunken wine-serving dwarf
Federico Marini (15:49:42): > and the hexenbiss-door
Federico Marini (15:51:55) (in thread): > munich
Federico Marini (15:52:20) (in thread): > that was communicated i guess at the end of eurobioc2017?
Federico Marini (15:52:39) (in thread): > if not munich, well, somewhere in continental europe
Kevin Rue-Albrecht (16:33:34) (in thread): > Ah right. It could have been Munich. I didn’t write it down
Kevin Rue-Albrecht (16:59:38): > holy molly: “Merging #227 into master will increase coverage by 6.67%.” !
Kevin Rue-Albrecht (16:59:49): > grand total of 85.86%
Kevin Rue-Albrecht (17:00:06): > the last % are always the hardest
2018-08-09
Aaron Lun (12:41:42): > alt_customis ready to merge. Please check it over tonight or tomorrow.
Aaron Lun (12:42:45): > Note that youmustbe running BioC-devel with an up-to-date version ofSingleCellExperiment.
Kevin Rue-Albrecht (12:47:38): > that’s my motto in life
Aaron Lun (12:48:49): > Whoever has push access to BioC should push it tomorrow.
Kevin Rue-Albrecht (12:49:01): > Will do.
2018-08-10
Aaron Lun (10:57:01): > Dammit@Charlotte Sonesonwe’re tied for papers this year. I have to crank up the paper mill FTW.
Kevin Rue-Albrecht (17:47:00): > Eh.@Aaron LunSorry for checking this late, but are you sure it’s ready for merge/push upstream? > I’ve just tested the custom table, and it works fine, but I don’t see any doc about it in thedefaultshelp page
Kevin Rue-Albrecht (17:47:24): > also it’ll need a version bump to release it
Kevin Rue-Albrecht (17:57:36): > oh, and reminder to self: I think the selectize widgets for custom plots/tables should be called “Receiverow|columnselection from:” rather than both being the same “Receive selection from:”
2018-08-11
Kevin Rue-Albrecht (06:50:25): > ahh for the last one I see that it’s a single functioncreate_selection_param_box_define_choicesthat generates the label. Nevermind, users can figure out that the items are different in the drop down
Aaron Lun (07:14:20): > Well, you can add an argument to change the name if you like.
Kevin Rue-Albrecht (07:52:45): > Right. Aside, from that you’re sure we’re good to bump up and release? No more small details to fix, or do we want to use that new devel release to find out those details to fix?
Kevin Rue-Albrecht (08:00:57): > I’ve just put in the version bump now.
Kevin Rue-Albrecht (08:03:37): > <!channel>I need to go out now for a few hours, so I won’t be home when Travis completes. If anyone wants to merge or add last minute changes, I’ll push upstream when I get home.
Aaron Lun (08:50:46): > I do want to break up theDefaults.Rdso that it isn’t so monolithic.
Kevin Rue-Albrecht (08:58:45): > Cool. Go for it. I’ve only updated/added a section for custom plots. Now that I think of it I think the Arguments column may not be described for custom panels. It’s shopping time here, so you’ve got a couple of hours at least
Aaron Lun (11:53:06): > Done.
Kevin Rue-Albrecht (12:48:39): > Alright, I’ll give it a check and get that pushed upstream asap
Aaron Lun (12:54:37): > If you’ve some time, I’d suggest moving the description of how to customize the initialization of the app into a separate vignette, between the ECM and custom vignettes.
Aaron Lun (12:55:00): > It probably warrants a more detailed explanation than the current small section in vignette no 4.
Kevin Rue-Albrecht (12:58:55): > “Changing the default start configuration” ?
Aaron Lun (13:03:23): > Yeah
Kevin Rue-Albrecht (13:08:59): > ok. I just started by fixing our bibtex reference. Vignette 4. has “K. et al. 2018” which was weird
Kevin Rue-Albrecht (13:10:28): > so.. number the “VignetteIndexEntry” is enough then? i see that your careful naming alphabetical of vignettes has gone sideways with “custom” (3) coming before “ecm” (2)
Aaron Lun (13:11:59): > That’s okay. Onlybasicneeds to be alphabetically first, as it generatessce.rds.
Kevin Rue-Albrecht (14:03:47): > hm.. soDefaults.Rdcompletely disappeared… so I create a “further reading” section that’s a bit more verbose
Kevin Rue-Albrecht (14:05:57): > do you think it’d be worth maintaining an index of all the individual?defaults*man page as a simple list of links in a master?defaults? Or does the vignette that I’ve just added fulfil that job?
Aaron Lun (14:16:43): > I get the feeling we have more to talk about in the initialization vignette.
Aaron Lun (14:16:51): > Perhaps we should discuss different aspects of visualization.
Aaron Lun (14:16:57): > e.g., setting data parameters
Aaron Lun (14:17:03): > or setting aesthetic parameters
Aaron Lun (14:17:08): > or setting selection parameters (tricky)
Aaron Lun (14:17:20): > and the ability to specify both integers and character fields for some fields.
Kevin Rue-Albrecht (14:18:39): > True. I was feeling a bit cheap to have a vignette with a single section (hence adding the second “further reading”… tongue in cheek), but I wasn’t super inspired about example right now
Kevin Rue-Albrecht (14:18:59): > We could build more examples into this vignette over time
Kevin Rue-Albrecht (14:19:47): > Perhaps not a roadblock to release the updated code. And users might send some useful feedback our way about things they’d like to know
Aaron Lun (14:20:30): > ¯*(ツ)*/¯
Aaron Lun (14:20:33): > Well do what you can
Aaron Lun (14:20:45): > A few more sections for the time being would flesh out the vignette
Kevin Rue-Albrecht (14:20:46): > don’t get me wrong, I agree with you that there’s a lot to talk about on the subject
Kevin Rue-Albrecht (14:21:40): > Thinking about it, it’s actually the amount of information to put in there that’s giving me a white page block, in a way
Kevin Rue-Albrecht (14:21:54): > good thing that you sent that “e.g.” list above
Aaron Lun (14:33:31): > Currently eating this two-year-past-expiry-date chocolate… let’s see how it goes.
Kevin Rue-Albrecht (15:02:36): > how’s the sugar rush?
Kevin Rue-Albrecht (15:02:38): > fermented?
Aaron Lun (15:04:29): > Dunno. It’s dark chocolate.
Aaron Lun (15:04:50): > If it went off (if such a thing is possible with dark chocolate), I’ll be throwing up in a few hours.
Kevin Rue-Albrecht (15:08:02): > I’m not a dark chocolate fan. Not sure I’d taste the difference. > Joke aside, it’s chocolate, I’m sure it’s fine
Kevin Rue-Albrecht (15:09:00): > When I was a kid we had Easter chocolates (the ones in shapes of rabbits, etc) that we had for a few years, and turned them into a mousse someday:slightly_smiling_face:
Kevin Rue-Albrecht (15:18:38): > haven’t started expanding the configure vignette yet, but I just added thesample assay plotin the basic vignette, and internally documented the argument I added to customize the label of row/column selectize
Aaron Lun (15:19:45): > :party_parrot:
Kevin Rue-Albrecht (15:19:46): > Dinner time now. Just to refresh the topic, I like the list that you made, it could make a nice 3-4 sections in the configure vignette:
Kevin Rue-Albrecht (15:20:08): > - data params > - aes params > - selection params > - fields accepting both integer/character
Aaron Lun (15:21:34): > sounds sensible to me. I’m going home now, I’ve had enough of dealing with the Tanay group’s stupid Mars-seq pipeline.
2018-08-12
Kevin Rue-Albrecht (06:39:04): > I’m still only in the data params, the X axis is a mess to explain/demonstrate, I might be overkill, but I hope to push something in the next 20min to give you an idea
Kevin Rue-Albrecht (06:53:50): > pushing the data args section, if you can give it a quick look
Aaron Lun (06:56:45): > I note that you’re referring to fields as"FIELD_NAME", e.g.,"XAxisColData". The quotes are not necessary, it makes it difficult to distinguish between the field name and the field value.
Aaron Lun (06:57:39): > Spacing between examples would help in the X-axis section; also use a numbered list to link the scenarios to the examples (e.g., 3i, 3ii).
Aaron Lun (06:58:50): > And don’t bother going through all panel types - just use the feature assay as an example, as that’s probably the most complicated panel type in terms of data parameters.
Kevin Rue-Albrecht (07:14:39): > Good first point. I’ll clean that up. > I don’t want to go through each panel type, but I thought it useful to demonstrate all major scenarios (Y, X, reduceDim slot, assay slot)
Kevin Rue-Albrecht (07:15:13): > Once I cover them in data params, I expect that should help me go faster through the aesthetic ones
Kevin Rue-Albrecht (07:19:56): > speaking of which, I think data params are covered
Kevin Rue-Albrecht (07:20:38): > Hopefully I can finish the other section in the next couple of hours, but I need to do some other stuff too this afternoon:neutral_face:
Kevin Rue-Albrecht (07:21:40): > how’d you like “Those parameters are accessible at runtime in the eponym collapsible box.“:dealwithit-parrot:
Kevin Rue-Albrecht (07:32:18): > and we know have “chocolate3” in our vignette
Aaron Lun (07:33:31): > eponymous
Kevin Rue-Albrecht (07:33:55): > oww, I Googled in wrong place then
Kevin Rue-Albrecht (09:13:33): > moving into the “selection params” section
Kevin Rue-Albrecht (09:13:54): > good practice even to remind myself… but what a pain
Aaron Lun (09:15:05): > going for a walk now, back in a bit.
Kevin Rue-Albrecht (09:15:14): > I’m getting a bit lazy and using colData plots for most examples, but I guess making the point for one should give users a fair idea. It would probably be even more confusing to jump between plot types, between paragraphs
Kevin Rue-Albrecht (09:15:15): > ok
Kevin Rue-Albrecht (10:23:40): > Alright, whenever you come back and have some time, could you please check what i just pushed?
Kevin Rue-Albrecht (10:23:52): > I gave a few examples of good and bad integer/character inputs
Kevin Rue-Albrecht (10:24:10): > but i need to move to my other things in the meantime
Aaron Lun (10:24:17): > “Setting the X-axis”; a and b are not on new lines?
Aaron Lun (10:24:31): > hm.
Aaron Lun (10:24:39): > does it show up like that in the HTML?
Kevin Rue-Albrecht (10:24:44): > i thought they were on separate lines in the HTML
Aaron Lun (10:24:54): > okay, maybe it’s just like that on github then.
Kevin Rue-Albrecht (10:24:57): > they’re sub points of 3, with an extra level of indentation
Kevin Rue-Albrecht (10:25:05): > oh, didn’t check GH
Aaron Lun (10:25:15): > I think GH only recognizes 2-space indents, not sure.
Aaron Lun (10:25:25): > you’ll need to look at the markdown rules for sublists.
Kevin Rue-Albrecht (10:25:26): - File (PNG): Screenshot 2018-08-12 15.25.14.png
Aaron Lun (10:25:35): > okay
Aaron Lun (10:25:42): > “examples 1 to 3”
Kevin Rue-Albrecht (10:25:52): > :sweat:
Aaron Lun (10:26:05): > geneidcolumn needs space.
Aaron Lun (10:27:31): > Also, I wouldn’t confuse people with the integer/character choice until you get to it. Just introduce it as characters, and then mention that they can be integers when you get to the section.
Kevin Rue-Albrecht (10:28:05): > makes me realise i have a “Neertheless”
Aaron Lun (10:29:46): > Also, for your description of 3b; I don’t think this is correct, as theXAxisFeatNamegets overridden by whatever is selected in the table (as a tablealwaysstarts with a selection). You could check this by asking for a differentXAxisFeatNamethan what is in the table’s initial selection.
Kevin Rue-Albrecht (10:30:06): > ah good point, I wasn’t sure about that one, so i set it in both
Kevin Rue-Albrecht (10:30:44): > or i might just leave theXAxisFeatNameblank in that case then
Aaron Lun (10:31:57): > yeah
Kevin Rue-Albrecht (10:32:55): > it’s in those moments where I feel like we’ve created a monster, so hard to keep track of it all.. which basically shows how nicely automated everything is ^^
Aaron Lun (10:33:30): > Also; you have a lot of “orphan sections”. This refers to cases like this: > > # Primary header 1 > > This is an orphan section > > ## Secondary header 1 > > blah blah > > ## Secondary header 2 > > blah blah balh > > # Primary header 2 > > This is okay, as there are no subheaders > > # Primary header 3 >
Aaron Lun (10:33:54): > So named as it doesn’t have a direct parent on the same level as the other text.
Aaron Lun (10:34:23): > This is problematic from a reference perspective, as you can’t refer to the orphan section without referring to the entire section encapsulated by the primary header.
Kevin Rue-Albrecht (10:35:38): > hm.. i see what you mean, although my idea was to refer specifically to the ‘orphan’ text, that is a kind of prelude to the subsections. I can see how it’s confusing though
Aaron Lun (10:36:00): > If you need a prelude, say “overview” or something.
Kevin Rue-Albrecht (10:36:11): > duh. fair enough
Aaron Lun (10:36:15): > And then put all the “refer to?blahDefaultsfor more details” in there.
Aaron Lun (10:36:23): > Rather than mentioning it in each subsection.
Kevin Rue-Albrecht (10:36:56): > Ah. some of them are a bit more specific, e.g. dimRed
Aaron Lun (10:37:02): > redDim.
Kevin Rue-Albrecht (10:42:45): > alright, that should be sort of fixed now
Aaron Lun (13:06:44): > talk to me
Aaron Lun (14:15:15): > well, I’m happy for you to push. Maybe others can give the new vignette a look-over.@Federico Marini@Charlotte Soneson.
Kevin Rue-Albrecht (14:24:45): > I was going to ask about pushing upstream. I think the vignette is a good first draft, while we figure out what else to tell users about
Charlotte Soneson (14:47:08): > I can have a look tomorrow, we’ve been away for a few days and are still traveling (without laptop:slightly_smiling_face:)
Aaron Lun (14:57:11): > my condolences
Charlotte Soneson (14:58:56): > It’s nice, I can recommend it!
Aaron Lun (15:00:50): > where did you go?
Charlotte Soneson (15:01:20): > We’ve been a few days in Rome and then for a wedding in the east of Italy.
Aaron Lun (15:02:13): > I see.
Kevin Rue-Albrecht (15:15:17): > Alright then. Pushing can wait tomorrow. I can look at things to add to the README in the meantime
Aaron Lun (16:54:38): > Did you not get it, or are you letting it slide?
Kevin Rue-Albrecht (17:03:45): > sorry: get what? let what slide?
Aaron Lun (17:17:43): > the pun
Kevin Rue-Albrecht (17:18:58): > omg i was blind:sunglasses:it was right there. I think I’m getting too used to it ^^
Aaron Lun (17:29:46): > :fiesta_parrot:
2018-08-13
Kevin Rue-Albrecht (02:34:35): > https://twitter.com/jefworks/status/1028741712896843778?s=12 - Attachment (twitter): Attachment > Too many #singlecell points in your figures? Crashing Illustrator? Check out ggrastr for #ggplot2 #rstats by @PetukhovViktor to rasterize just the points layer. Maintain vectorized text for publication quality. Ideal for >10000 points. https://github.com/VPetukhov/ggrastr #dataviz #bigdata https://pbs.twimg.com/media/DkbS-NXUUAAE-0T.jpg
Aaron Lun (04:11:29): > Is this faster, or just lighter?
Kevin Rue-Albrecht (04:16:11): > Dunno, haven’t tried yet. My feeling from the heatmap geom_raster is that computation is similar, but that rendering is faster for the raster. > On a different point, Wolfgang Huber makes a comment in the same thread that geom_hexbin (hexagonal bins) is a lighter alternative to geom_point when the objective is merely to show the density of points.
Aaron Lun (04:19:48): > we can’t use bins, colours are allocated for something other than density.
Kevin Rue-Albrecht (04:19:59): > .. and that too
Kevin Rue-Albrecht (05:08:39): > Using the ?iSEE example > `> data.frame( > reducedDim(sce, “TSNE”), > colData(sce) > ) > > # geom_point > t1 <- Sys.time() > ggplot(ggdata, aes(X1, X2, color=driver_1_s)) + > geom_point() > t2 <- Sys.time() > difftime(t2, t1) > > Time difference of 0.496145 secs > # geom_point > t1 <- Sys.time() > ggplot(ggdata, aes(X1, X2, color=driver_1_s)) + > geom_point_rast() > t2 <- Sys.time() > difftime(t2, t1) > > Time difference of 1.036268 secs >
Kevin Rue-Albrecht (05:10:03): > Similar story if writing to file: > > tf <- tempfile(fileext = ".pdf") > > # geom_point > t1 <- Sys.time() > gg <- ggplot(ggdata, aes(X1, X2, color=driver_1_s)) + > geom_point() > ggsave(tf, gg) > t2 <- Sys.time() > difftime(t2, t1) > > # geom_point > t1 <- Sys.time() > ggplot(ggdata, aes(X1, X2, color=driver_1_s)) + > geom_point_rast() > ggsave(tf, gg) > t2 <- Sys.time() > difftime(t2, t1) >
Aaron Lun (05:12:19): > That’s a decent boost in speed.
Aaron Lun (05:12:35): > Is it a swap-in replacement?
Kevin Rue-Albrecht (05:13:05): > hang on, a boost? I’m computing theduration, so the raster is actually takingmoretime
Kevin Rue-Albrecht (05:13:49): > I just did it with the TCGA tour data: > > > gg <- ggplot(ggdata, aes(X1, X2, color=CancerType)) + > + geom_point() > > tf <- tempfile(fileext = ".pdf") > > > > # geom_point > > t1 <- Sys.time() > > gg <- ggplot(ggdata, aes(X1, X2, color=CancerType)) + > + geom_point() > > ggsave(tf, gg) > Saving 11.8 x 8.6 in image > > t2 <- Sys.time() > > difftime(t2, t1) > Time difference of 0.7534039 secs > > > > # geom_point > > t1 <- Sys.time() > > ggplot(ggdata, aes(X1, X2, color=CancerType)) + > + geom_point_rast() > > ggsave(tf, gg) > Saving 11.8 x 8.6 in image > > t2 <- Sys.time() > > difftime(t2, t1) > Time difference of 3.054513 secs >
Aaron Lun (05:13:56): > oh, sorry
Aaron Lun (05:14:08): > didn’t notice your direction
Aaron Lun (05:14:43): > well, didn’t know what it was computing indifftime
Aaron Lun (05:14:56): > Okay then. That’s that I guess.
Kevin Rue-Albrecht (05:15:06): > so, to answer your question, it seems that: > - output file = lighter (obviously) > - duration = longer
Aaron Lun (05:15:20): > not useful to us.
Kevin Rue-Albrecht (05:15:35): > No worries. difftime is probably not the most elegant way, but I was too lazy to run microbenchmark
Charlotte Soneson (07:21:05): > New vignette looks nice! Maybe we should refer to the “Display panel settings” tool somewhere, since that will show all the current settings?
Kevin Rue-Albrecht (07:27:08): > Oww. I forgot all about that one. Can you add it to the vignette please?
Charlotte Soneson (07:27:40): > Yes, in a little while. Have to rush off to a meeting now.
Aaron Lun (08:07:23): > Request getting this done ASAP; I will implement another feature in the afternoon.
Charlotte Soneson (08:22:23): > Ok, for now I just added a sentence in the vignette pointing out theDisplay panel settings.
Charlotte Soneson (08:22:30): > Pushed to thealt_custombranch
Aaron Lun (08:23:38): > Looks good.@Kevin Rue-Albrechtmerge and push when ready.
Federico Marini (08:24:55): > Aaron, we need as of now 9 colors for our palette
Federico Marini (08:25:04): > do you think of adding more panel types?
Aaron Lun (08:25:24): > Yes, one more - a colStatTable.
Aaron Lun (08:25:32): > For consistency with rowStat + featAssay, we should have colStat + sampAssay.
Federico Marini (08:25:54): > so 10. Ok, I’d try for that same purpose to get somewhat paired colors
Aaron Lun (08:26:10): > I wouldn’t worry too much about the pairing, but 10 colours total, yes.
Federico Marini (08:26:18): > at least for those panels which have counterparts
Aaron Lun (08:26:32): > If anything, I would suggest a difference in the colours for row- and column-based panels
Aaron Lun (08:26:41): > or for tables vs plots
Federico Marini (08:26:50): > that was my first choice
Aaron Lun (08:26:50): > Well, a lot of choices, i’ll leave it to you.
Federico Marini (08:27:02): > some kind of “axis” to distinguish better
Aaron Lun (08:27:42): > :parrotconga:
Aaron Lun (08:28:07): > :parrotconga::parrotconga::parrotconga::parrotconga::parrotconga::parrotconga:
Kevin Rue-Albrecht (08:32:55): > CouldRColorBrewer::display.brewer.pal(12, "Paired")be a good start?
Aaron Lun (08:36:22): > I would prefer that we avoid blue, as that’s the colour of the dashboard.
Aaron Lun (08:36:29): > Unless we can change that…
Kevin Rue-Albrecht (08:37:40): > Arf.. forgot about that. Nevermind then. We’ll just have to pick and match colors as we’ve done so far.
Federico Marini (08:42:30): > @Aaron Lunand@Charlotte Sonesonwhere are you staying while in HD?
Federico Marini (08:42:46): > the dashboard color can be changed
Federico Marini (08:43:00): > has still less options than the panels
Kevin Rue-Albrecht (08:43:17): > Without further ado, merging and pushing upstream then. > I meant to add a little example of text field for the CUSTOM_SUMMARYstat_fun, but I don’t want to trigger another two builds (branch and PR) just for that
Federico Marini (08:43:38): > https://rstudio.github.io/shinydashboard/appearance.html
Kevin Rue-Albrecht (08:43:38): > is that alright?
Federico Marini (08:43:59): > fine for me. same goes for the vignettes, nice job on that
Kevin Rue-Albrecht (08:44:09): > cool it’s going in then
Aaron Lun (08:44:32): > IT’s fines.
Aaron Lun (08:44:56): > Yes, changing to white would give us more options
Charlotte Soneson (08:45:24) (in thread): > I have been booked athttps://villamarstall.de/
Aaron Lun (08:47:41): > @Federico MariniI’m staying the Neidermeyer hotel? Or something like that.
Kevin Rue-Albrecht (08:48:31): > Alright, it’s upstream. I checked by cloning the bioc repo in a new folder
Kevin Rue-Albrecht (08:48:39): > behold version 1.1.4
Aaron Lun (08:49:26): > you can close the two related issues.
Kevin Rue-Albrecht (08:50:40): > done
Federico Marini (08:53:58) (in thread): > hm cant find it
Federico Marini (08:54:29) (in thread): > I was looking for something cheap because I’m anyway one hour away of train
Aaron Lun (08:55:00) (in thread): > NH Heidelberg
Aaron Lun (08:55:07) (in thread): > Bergheimer Str
Federico Marini (08:57:06) (in thread): > ok
Federico Marini (08:57:09) (in thread): > down on the river
Federico Marini (08:57:21) (in thread): > or somewhat close well
Federico Marini (08:58:16) (in thread): > the villa marstall where@Charlotte Sonesonis staying is out of my pockets:smile:
Federico Marini (08:58:37) (in thread): > classy
Federico Marini (08:58:55) (in thread): > I’ll go low budget because otherwise they send me home commuting:smile:
Charlotte Soneson (08:59:01) (in thread): > :slightly_smiling_face:
Charlotte Soneson (08:59:06) (in thread): > I did not book myself…
Aaron Lun (08:59:12) (in thread): > ’cause the swiss have money pouring out of their ass
Aaron Lun (08:59:22): > :party_parrot:
Charlotte Soneson (08:59:24) (in thread): > It’s the Germans who are paying this one…
Federico Marini (08:59:43) (in thread): > I’ll go to lotte
Charlotte Soneson (09:00:15) (in thread): > Huh?
Federico Marini (09:00:15) (in thread): > it is a hostel but since I am no speaker i do not need my bubble to reach inner peace:smile:
Aaron Lun (09:00:23) (in thread): > what you got them to pay?
Aaron Lun (09:00:26) (in thread): > No one offered to pay for me
Charlotte Soneson (09:00:29) (in thread): > :slightly_smiling_face:
Charlotte Soneson (09:00:35) (in thread): > I should make sure I do that then
Federico Marini (09:00:40) (in thread): > wut
Federico Marini (09:00:46) (in thread): > unverschämtheit!
Federico Marini (09:01:00) (in thread): > anyway, I’ll stay in the central altstadt as well
Charlotte Soneson (09:01:03) (in thread): > Well, they invited me to talk and booked me at a hotel.
Federico Marini (09:06:05) (in thread): > Aaron, after all John picked you for the looks. Something’s gone wrong here?:stuck_out_tongue:
Federico Marini (09:11:32): > <!channel>I’m putting together a collection of links for the colors, so we can maybe vote upfront/preselect some candidates?
Federico Marini (09:12:03): > To make it a personal thing, a nice option would be based on true life
Federico Marini (09:12:38): > Alva (a.k.a. kid nr.3, biological batch#2) has this toy:https://www.lunatur.com/Fuer-Kinder/Regenbogenwippe - Attachment (lunatur.com): Regenbogenwippe > Die Regenbogenwippe ist ein Spielmöbel mit vielen unterschiedlichen Einsatzmöglichkeiten: Zum Draufklettern und Hochziehen, zum Runterrutschen, als Babywippe oder Puppenbett, zum wild Schaukeln oder Durchkriechen, als Stehschaukel, als Kaufladen und zum Sitzen.
Aaron Lun (09:12:38) (in thread): > Dammit
Aaron Lun (09:12:42) (in thread): > well, whatever
Aaron Lun (09:12:51): > sweet
Federico Marini (09:12:53): > and I really fancy the color palette
Aaron Lun (09:13:04): > I want one as well
Federico Marini (09:13:09): > especially since we can “name” most of them
Aaron Lun (09:13:15): > probably a bit too big for one tho
Federico Marini (09:13:22): > without using “firebrick2”
Kevin Rue-Albrecht (09:13:36): > works as a color paletteandan office chair? I’m in!
Federico Marini (09:14:14): > click on pic #5
Federico Marini (09:14:21): > and see if you can code like that:smile:
Kevin Rue-Albrecht (09:15:28): > looks to me that the training set is im’balanced” (ah-ah)
Federico Marini (09:15:33): > ahaha
Federico Marini (09:15:38): > jeeZuZ
Federico Marini (09:15:49): > the hot air did permanent damages:smile:
Federico Marini (09:19:44): > anyway if you also like that, I can take a pic and then extract the hex colors
Kevin Rue-Albrecht (09:20:26): > that’s cool with me
Kevin Rue-Albrecht (09:20:50): > plus I can see 2 blues and 2 greens, with address your row/column pairings
Federico Marini (09:21:28): > yup. put on top a couple of greys, if we want, and we cover the two top PCs
Federico Marini (09:23:39): > but still, for completeness:
Federico Marini (09:23:43): > 1-https://github.com/gadenbuie/ggpomological2 -https://github.com/jaredhuling/jcolors3 -https://github.com/johannesbjork/LaCroixColoR4 -https://github.com/kwstat/pals5 -https://github.com/m-clark/visibly6 -https://www.awwwards.com/sites/coloured-lines7 -https://github.com/EmilHvitfeldt/r-color-palettes8 -http://paletton.com/#uid=1000u0kllllaFw0g0qFqFg0w0aF9 -https://learnui.design/tools/data-color-picker.html#divergent10 -http://tools.medialab.sciences-po.fr/iwanthue/
Federico Marini (09:23:58): > some of them are “just” tools
Federico Marini (09:26:09): > with my favs being (on top of the csaw- whoops, see-saw)
Federico Marini (09:28:07): > lacroix_palette(type = “paired”) > the cols from the awwwards > from here,https://github.com/EmilHvitfeldt/r-color-palettes-> the tableau cyclic - Attachment (GitHub): EmilHvitfeldt/r-color-palettes > r-color-palettes - Comprehensive list of color palettes available in r
Kevin Rue-Albrecht (09:33:33): > holy crap some people really have time on their hands ^^
Aaron Lun (09:46:39): > well, just choose a colour scheme with a good spread
Federico Marini (11:33:00): > for you to try out
Federico Marini (11:33:08): > > colpal_1 <- c( > "#F2B701", > "#E47E04", > "#DB0230", > "#B00258", > "#7C378A", > "#3565AA", > "#07A274", > "#7BB854", > "#515356", > "#B0B0B0") > scales::show_col(colpal_1) >
Federico Marini (11:33:22): > later tonight I’ll try to take a pic of the csaw:slightly_smiling_face:
Federico Marini (11:34:49): > or this way as well
Federico Marini (11:34:54): > > n <- length(colpal_1) > image(1:n, 1, as.matrix(1:n), col = colpal_1, ylab = "", xaxt = "n", > yaxt = "n", bty = "n") >
Aaron Lun (12:37:17): > … and it’s mostly done. Check outcolstattab. Many tests will break; any volunteers?
Aaron Lun (13:06:43): > Testing of panel deletion is especially necessary, as these get the least tested in general.
Aaron Lun (13:28:18): > Okay, I’ve made an issue. Someone can take over from here.
Federico Marini (15:52:59): > liking this one as well:http://projects.susielu.com/viz-palette?colors=[%22#9e0142”,“#d53e4f”,“#f46d43”,“#fdae61”,“#fee08b”,“#e6f598”,“#abdda4”,“#66c2a5”,“#3288bd”,“#5e4fa2”]&backgroundColor=“white”&fontColor=“black”
Federico Marini (15:54:04): > should be something like spectral with 10 values
2018-08-14
Federico Marini (04:47:26): > here’S the rainbow one from the seesaw
Federico Marini (04:47:37): > > colpal_csaw <- rep(c( > "#9E0D12", > "#CA892F", > "#C7AA0E", > "#91AB86", > "#176637", > "#8BBFE4", > "#0372D3", > "#8A2D66", > "#C93E9B" > ),2) > scales::show_col(colpal_csaw) > n <- length(colpal_csaw) > image(1:n, 1, as.matrix(1:n), col = colpal_csaw, ylab = "", xaxt = "n", > yaxt = "n", bty = "n") >
Federico Marini (04:48:08): > with the source for full reproducibility:smile:
Federico Marini (04:48:09): - File (JPEG): 20180814_102116.jpg
Federico Marini (04:48:25): - File (JPEG): 20180814_102137.jpg
Federico Marini (04:51:35): > this one’s 9 cols, add a gray on top
Aaron Lun (04:58:28): > hah lol
Aaron Lun (08:00:11): > can you instantiate the colours in the app and give us a screenshot?
Federico Marini (16:31:51): > on it
Federico Marini (17:00:52): > this should give you an idea
Federico Marini (17:01:27): > with 8cols + 2 grays - not shown here - File (PNG): Screen Shot 2018-08-14 at 23.00.27.png
Federico Marini (17:03:49): > and this is the csaw inspired - File (PNG): Screen Shot 2018-08-14 at 23.03.06.png
2018-08-15
Kevin Rue-Albrecht (03:10:04): > I prefer the top one (marked by a reaction)
Kevin Rue-Albrecht (03:11:13): > I find the colors are sharper than the second (csaw) version. Especially the second row of csaw itches me a bit
Federico Marini (04:59:31): > the order can be changed, yet the first one convinces me more as well
Kevin Rue-Albrecht (05:01:41): > Actually, based on true life, that just came back to mind:https://www.youtube.com/watch?v=LFBOeXP5lH4 - Attachment (YouTube): Learn colors with THE WOODEN XYLOPHONE TRUCK | Color Learning Videos for Kids
Kevin Rue-Albrecht (05:02:49): > In particular, this one has 8 colors too:https://images-na.ssl-images-amazon.com/images/I/51CnVKynH%2BL._SL500_AC_SS350_.jpg - Attachment: Attachment
Federico Marini (05:05:38): > for the seesaw I could try with photos with better lighting
Federico Marini (05:05:50): > but I think the upper screenshot could be good
Kevin Rue-Albrecht (05:06:58): > Yup. Your top one is definitely a good start. I just offered the glockenspiel for fun and maybe color refinement if you like
Aaron Lun (05:25:12): > Hm.
Aaron Lun (05:25:35): > I tend to agree
Aaron Lun (06:31:01): > Anyway, those with some spare time should try to address the issues I just raised.
2018-08-16
Aaron Lun (06:29:34): > I don’t think we need half the stuff in the xtreme vignette, as we have full examples in the paper.
Aaron Lun (06:39:01): > Well, if no one says anything, I’m going to delete the xtreme vignette.
Aaron Lun (06:42:06): > Well, it’s done.
Aaron Lun (06:45:49): > Made a PR, but it will break as the latestBiocStyleis not yet available.
Kevin Rue-Albrecht (06:52:26): > Ok, i’ll look at it asap. Can’t remember what exactly was in xtreme, was it the Dropbox and EHub things?
Aaron Lun (06:53:01): > Yeah, and mass cytometry. But both cytof and Ehub now live as full-fledged examples iniSEE2018.
Aaron Lun (06:53:15): > Probably a link to the paper and github repo wouldn’t hurt somewhere.
Aaron Lun (06:54:47): > Also; I notice that row-based plots don’t have faceting or shape options in the UI.
Aaron Lun (06:54:54): > Something to add to thecolstattabbranch.
Kevin Rue-Albrecht (07:01:39): > Oh right. Didn’t think of that. I would have imagined that I did that, at the time
Kevin Rue-Albrecht (07:03:39): > I guess the Dropbox section of the xtreme vignette disappears altogether? That fell perfectly under the definition of an “extreme” use case. I don’t know where else it could be reintroduced
Aaron Lun (12:44:30) (in thread): > … and I just realized that’s because the row data doesn’t have any categorical factors. Clever girl.
Aaron Lun (12:45:00): > At any rate… only tests remain incolstattab.
Kevin Rue-Albrecht (13:52:05): > ok
Aaron Lun (13:59:02): > as in, only tests remainto do.
Aaron Lun (13:59:24): > But you can see how cool it is now to select samples and have them light up on the red dim plot.
Kevin Rue-Albrecht (13:59:47): > I understood that. Sorry for the short reply. I’m actually running check right now, to give myself a starting point
Aaron Lun (14:03:31): > :shuffle-parrot:
Kevin Rue-Albrecht (14:03:54): > omg Undocumented arguments in documentation object ‘.create_selection_param_box’ > ‘...’ ‘field’:stuck_out_tongue_winking_eye:
Aaron Lun (14:04:01): > NO
Aaron Lun (14:04:07): > :sad-parrot:
Kevin Rue-Albrecht (14:33:55): > I have a love/hate view of.setup_memory
Kevin Rue-Albrecht (14:36:11): > it’s doing so many smart things, but my god there are so many tests to update when adding a new argument
Aaron Lun (14:37:01): > yeah…. well…
Kevin Rue-Albrecht (14:37:02): > not too much thinking to do though, once the first test done, it’s all mechanical
Aaron Lun (14:37:37): > There’s probably a few coverage gaps due toif/elses for row vs column stat tables
Aaron Lun (14:37:47): > That should be fairly fixable; just duplicate all tests for the row table.
Kevin Rue-Albrecht (14:37:59): > still, I’m sure we’re saving time on the long run with the unit tests
Kevin Rue-Albrecht (14:38:38): > well.. at the moment, I’m just adding the new required argument and test data where necessary to get the tests to pass again
Kevin Rue-Albrecht (14:38:50): > once they pass, we can see where the coverage gaps are
Aaron Lun (14:39:02): > yep
Kevin Rue-Albrecht (15:15:26): > @Aaron Luni’m facing the unit test “.make_featAssayPlot works for YAxis set to Feature name” line 590 intest_plotting.R
Kevin Rue-Albrecht (15:16:03): > we don’t support anymore character gene names, do we?
Kevin Rue-Albrecht (15:16:32): > it fails because of line 183 inplotting.R > > Error in sprintf("plot.data <- data.frame(Y=assay(se, %i, withDimnames=FALSE)[%i,], row.names = colnames(se))", : > invalid format '%i'; use format %s for character objects >
Kevin Rue-Albrecht (15:16:54): > obviously,%icannot be substituted by a character value
Aaron Lun (15:58:27): > That’s right
Aaron Lun (15:58:46): > By the time it gets into that function, it should have been coerced into an integer.
Aaron Lun (15:59:01): > This is necessary to allowwithDimnames=FALSEto avoid copying the matrix when names get assigned.
Aaron Lun (16:00:07): > That’s a more general point - allassaycalls should havewithDimnames=FALSEwhere possible.
Kevin Rue-Albrecht (16:44:13): > I’m back - thanks
Kevin Rue-Albrecht (16:45:19): > i confused myself for a second there, but ok: as you said,at this point(when the internal.make_featAssayPlotis called) all those fields should already be converted to integers
Kevin Rue-Albrecht (16:46:37): > before that, users can still pass toiSEE(*args)someDataFrames that contain character values
Kevin Rue-Albrecht (16:54:02): > nice catch.gene_axis_label->.feature_axis_label
Kevin Rue-Albrecht (16:54:38): > 4 tests left to go over here
Aaron Lun (17:04:27): > FINISH HIM:-1:
Kevin Rue-Albrecht (17:11:06): > line 85 oftest_selection_links, I’m getting a warning : > > Warning message: > In .encode_panel_name(names) : NAs introduced by coercion >
Kevin Rue-Albrecht (17:11:24): > not crashing the test, but I’m just trying to understand whether it can be avoided
Kevin Rue-Albrecht (17:12:11): > overallexpect_erroris getting the error that it wants, but I’m curious about the warning
Kevin Rue-Albrecht (17:14:34): > arghh i think it’s simply related to the error, “whee!” being invalid raises a warning before causing the error
Kevin Rue-Albrecht (17:15:22): > yep, nevermind: “whee!” doesn’t like being converted to integer
Kevin Rue-Albrecht (17:26:53): > drum rolls I think I’m attacking the last filetest_track_code
Kevin Rue-Albrecht (17:27:30): > it’s so sad that I know by heart the full list of arguments toiSEE(...)in order
Kevin Rue-Albrecht (17:35:33): > moment of truth: running check locally
Kevin Rue-Albrecht (17:42:45): > damn, another 2 tests fail:sob:
2018-08-17
Aaron Lun (08:07:35): > .sanitize_memoryhas been fixed.
Kevin Rue-Albrecht (08:08:15): > Cool. Thanks. Does that mean you updated the unit tests at the same time? or do they need update now?
Aaron Lun (08:08:49): > I did it for sanitize_memory, but coverage is not fixed for the new sampAssay or colStat panels.
Kevin Rue-Albrecht (09:40:26): > I really need to get some stuff done over here:sweat:can’t we just PR the current state and open an issue for extra coverage?
Aaron Lun (09:42:42): > Or we could just wait until you’re free.
Aaron Lun (10:37:23): > currently grinding throughtest_table_links.R.
Kevin Rue-Albrecht (12:22:17): > Thanks for that
Kevin Rue-Albrecht (12:23:07): > Damn I left my teeth unchecked too long: earned myself a follow up treatment:sleepy:
Aaron Lun (12:39:09): > lol
Aaron Lun (12:44:45): > I think we’ll be done if we have some tests forsampAssayPlots.
Aaron Lun (21:09:24): > flossing technique
Aaron Lun (21:09:31): > oops wrong text field
2018-08-18
Kevin Rue-Albrecht (05:05:17): > more like a garbage collection error, but yeah that’s the idea
Kevin Rue-Albrecht (05:05:57): > ps: i won’t be online today, a friend is visiting the O place
Aaron Lun (08:54:56): > WHAT
Aaron Lun (11:30:30): > God, two out of four tasks done today
Aaron Lun (11:30:34): > Let’s FINISH THIS
Aaron Lun (13:38:50): > ugh, 3 out of four, but I’m out of juice.
2018-08-19
Aaron Lun (08:08:39): > talk to me people
Kevin Rue-Albrecht (09:48:03): > alive and well over here
Kevin Rue-Albrecht (10:12:46): > i’m going to check how we’re doing with regard to tests
Aaron Lun (10:33:55): > We’re all good, except for coverage.
Kevin Rue-Albrecht (10:37:29): > that’s what i remembered you saying
Kevin Rue-Albrecht (10:37:43): > i’m just updating a whole bunch of packages, including BiocStyle
Kevin Rue-Albrecht (10:38:10): > nice addition to the Biocpkg(…) macro
Aaron Lun (10:41:51): > Note that the current BiocStyle is buggy, Martin just fixed it today.
Kevin Rue-Albrecht (10:42:37): > commitmissing closing ')' in output2 hours ago
Kevin Rue-Albrecht (10:44:44): > anyway, buggy or not, the edit you made inbasic.Rmdrequired me to update anyway, otherwise the vignette crashes R CMD check
Aaron Lun (11:36:32): > how are we going over there?
Kevin Rue-Albrecht (11:37:26): > juggling a couple of things, I left CMD check running after adding colStatTable to test_dynamicsUI.R and didn’t check on it since
Kevin Rue-Albrecht (11:38:02): > woops, crash in test_ui.R
Kevin Rue-Albrecht (11:53:34): > fixing a sprintf that is supposed generate an error message… except that it has a typo.. so it does throw an error itself xD
Aaron Lun (11:53:40): > lol
Kevin Rue-Albrecht (11:53:53): > stop(sprintf("'%s' is not a recognized panel mode"), mode)
Kevin Rue-Albrecht (11:54:10): > , modeis outside the parentheses ofsprintf^^
Kevin Rue-Albrecht (11:55:24): > in a similar topic, let me dig up a recent tweet that I liked for the concept
Kevin Rue-Albrecht (11:56:11): - File (PNG): Pasted image at 2018-08-19, 4:55 PM
Aaron Lun (11:56:57): > ha lol
Kevin Rue-Albrecht (12:01:48): > damn I crashed my R session because i resized the Viewer panel while rendering the coverage report
Kevin Rue-Albrecht (12:02:45): > if you didn’t know it yet,covr::package_coveragereturns a value that can be plugged into theirreport(...)function, which opens it as a Shiny app
Kevin Rue-Albrecht (12:03:00): > perhaps even more convenient that the codecov website
Kevin Rue-Albrecht (12:09:28): > Pushing another mighty 3 extra lines of coverage as soon as it checks out locally
Aaron Lun (12:09:36): > oh yeah
Kevin Rue-Albrecht (12:10:42): > .choose_linkwhich probably only you and me know about, considering that across all the tests written, it never happens that “chosen” is not in “available” ^^
Aaron Lun (12:12:05): > Yeah, I wonder if that scenario is protected against somewhere upstream.
Kevin Rue-Albrecht (12:15:48): > in any case, you’ll excuse me for not whipping up realistically named examples: i just used dummy values likelettersfor the unit tests
Kevin Rue-Albrecht (12:17:35): > and i guess that all the initialsanitization- especially.name2indexprobably removes invalid values frommemory
Kevin Rue-Albrecht (12:18:04): > which sends only clean stuff down to.choose_link
Aaron Lun (12:18:06): > … probably?
Aaron Lun (12:18:15): > well, yes.
Aaron Lun (12:18:19): > for the links.
Kevin Rue-Albrecht (12:18:44): > https://twitter.com/i/status/1030453054540275713 - Attachment (twitter): Attachment > Your code before refactoring
Aaron Lun (12:25:08): > ha
Kevin Rue-Albrecht (12:25:39): > the caption is so perfect ^^
Kevin Rue-Albrecht (12:26:51): > alright, that one next single line of extra coverage should push it from 99.82 -> 100%
Kevin Rue-Albrecht (12:26:58): > for dynamicUI
Aaron Lun (12:27:31): > sweet
Kevin Rue-Albrecht (12:28:01): > “sample names are generated for internal iSEE metadata if colnames(se) is NULL”
Aaron Lun (12:28:31): > huh?
Kevin Rue-Albrecht (12:28:32): > probably relevant for 10x data
Kevin Rue-Albrecht (12:29:04): > > if (is.null(colnames(se))) { > out$sample_names <- sprintf("Sample %i", seq_len(ncol(se))) > } >
Kevin Rue-Albrecht (12:29:36): > I guess you hadDropletUtils::read10xCountsin mind there
Aaron Lun (12:29:42): > Well… not really.
Aaron Lun (12:29:52): > If there’s no sample names, the selecti’zes won’t work.
Aaron Lun (12:30:09): > you’ll just end up with an empty dropdown - as in, empty strings.
Kevin Rue-Albrecht (12:30:59): > yeah, don’t worry I understood the why
Kevin Rue-Albrecht (12:31:20): > I was just wondering how often people would not have named their samples
Kevin Rue-Albrecht (12:31:57): > but then I remembered the DropletUtils import function, which makes sense as the barcode are kinda useless colnames
Kevin Rue-Albrecht (12:32:33): > the one scenario where they can be interesting being for sequence bias, I imagine
Kevin Rue-Albrecht (12:32:55): > anyway, dynamicsUI is fully covered now
Kevin Rue-Albrecht (12:33:43): > I should probably look at places where I can cover more than 1 line per unit test now, if we want a real coverage bump ^^
Kevin Rue-Albrecht (12:34:24): > lasso.Rcould be fun
Kevin Rue-Albrecht (12:35:01): > any chance you could tackle you subsetPointsbyGrid? 58 lines over there
Kevin Rue-Albrecht (12:35:26): - File (PNG): Screenshot 2018-08-19 17.35.06.png
Aaron Lun (12:35:59): > I think we should just get it to a state where we can merge the PR without codecov complaining.
Kevin Rue-Albrecht (12:36:18): > ok, well, lasso would definitely do the trick
Aaron Lun (12:36:46): > I suggest checking that the output of theevalis the same as a direct call tomgcv::in.out
Kevin Rue-Albrecht (12:38:53): > Sure. I guess the most ‘painful’ part will just be to simulate a click event. I just need to run a Shiny app that print one for that
Aaron Lun (12:39:30): > Yep. I think there’s also some checks for unclosed lassos that probably need to be triggered.
Aaron Lun (12:39:59): > The annotation tests are also probably easy; just run them, there’s not really anything else to do.
Kevin Rue-Albrecht (12:40:37): > True. I just haven’t looked much at that code at all so far, tbh. Probably not as painful as I would imagine I’m sure
Kevin Rue-Albrecht (12:48:48): > Hmm, when simulating a click, should I bother making also the slots that are unused in our functions? > A real click looks like: > > List of 6 > $ x : num -4.44 > $ y : num 12.1 > $ mapping:List of 2 > ..$ x: chr "X" > ..$ y: chr "Y" > $ domain :List of 4 > ..$ left : num -14.1 > ..$ right : num 10.9 > ..$ bottom: num -12 > ..$ top : num 16.4 > $ range :List of 4 > ..$ left : num 38.7 > ..$ right : num 379 > ..$ bottom: num 466 > ..$ top : num 23.8 > $ log :List of 2 > ..$ x: NULL > > but we never usedomain,range, andlogfor instance. Shall I omit them from the test object, to minimize confusion?
Aaron Lun (12:51:20): > There’s no need to simulate individual clicks.
Kevin Rue-Albrecht (12:51:33): > i do need one for.update_lasso
Aaron Lun (12:51:34): > Just make a lasso in the app, pull out the lasso object, and paste it into the test.
Aaron Lun (12:51:38): > Oh.
Aaron Lun (12:51:56): > um…
Aaron Lun (12:52:17): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (12:53:22): > I’ll figure out the MWE
Kevin Rue-Albrecht (13:03:19): > ok, I think I nailed.update_lasso. Checking now
Kevin Rue-Albrecht (13:05:10): > yup. 60% of lasso.R covered
Kevin Rue-Albrecht (13:05:25): > just need to coverlassoPointsnow. The mgcv in.out thing
Aaron Lun (13:13:34): > hold on, the annotation functions should be run in the examples. Does the cov not count the examples?
Kevin Rue-Albrecht (13:13:41): > nope
Kevin Rue-Albrecht (13:13:49): > only tests/testthat
Kevin Rue-Albrecht (13:14:40): > because coverage only cares about unit tests
Kevin Rue-Albrecht (13:17:01): > argh.. i need to throw in some faceting to test all of lasso.R:sob:
Aaron Lun (13:30:20): > I’ve covered annotation and subsetting.
Kevin Rue-Albrecht (13:33:03): > ok perfect.
Kevin Rue-Albrecht (13:33:37): > for a weird reason, when I try to pull it warns me about conflicts with my newtest_lasso.R?!
Aaron Lun (13:33:50): > ¯*(ツ)*/¯
Aaron Lun (13:33:53): > Didn’t touch that.
Aaron Lun (13:34:00): > Well, it didn’t even exist.
Aaron Lun (13:34:07): > double ¯*(ツ)*/¯
Kevin Rue-Albrecht (13:34:08): > exactly
Kevin Rue-Albrecht (13:34:11): > meh
Aaron Lun (13:34:20): > I’m going to have dinner now, back in a bit.
Kevin Rue-Albrecht (13:34:23): > stash, pull, apply, and commit
Kevin Rue-Albrecht (13:34:24): > sure
Kevin Rue-Albrecht (13:34:53): > thanks for the help
Kevin Rue-Albrecht (14:24:10): > this is the link where codecov shows the lines of the PRdiffthat are not covered by unit tests:https://codecov.io/gh/csoneson/iSEE/compare/1b8758c62c8812edfee7d36ce2ede757531b9db2…e33d4cb8dd328ddd136696789541087c31cc7c34/diff
Kevin Rue-Albrecht (14:25:35): > don’t mix up the light red (lines deleted in thediff, but covered by test) with the dark red ones, which are the new lines that don’t have coverage, or existing ones that have lost coverage
Kevin Rue-Albrecht (14:34:36): > I’m taking care ofplotting.R. I saw that.make_sampAssayPlotisn’t covered. That should make codecov happier about covering the diff
Aaron Lun (16:32:15): > Happy to see the merge
Kevin Rue-Albrecht (17:05:22): > Alright. Obviously, not 100% the diff is covered by unit tests, but we pushed it over the line with today’s work. I guess the rest can wait the next PR
Kevin Rue-Albrecht (17:14:43): > PR merged. Please everyonepull masterand branch again.
Aaron Lun (18:59:45): > @Federico Marini@Charlotte Sonesonshould play around with the new features before we push to devel.
Aaron Lun (18:59:52): > Also we should make a decision about these damn colors.
2018-08-20
Kevin Rue-Albrecht (04:35:32): > I thought@Federico Mariniwas going to ask the kids their opinion on the colors:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (04:36:42): > but yeah, once Fede and Charlotte give some feedback we should open a new branch, bump, and I can take the opportunity to add a couple more tests for thesampAssayPlotthat I didn’t get in this weekend
Federico Marini (05:56:04): > For the colors: more than asking the kids I wanted to take ou t the rainbow of the csaw
Federico Marini (05:56:18): > but it seems the first one I gave you to check seems fine?
Aaron Lun (08:48:18): > Yes, I would go with the first set of colors. See if you can change the dashboard to white as well.
Aaron Lun (08:48:32): > Assuming white isn’t one of the rainbow colors.
Aaron Lun (08:48:39): > Which it shouldn’t be anyway.
Federico Marini (10:05:41): > it’s not
Federico Marini (10:05:57): > apart from the ones you saw in the snapshot, i got 2 greys
Federico Marini (10:06:03): > for the custom plots
Federico Marini (10:06:45): > which should appear only for power users anyway, i.e. they could ideally even find out where the color settings are, and “fix” this if they like something else better
Federico Marini (10:07:26): > re: background, here are the options
Federico Marini (10:07:27): > https://rstudio.github.io/shinydashboard/appearance.html
Federico Marini (10:08:11): > skins are more limited in terms of choices
Federico Marini (10:08:56): > btw - with 3/4 of the team meeting next week, we should focus also on the legere topic of finalizing the sticker
Aaron Lun (10:13:43): > White makes the most sense for the background.
Aaron Lun (10:14:00): > Not sure we can easily use user-supplied colours… I guess it’s possible, but a bit painful IMO.
Federico Marini (11:50:06): > Immatry with white
Federico Marini (12:03:10): - File (PNG): Screen Shot 2018-08-20 at 18.02.36.png
Federico Marini (12:03:18): > skin’sblackbut so it is
Federico Marini (12:03:35): > I’m not the the biggest fan of the bordered elements on top
Aaron Lun (12:03:35): > that looks okay
Federico Marini (12:03:40): > but looks clean
Federico Marini (12:04:05): > or at least, no conflicts with shades of blue
Federico Marini (12:05:04): > up to check the rest of the things
Federico Marini (12:05:15): > but it is pushed tonewcolors
Federico Marini (12:05:56): > heading home:wink:
Kevin Rue-Albrecht (12:40:07): > I can get used to these colors
Aaron Lun (13:01:45): > Looks good enough. The borders are a bit weird, but hey.
Kevin Rue-Albrecht (13:09:09): > … we should parameterize the panel colors:wink:
Kevin Rue-Albrecht (16:58:45): > A round of pints goes to whoever unit tests.regenerate_unselected_plot
Kevin Rue-Albrecht (16:59:12): > I’m going through the rest ofiSEE-extras.R
Aaron Lun (17:51:43): > I don’t think that’s possible, it relies on asession.
Kevin Rue-Albrecht (17:57:22): > I saw that:wink:
Kevin Rue-Albrecht (18:01:26): > alright, going to recharge, I just got the table links and.get_selected_pointsleft for tomorrow
Kevin Rue-Albrecht (18:01:44): > saving a bit of energy for the rest of the week
Aaron Lun (18:06:12): > k
2018-08-21
Kevin Rue-Albrecht (05:47:34): > Something’s messed up with the selection link ofColumn statistics table 1
Kevin Rue-Albrecht (05:49:18): > first, it’s label says “receive row selection from”
Kevin Rue-Albrecht (05:49:39): > that’s probably on me, although I was pretty sure i checked them all when I did the update
Kevin Rue-Albrecht (05:50:10): > but then it also doesn’t seem to react to selections in the source panels
Aaron Lun (05:50:39): > As in, to or from?
Kevin Rue-Albrecht (05:50:51): > the receiving link of Column statistics table
Kevin Rue-Albrecht (05:52:11): > column stat table receiving a ”row selection” - File (PNG): Screenshot 2018-08-21 10.51.07.png
Kevin Rue-Albrecht (05:52:29): > (I’m onnewcolors, btw)
Aaron Lun (05:55:22): > probably something in the observers.
Aaron Lun (05:55:30): > Or something in the set up of the selection links.
Kevin Rue-Albrecht (05:55:36): > I think the latter
Aaron Lun (05:55:39): > does the graph respond correctly?
Kevin Rue-Albrecht (05:55:46): > as in, we’re not even offering the right links
Kevin Rue-Albrecht (05:56:11): > links don’t work either, I guess the graph building function figures out that the selected link is invalid
Kevin Rue-Albrecht (05:56:47): > I mean the graph is devoid of links if I select either options screenshot’ed above
Kevin Rue-Albrecht (05:57:36): > I might have got it:if (mode %in% linked_table_types) {
Kevin Rue-Albrecht (05:57:48): > with linked_table_types ="rowStatTable" "colStatTable"
Kevin Rue-Albrecht (05:58:23): > oh no wait, that’s only the cause of the wrong label (“receive row selection from)”
Kevin Rue-Albrecht (05:58:34): > not the cause of the wrong choices in the selectize
Aaron Lun (06:05:26): > I don’t understand, those choices are the right choices.
Aaron Lun (06:05:34): > oh wait
Aaron Lun (06:05:45): > yes, right, okay.
Aaron Lun (06:05:51): > Should just be a UI thing, then.
Kevin Rue-Albrecht (06:12:09): > Actually, I can only find the observer setup for rowStatTable line 777 of dynamicUI, even though it states “tables” in a general way > > # Linked selection choice observers for the tables. ---- > max_tabs <- nrow(pObjects$memory$rowStatTable) > for (id in seq_len(max_tabs)) { > local({ > mode0 <- "rowStatTable" >
Aaron Lun (06:13:21): > in dynamicUI?
Aaron Lun (06:13:26): > You don’t mean in iSEE-main.R?
Kevin Rue-Albrecht (06:13:41): > oh sorry yeah
Aaron Lun (06:14:12): > that’s probably a mistake then.
Aaron Lun (06:14:24): > Though I will have to look at it; give me 15.
Kevin Rue-Albrecht (06:14:50): > no worries, I was tracking down where the selectize is generated, but maybe it’s bigger than that
Kevin Rue-Albrecht (06:25:38): > I found it, I think
Kevin Rue-Albrecht (06:25:43): > .create_selection_param_box_define_choices(mode, id, param_choices, .selectByPlot, row_selectable, source_type)
Kevin Rue-Albrecht (06:25:55): > row_selectableis applied to both rowStat and colStat tables
Aaron Lun (06:26:08): > Yeah, that would be right.
Kevin Rue-Albrecht (06:30:34): > ok so, now, the box is correct, selection outgoing from colStatTable work (e.g. highlight a sample in a redDimPlot), but incoming selections still don’t work (e.g. restricting the table to samples selected elsewhere)
Aaron Lun (06:54:14): > can you push what you have?
Kevin Rue-Albrecht (06:56:21): > Sure. Just give me a minute to finish 4-space indenting dynamicUI… I started:sweat_smile:
Aaron Lun (07:39:58): > can’t see the change to black…
Kevin Rue-Albrecht (07:40:13): > to black?
Aaron Lun (07:40:47): > oh wait, that was fed’s commit.
Kevin Rue-Albrecht (07:40:48): > arf you mean the useful vs the indent changes?
Kevin Rue-Albrecht (07:40:53): > ah
Aaron Lun (07:41:00): > Anyway, bug should be fixed.
Kevin Rue-Albrecht (07:41:07): > oh cool
Kevin Rue-Albrecht (07:45:12): > oh then, just had to loop over the two types of tables? damn the level of refactoring in this package ^^
Aaron Lun (17:24:10): > Crunched 2 reviews today.
Aaron Lun (17:24:43): > grep("boss", "Aaron")returns 1.
Kevin Rue-Albrecht (17:25:40): > greplwould say “TRUE that”
Kevin Rue-Albrecht (17:26:46): > I’m not too ashamed of myself either - finished covering iSEE-extras this evening
Kevin Rue-Albrecht (17:27:19): > I’ve googled a bit about Shiny sessions, but I can’t find anything about simulating/emulating one
Kevin Rue-Albrecht (17:30:32): > Well done us - File (PNG): Screenshot 2018-08-21 22.30.02.png
Kevin Rue-Albrecht (17:53:57): > Btw, I noticed thatiSEE:::didn’t seem needed to test internal function. > I might have been the one starting that initially to benefit of the autocompletion of RStudio, but i recently realised thatdevtools::load_all()makes all the internal functions available in the global environment. > I’ve justed pushed a commit thatsedremoves alliSEE:::from tests
Kevin Rue-Albrecht (18:00:30): > can always revert it if I missed something, but it passedcovr::package_coverageon my laptop after the update
Aaron Lun (18:03:38): > Ah - I don’t think that’s a good idea, as I test viasource.
Aaron Lun (18:04:05): > You see those commands at the top of some of the files? Those are from me.
Kevin Rue-Albrecht (18:05:02): > ahhh ok
Kevin Rue-Albrecht (18:05:25): > somehow Travis looks happy too - it’s just completing the build
Aaron Lun (18:05:42): > Besides, it’s generally good to clarify that things are internal and meant to be so.
Kevin Rue-Albrecht (18:06:03): > I was wondering about that clarification of internals too
Kevin Rue-Albrecht (18:06:13): > alright, no worry. easy to revert
Aaron Lun (18:06:36): > It’s equivalent to theusing namespace std;arguments that pop up on C++ threads on Stackoverflow.
Kevin Rue-Albrecht (18:07:36): > pushing the revert now
Kevin Rue-Albrecht (18:07:52): > note the bonus: a test to coverzzz.R
Kevin Rue-Albrecht (18:08:18): > Poor little one. Always the last one:cry:
Kevin Rue-Albrecht (18:10:05): > with that, the last two big ones are heatmap.R and plotting.R with 81 and 100 lines left to cover, respectively
Kevin Rue-Albrecht (18:10:45): > picture to illustrate - File (PNG): Screenshot 2018-08-21 23.10.19.png
2018-08-22
Aaron Lun (09:31:33): > you guys should check outhttps://github.com/LTLA/AntiMagic2018for some laughs - Attachment (GitHub): LTLA/AntiMagic2018 > AntiMagic2018 - A cathartic diatribe against the use of the MAGIC algorithm.
Kevin Rue-Albrecht (09:34:09): > I like the title “Avoiding illusions” … (MAGIC) ^^
Kevin Rue-Albrecht (09:36:31): > what the hell fig.3:laughing:
Aaron Lun (09:44:00): > figure 3? the p-value histogram? I would have thought figure 4 was more WTF.
Aaron Lun (09:44:09): > Well, they’re all pretty screwy
Federico Marini (09:46:45): > (I would have an idea for a logo if you ask - pretty obvious if you played magic the gathering back in the days)
Aaron Lun (09:47:31): > 5 to cast multicolor?
Aaron Lun (09:47:44): > or maybe it’s colourless… because it’s an artifact.
Federico Marini (09:49:23): > GENAU
Federico Marini (09:49:32): > call your repo counterspell:smile:
Federico Marini (09:49:58): > omg
Federico Marini (09:50:02): > that game was great
Federico Marini (09:50:04): > https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwifx_zS5oDdAhUIZFAKHWcRBD8QjRx6BAgBEAU&url=https%3A%2F%2Fmagiccards.info%2F4e%2Fen%2F65.html&psig=AOvVaw2Xv4JwOK-xAltUsSV5q-qy&ust=1535032182212570
Federico Marini (09:59:42): > wow
Federico Marini (09:59:50): > fig 4b is the birth of a dragon hatchling
Aaron Lun (10:00:07): > 1UU 2/1 flying
Federico Marini (10:00:19): > (yes, I am thirsty about the last season of GoT)
Kevin Rue-Albrecht (10:00:32): > pardon my French: I first saw a snail
Kevin Rue-Albrecht (10:00:37): > :sweat_smile:
Aaron Lun (10:01:11): > 2G 1/3 tap to ignore damage dealt by creatures this turn.
Federico Marini (10:02:04): > 1G1R1B1B1W 5/5 Trample, 0: ignore the latest dana pe’er paper
Federico Marini (10:02:21): > or her bollocks idea to patent tsne on scrnaseq
Aaron Lun (10:03:56): > 10 to cast for a 5/5 trample?
Federico Marini (10:04:11): > just one of each:smile:
Aaron Lun (10:04:21): > Then you just denote it as GRUBW
Federico Marini (10:04:23): > but hey look at the special skill
Aaron Lun (10:04:25): > yeah
Federico Marini (10:04:49): > use it multiple times a turn
Federico Marini (10:04:50): > :stuck_out_tongue:
Federico Marini (10:05:02): > pity it does not work retroactively
Federico Marini (10:05:20): > because, HA HA HA, that would also be MAGIC
Federico Marini (10:05:32): > ok, I need a break
Kevin Rue-Albrecht (13:51:38): > Anyone interested in some CSS/JS tonight?
Aaron Lun (13:52:18): > Why, what are you trying to do?
Kevin Rue-Albrecht (13:52:59): > I’ve added some CSS to make the custom arguments text fields monospace
Kevin Rue-Albrecht (13:53:13): > but:
Kevin Rue-Albrecht (13:54:24): > 1) it’s currently done through a call toshiny::tags, which may be better defined through a CSS file stored God knows where
Aaron Lun (13:54:33): > no, that’s fine.
Aaron Lun (13:54:42): > as long as it’s done in thedynamicUI.R?
Aaron Lun (13:54:53): > and not floating around iniSEE-main.R.
Kevin Rue-Albrecht (13:55:33): > 2) at the moment, it’ll affectalltext fields, I was kind of hoping of a way to restrict to inputs named “*_Arguments”
Kevin Rue-Albrecht (13:56:01): > argh, it’s in iSEE-main, at the start of thedashboardBody
Aaron Lun (13:57:51): > ¯*(ツ)*/¯
Aaron Lun (13:57:58): > I don’t know enough CSS to deal with the second
Kevin Rue-Albrecht (13:57:59): > actually, it’s sitting with another pre-existingtags$headcall
Aaron Lun (13:58:22): > the first is not really a problem. The second sounds like you’ll have to define a new CSS class for this field.
Kevin Rue-Albrecht (13:59:02): > i’ve dealt with CSS classes a while back, but never within Shiny apps
Kevin Rue-Albrecht (14:00:28): > Anyway, i don’t see another immediate use oftextareainputs. Maybe not worth the headache yet?
Aaron Lun (14:01:01): > ¯*(ツ)*/¯
Aaron Lun (14:01:11): > I think you want to deal with this now rather than later
Aaron Lun (14:01:27): > Because future you or me will be swearing at the current you or me.
Kevin Rue-Albrecht (14:01:36): > Now, second change of the day, probably more controversial, you’ll probably want to have a look at ffc6162eeb09d3e7b3bda554ea5ea792309445bd
Kevin Rue-Albrecht (14:01:47): > damn,https://github.com/csoneson/iSEE/commit/ffc6162eeb09d3e7b3bda554ea5ea792309445bd - Attachment (GitHub): add submit button to update custom arguments when ready, not while ty… · csoneson/iSEE@ffc6162 > …ping
Kevin Rue-Albrecht (14:02:10): > I’ve got a couple of reasons to suggest this
Aaron Lun (14:02:23): > No, I understand.
Kevin Rue-Albrecht (14:02:39): > first, I don’t like the plot refreshing and failing 3 times while I type
Aaron Lun (14:02:48): > However, this means that the observer needs to be modified.
Kevin Rue-Albrecht (14:02:53): > i’ve done it all
Kevin Rue-Albrecht (14:03:07): > second, i don’t think users will update the custom arguments frequently
Kevin Rue-Albrecht (14:03:54): > it’s more like “try a few different config, pick a favorite, close the collapsibleBox and only play with point selections”
Aaron Lun (14:03:56): > okay, the commit is fine.
Kevin Rue-Albrecht (14:04:51): > I left theforloop there in case the number of things to track increases again above 1
Kevin Rue-Albrecht (14:05:18): > but basically, i’m watching the button, and updating the arguments when clicked, if they changed.
Kevin Rue-Albrecht (14:05:29): > thanks for all the code infrastructure that was already there
Kevin Rue-Albrecht (14:08:02): > Back to the first topic, I don’t know (yet) how to assign a new class to one of Shiny’s input widgets, but I can look.
Kevin Rue-Albrecht (14:08:42): > … next time I’ll Google first… (https://rdrr.io/cran/shinyjs/man/classFuncs.html) - Attachment (rdrr.io): classFuncs: Add/remove CSS class in shinyjs: Easily Improve the User Experience of Your Shiny Apps in Seconds > Add or remove a CSS class from an HTML element. addClass adds a CSS class, removeClass removes a CSS class, toggleClass adds the class if it is not set and removes the class if it is already set. addCssClass, removeCssClass, and toggleCssClass are synonyms that may be safer to use if you’re working with S4 classes (since they don’t mask any existing S4 functions). If condition is given to toggleClass, that condition will be used to determine if to add or remove the class. The class will be added if the condition evaluates to TRUE and removed otherwise. If you find yourself writing code such as if (test()) addClass(id, cl) else removeClass(id, cl) then you can use toggleClass instead: toggleClass(id, cl, test()). CSS is a simple way to describe how elements on a web page should be displayed (position, colour, size, etc.). You can learn the basics at W3Schools.
Kevin Rue-Albrecht (14:31:50): > Here’s a fun one to put iniSEE_server(...):observeEvent(input$iSEE_info, {print(names(reactiveValuesToList(input)))})
Kevin Rue-Albrecht (16:34:58): > small victory: i disable the button to submit custom arguments when the text box matches current arguments
Aaron Lun (16:36:33): > That seems like overkill
Kevin Rue-Albrecht (16:37:27): > It was that or changing the color of the button. I think users should know when the box shows something different than the active parameters
Kevin Rue-Albrecht (16:38:12): > Though I also wonder how many observers the app can stand before going slow
Kevin Rue-Albrecht (16:40:23): > anyway, that’s a by-product of my search to add a CSS class to thetextareainputs… but shinyjs only works on the server side. I haven’t figure yet how to add a custom ‘static’ class , i.e. constant from start to end of the app
Aaron Lun (16:43:29): > I guess
Aaron Lun (16:44:36): > The heatmap multi-gene selectize would benefit from the same mechanism.
Kevin Rue-Albrecht (16:45:04): > Oh. Didn’t think of those yet. Fair enough.
Aaron Lun (16:45:13): > If people want to arrange the genes, they have to wait for the plot to update at every cycle.
Kevin Rue-Albrecht (16:46:20): > Speaking of overkill, it’s also possible to change the label of the button. e.g. “Submit” when it’s clickable… “Up to date” when it’s not?
Aaron Lun (16:48:26): > “Update” vs “Up to date”. Plus colour change. Plus non-selectable. That should make it… pretty damn obvious.
Kevin Rue-Albrecht (16:48:34): > lol
Aaron Lun (16:48:58): > If you do the same thing for the heatmaps, roll it out into a separate loop to avoid re-writing it.
Kevin Rue-Albrecht (16:49:56): > i’m just going through the options. Haven’t figured the button color yet, but a while back i was playing with bootstrap and just had to switch the button class
Kevin Rue-Albrecht (16:50:09): > a separate loop for… what? all the buttons?
Aaron Lun (16:51:09): > The button feature switcher, but I just realized that the observers won’t be the same. Better to have aniSEE-extrafunction that defines the switching button class.
Kevin Rue-Albrecht (16:52:29): > ok. sure
Kevin Rue-Albrecht (17:50:54): > Refactored the button toggle to a function. Another one that can’t be unit tested (shinjs requires running within a UI)
Kevin Rue-Albrecht (17:51:00): > Suggestions welcome for the name.disableButtonIf
Kevin Rue-Albrecht (17:51:28): > (updateActionButtonwas already taken)
Aaron Lun (17:57:12): > How does it detect thesession, then? It doesn’t take any arguments - it must pick it up from the environment, then.
Aaron Lun (18:09:59): > Also; a consequence of the need to “submit” means that any updates to the UI (e.g., adding a new plot) will discard any edits to the text area thathaven’tbeen submitted. This breaks the illusion that everything is reproducible from memory - a problem that is currently confined to the DataTables.
Kevin Rue-Albrecht (18:12:04): > I pass the session to the function
Kevin Rue-Albrecht (18:12:15): > seems to work by reference
Kevin Rue-Albrecht (18:13:00): > i’m not sure whether your last point is a problem
Aaron Lun (18:13:26): > Yes, it is, because you want things to be preserved across UI manipulations.
Aaron Lun (18:13:45): > The fact that DataTables revert to their original state is already pretty annoying.
Kevin Rue-Albrecht (18:13:52): > as far as I’m concerned, everything is reproducible from memory, as memory remembers what was submitted
Aaron Lun (18:14:26): > But the state of the custom panel shouldn’t be affected by UI manipulations elsewhere
Kevin Rue-Albrecht (18:14:29): > things that weren’t submitted are just hanging about in the aether, and as such they are not part of the reproducible part of the app
Aaron Lun (18:15:31): > That’s unexpected behaviour to me.
Aaron Lun (18:15:39): > It’s already bad enough with the DataTables.
Kevin Rue-Albrecht (18:15:51): > I actually haven’t checked.. do you mean that the textbox is reset when a new selection comes in?
Aaron Lun (18:16:20): > The textbox will be reset from memory when the UI is regenerated.
Aaron Lun (18:17:03): > This is non intuitive, especially given that everything else (including the panel open/close!) is preserved. Aside from the DataTables, of course, but I mentioned that already.
Aaron Lun (18:19:27): > To fix this, we will need to duplicate theArgsfield in memory, basically having one field for “visible args” and another field for “memory args”. The advantage is that you will be able to include a button that says “reset to last plot parameters” or something like that.
Kevin Rue-Albrecht (18:20:20): > yep, i entirely agree with the strategy you just outlined
Kevin Rue-Albrecht (18:21:14): > although i must say, I really don’t expect anyone but ultra-power users to complain about a thing like this
Kevin Rue-Albrecht (18:21:45): > btw, heat map “cluster button” is done
Aaron Lun (18:24:03): > Hold off on the heatmap until we get this thing sorted.
Aaron Lun (18:24:35): > In fact, it’s even harder to do this for heatmap clustering, because the only way to check if it’s already clustered is to cluster it again.
Aaron Lun (18:25:14): > It’s the heatmap feature list itself that I was referring to, regarding the addition of a submit button.
Aaron Lun (18:26:15): > Needless to say, this would also require a secondary internal field for the gene names in the plot vs the gene names in the memory
Kevin Rue-Albrecht (18:26:27): > eh. correct me if i’m wrong, but I think I put in a simple way to detected whether it’s already clustered
Aaron Lun (18:26:50): > From a fresh app instance?
Kevin Rue-Albrecht (18:27:07): > ow
Kevin Rue-Albrecht (18:27:15): > no
Kevin Rue-Albrecht (18:27:46): > only from the logic of user actions within an instance
Aaron Lun (18:28:17): > Yeah, that’s the problem. The state becomes dependent on things that are outside of memoyr.
Kevin Rue-Albrecht (18:28:17): > but i get you: you’re thinking if a user already provides a clustered list?
Aaron Lun (18:28:32): > Yes, or not.
Kevin Rue-Albrecht (18:28:49): > alright let’s roll back in the discussion a moment
Kevin Rue-Albrecht (18:31:04): > within an instance, buttons are following the logical progression of actions: > 1) if you just clustered, it is -> disable button > 2) if you import new features, we can assume it is isn’t clustered -> reenable > 3) if when you import, no new feature is imported -> stay disabled
Kevin Rue-Albrecht (18:31:33): > with the bonus > if you clear features, disable the button, but still label it “cluster features” so that users know what this button is about
Aaron Lun (18:32:03): > There’s no problem with the logic here. It’s when you have UI regeneration, that state information is lost.
Kevin Rue-Albrecht (18:32:27): > and if by miracle the user preconfigures the app with a list of pre-clustered features, well… enable the button anyway, and offer them a one in an instance lifetime chance to recluster it
Kevin Rue-Albrecht (18:32:40): > owwwwwww
Aaron Lun (18:33:04): > Then the button will always re-enable, even if I just clustered it.
Kevin Rue-Albrecht (18:33:14): > :sob:
Aaron Lun (18:33:38): > Yes.
Aaron Lun (18:34:19): > One could argue that it’s a finicky detail, but this entire line of work is finicky detail.
Kevin Rue-Albrecht (18:34:37): > hang on a sec
Kevin Rue-Albrecht (18:35:04): > maybe i got you wrong, but I just tested the scenario as I understood it, and i don’t see the problem
Aaron Lun (18:35:25): > What did you do?
Kevin Rue-Albrecht (18:36:20): > - i’ve put a few features in the heatmap > - clustered (the button got disabled) > - I’ve linked the heat map to the redDimPlot > - I’ve made a selection in the redDimPLot > - the heatmap figure regenerated, but the buttons stayed put (“cluster features” button remained disabled)
Kevin Rue-Albrecht (18:36:29): > is that what you were talking about?
Aaron Lun (18:36:35): > No.
Aaron Lun (18:36:44): > Create a new plot, or move the plots.
Kevin Rue-Albrecht (18:36:55): > oh. reordering
Kevin Rue-Albrecht (18:37:31): > damn
Kevin Rue-Albrecht (18:37:37): > it does get reenabled
Aaron Lun (18:38:46): > There is a way to sneakily overcome this - by storing state information in attributes of the memory.
Kevin Rue-Albrecht (18:38:48): > to be fair, clicking it does not take too long, asupdateSelectizeInputfigures out that it is being updated to the same thing, but still, I get your point
Aaron Lun (18:40:04): > I need to watch my late night soothing anime now, but have a think about the attributes.
Kevin Rue-Albrecht (18:40:45): > ok. enjoy!
Kevin Rue-Albrecht (18:41:21): > i’ll be a bit slower the next couple of weeks, as i just got word from the Genome Informatics conference today that they want a poster for the 17th of September
Kevin Rue-Albrecht (18:46:21): > damn.. i think i really need to revert all my heatmap changes: the clustering button should also be re-enabled if users change the centering or scaling of features
Aaron Lun (19:07:03): > I think that would be safest.
2018-08-23
Aaron Lun (09:19:38): > For the custom arguments field: I would suggest the following strategy.****(1)****Add a newVisibleArgsfield tocustomDataPlotDefaultsandcustomStatTableDefaults. This should be set toNA_character_.****(2)****Insetup_memory(), replace allNAinstances with the values ofArguments.****(3)****In the observer for the text area, updateVisibleArgswith the input value but do not trigger replotting. Incidentally, you can control the button shape at this point as well, by checking whether the value inVisibleArgsis identical to the value inArguments.****(4)****In the observer for the button, copyVisibleArgsintoArgumentsand trigger replotting.****(5)****TherenderPlotcall should only consider values inArguments; this is the same behaviour as before.
Aaron Lun (09:21:11): > You can also modifydynamicUI.Rto set the button shape upon UI (re)generation by considering the equality ofVisibleArgsandArguments.
Kevin Rue-Albrecht (09:22:39): > it all makes sense, though I can’t guarantee when I can get on that
Aaron Lun (09:56:48): > Heh, accidentally linked my beachmat vignettes to iSEE.
Aaron Lun (09:56:55): > Someone’s going to be pretty confused if they follow that link.
Kevin Rue-Albrecht (10:09:16): > and there I was, wishing for more traffic on our vignettes..:wink:
2018-08-24
Federico Marini (06:06:23): > In case you wonder whattitlelogodoes
Federico Marini (06:06:38): - File (PNG): Screen Shot 2018-08-24 at 12.05.52.png
Federico Marini (06:06:49): > corporate branding FTW
Kevin Rue-Albrecht (06:06:56): > neat
Kevin Rue-Albrecht (06:07:15): > is it me or does the hexagon look wider than high?
Federico Marini (06:07:21): > we can replace that with whatever final
Federico Marini (06:07:22): > can be
Federico Marini (06:07:31): > shouldnt
Federico Marini (06:07:45): > more like a proof of concept
Federico Marini (06:08:00): > now that the skin’s B&W it looks quite cool
Federico Marini (06:08:36): > plus it would make extra sense once we will show up with hex sticker AND tshirt
Federico Marini (06:09:27): > and/or combine with a slide with the bearded hipster meme
Aaron Lun (06:15:25): > Looks a bit crowded to me.
Kevin Rue-Albrecht (06:27:17): > On a different note, I haven’t figure yet how to give the textAreaInput widgets for custom arguments their own special class, to distinguish their CSS from all other future textAreaInput boxes. > The relevant code is: > - call totextAreaInputhttps://github.com/csoneson/iSEE/blob/a90e84ab71df905e86ad99c8222084e55a471898/R/dynamicUI.R#L224- new CSS file to store our custom styles:https://github.com/csoneson/iSEE/blob/newcolors/inst/www/iSEE.css- the source code ofshiny::textAreaInputreturns a<div>that doesn’t take custom class names > What comes to mind now would be to write a wrapper aroundshiny::textAreaInputthat edits the class attribute of the returned<div>to add an extra class. > Having played withshinyjs, all its functions work work on the server side and imply observers. Here we just want to add an extra ‘static’ class when the thetextAreaInputis initialized
Aaron Lun (06:44:04): > Just use standard CSS. Look at how I modified the panel classes, for example.
Kevin Rue-Albrecht (06:53:50): > Ok. I’ve seen yourpanel_colors.Rbut it’s not so much defining the CSS class my problem, as adding a custom class to thetextAreaInput
2018-08-25
Aaron Lun (08:37:06): > ARGH STUDENTS
Kevin Rue-Albrecht (10:12:10): > What have they done again? ^^
Aaron Lun (10:19:24): > Emailing me to ask about marks
Aaron Lun (10:35:21): > ugh. Stuck dealing with this boring project, and I don’t have the heart to tell my collaborator that I just don’t care.
Kevin Rue-Albrecht (10:37:03): > Argh. Tell me about the latter
Kevin Rue-Albrecht (10:37:31): > Having to put together a poster about a lincRNA project that just doesn’t want to die
Aaron Lun (10:47:26): > yeah, hate those lincs.
Aaron Lun (12:38:07): > y’know, I’ve been noticing this for a while, but we’re steadily losing watchers on the repo. Probably because we make so much noise:party_parrot:
Kevin Rue-Albrecht (12:38:53): > so much noise? ahh you mean actively committing and they get notifications of the PRs ?
Aaron Lun (12:39:20): > yeah
Aaron Lun (12:39:35): > every commit to a PR gets an email
Kevin Rue-Albrecht (12:39:43): > lol forgot about that
Aaron Lun (12:39:47): > not that it really matters
Kevin Rue-Albrecht (12:39:58): > i thought it was only when the PR was merged that an email was fired
Aaron Lun (12:40:22): > @Charlotte Sonesonare you talking about iSEE at SOUND?
Charlotte Soneson (12:40:27): > yes
Charlotte Soneson (12:40:29): > a bit
Charlotte Soneson (12:40:55): > it’s also on Bioconductor now, so GitHub may not be their main source anymore
Kevin Rue-Albrecht (12:40:59): > well, we might want to close PRs when we realise that they involve several more commits. We can always reopen them when we’re ready to merge again
Aaron Lun (12:41:16): > No, that’s fine, I think.
Aaron Lun (12:41:18): > I like getting the emails.
Kevin Rue-Albrecht (12:42:20): > Also, I thought we always had that number of watchers (~7), at least I don’t remember going above 10. Did we?
Aaron Lun (12:46:31): > Think it was 8.
Aaron Lun (12:46:36): > Maybe even 9.
Kevin Rue-Albrecht (12:47:24): > Meh, mot much to worry about then. As Charlotte said, I’m sure they’re going with the updates when they hit Bioconductor
Kevin Rue-Albrecht (12:48:38): > Although, I keep an eye on the download stats, and I feel that we’re still only hitting a small number of power users.
Aaron Lun (12:49:36): > That’s why@Charlotte Sonesonis going to blow them away on Tuesday.
Kevin Rue-Albrecht (12:50:50): > Shame I’m stuck with presenting that lincRNA poster at Genome Informatics in September. Can’t really argue to travel anywhere else these days.
2018-08-26
Federico Marini (07:53:33): > @Charlotte Sonesonand@Aaron Lun: are you in HD tomorrow already?
Charlotte Soneson (07:54:10): > I’ll arrive in the evening
Federico Marini (07:54:11): > are you in for the invited dinner or only for the workshop?
Federico Marini (07:54:13): > ok
Charlotte Soneson (07:54:22): > And go to the invited dinner
Federico Marini (07:54:29): > okiedokie
Federico Marini (07:55:05): > then I’ll setup something with some friends
Aaron Lun (09:06:00): > I’ll be in HD on Monday evening, so I’ll miss the first dinner
Aaron Lun (11:21:16): > I HAVE NOTHING LEFT IN THE TANK.
Aaron Lun (11:21:28): > I am going to do nothing for the rest of the day.
Aaron Lun (11:31:42): > Also, I hope one of you two are going to the Tuesday dinner, because I don’t want to get lost going there.
Charlotte Soneson (11:32:51): > I’m going. I was told there’d be a shuttle from the hotel where I’m staying at least.
Aaron Lun (11:33:29): > Oh. Hm.
Aaron Lun (11:33:55): > @Federico Marini? Do you know your way around?
Aaron Lun (11:34:12): > Maybe I’ll just bum along with John.
Aaron Lun (11:34:18): > Man, I don’t want to have to withdraw euros.
Charlotte Soneson (11:34:29): > There are worse things…
Aaron Lun (11:37:13): > is anyone going to Frankfurt airport?
Aaron Lun (11:37:22): > on wednesday afternoon?
Aaron Lun (11:37:29): > and willing to wait for me?
Charlotte Soneson (11:37:37): > I’m going back by train
Aaron Lun (11:45:25): > Hm.
Aaron Lun (11:45:37): > Are you passing through frankfurt?
Aaron Lun (11:45:44): > I don’t know how these trains work
Aaron Lun (11:46:12): > Apparently there’s at least one change to get from heidelberg to frankfurt, which is pretty scary.
Aaron Lun (11:46:30): > It wouldn’t be so bad if I was also going in by train - then I’d be able to figure it out ahead of time - but that won’t be the case.
Charlotte Soneson (11:47:12): > Nope, other direction. Heidelberg-Karlsruhe
Charlotte Soneson (11:48:25): > I’m sure there are others going to the airport after the meeting. At least Simone said there would be shuttles, so probably you can join in on one of those.
Aaron Lun (11:48:26): > Ah damn.
Aaron Lun (11:48:31): > Okay, good.
Aaron Lun (11:48:35): > Yeah, sweet.
Aaron Lun (11:48:59): > How come you got all these emails from Simone? I didn’t get anything.
Aaron Lun (11:49:03): > Maybe because I didn’t register?
Aaron Lun (11:49:04): > Hm.
Charlotte Soneson (11:50:23): > Probably for the invited speakers
Aaron Lun (11:51:12): > Yeah, workshop presenters are second-class citizens I guess
Aaron Lun (11:51:27): > Even though I have to talk for 6x as long.
Charlotte Soneson (11:51:37): > Haha, true
Federico Marini (14:43:02): > no info for me:disappointed:
Federico Marini (14:43:14): > dunno if i can join at my expenses
Federico Marini (14:43:35): > but about heidelberg, I have a few sweet spots:stuck_out_tongue:
Federico Marini (14:43:53): > if someone wants a mega breakfast, the schwarzer wal is The place
Aaron Lun (15:42:16): > I’ll probably be having brekky with the J-bone.
Aaron Lun (15:44:41): > Oh, can anyone give me timings on ggplot with >1 million points? Been trying to do that on the 1.3 mil data set (see#bigdata-rep) and it takes forever.
2018-08-27
Federico Marini (03:16:43): > no iSEE in the workshop email?:stuck_out_tongue:
Federico Marini (03:17:05): > dammit it would have been a good chance to bump up the downloads:dealwithit-parrot:
Charlotte Soneson (03:19:24): > I’m afraid not, at least not in the ones I’ve received:slightly_smiling_face:
Federico Marini (07:43:11): > HD is quite wet-ty now
Federico Marini (07:43:28): > shorts are still fine, yet
Aaron Lun (08:45:58): > @Kevin Rue-AlbrechtWhat we discussed is now done.
Kevin Rue-Albrecht (08:57:03): > Neat, thanks!
Kevin Rue-Albrecht (08:59:02): > Sorry that I’m not of much help these days. No recreation until I finish that damn lincRNA poster or die trying
Aaron Lun (17:23:20): > Man, finally got to HD.
Aaron Lun (17:23:33): > Anyone want to party hard with me?
Kevin Rue-Albrecht (17:25:32): > have fun guys!
Aaron Lun (17:44:06): > In any case, once you’re done with your poster, can you get it passing check and merge the PR?
2018-08-28
Aaron Lun (01:50:20): > ugh. I feel like shit.
Federico Marini (03:12:22): > ssup
Aaron Lun (03:16:37): > can’t you see me?
Aaron Lun (03:16:44): > I’m watching you.
Aaron Lun (03:17:22): > looks like you’re reading facebook.
Federico Marini (03:18:20): > eehheheh
Federico Marini (03:18:44): > taking the slackin’ to the next level
Aaron Lun (03:18:50): > :party_parrot:
Federico Marini (03:19:13): > had to, after reading a reviewer comment from a ms:
Federico Marini (03:19:31): > > As a bioinformatician, the only noteworthy supplement that I would suggest to the authors is to try a couple of additional tools for differential gene expression detection other than DESeq2 (e.g. edgeR, RNentropy). In fact, in cases like this one where the expression level differences are subtle, a simple majority approach with three tools using different statistics may improve on one hand the detection of deregulated genes and on the other limit the number of false positives. >
Federico Marini (03:19:36): > WUT
Federico Marini (03:20:27): > -> curiously being RNentropy a tool developed by the lab of one of the editors
Federico Marini (03:20:29): > HMMMM
Aaron Lun (03:21:44): > lol
Aaron Lun (03:22:34): > That’s a silly proposed approach. It’ll kneecap your power.
Federico Marini (03:23:44): > and it is a cran pkg with no vignette
Federico Marini (03:23:52): > jesus
Aaron Lun (03:24:51): > If you want you can write a Bioconductor question about it and I can answer and you can cite it. Though not now, because I forgot my support site password.
Federico Marini (03:25:02): > downloads: 91/month
Federico Marini (03:25:06): > aaaaaah
Federico Marini (03:25:08): > :stuck_out_tongue:
Federico Marini (03:25:38): > I mean, the samples just have a supermild effect
Federico Marini (03:25:40): > ok
Federico Marini (03:25:59): > let’s focus on lukas, their suggestion is just silly
Aaron Lun (03:25:59): > Wait till I get back, tho
Federico Marini (03:51:05): > do we have an hashtag for the meeting?
Federico Marini (03:53:24): > btw smita is also co-corresponding on the MAGIC paper
Aaron Lun (04:00:16): > ¯*(ツ)*/¯
Aaron Lun (04:00:26): > sounds like she’s going to go into MAGIC now
Aaron Lun (04:00:34): > I remember these figures from the paper.
Aaron Lun (04:01:20): > … and, there we go.
Federico Marini (04:01:40): > yoooooup
Aaron Lun (04:15:40): > ARGH
Aaron Lun (04:15:43): > ARGH
Aaron Lun (04:15:45): > ARGH
Federico Marini (04:20:21): > where’s the E in saucie?
Charlotte Soneson (04:20:49): > Embedding, I think
Federico Marini (04:22:14): > Aaron could also say “Enough”:slightly_smiling_face:
Aaron Lun (04:23:07): > :sad-parrot:
Federico Marini (04:25:30): > call me biased, but I don’t buy many things from this talk - or coming from the Pe’er lab
Aaron Lun (04:26:06): > Yeah, we’ve all had that feeling.
Federico Marini (05:02:17): > I don’t see RNentropy in the list:party_parrot:
Aaron Lun (05:02:26): > lol
Federico Marini (05:15:39): > uh, entropy indeed:stuck_out_tongue:
Kevin Rue-Albrecht (05:41:21): > any way to follow/catch up on the talks?
Aaron Lun (05:44:43): > Probably not… there is a camera tho
Federico Marini (05:44:53): > not really, there’S not that much livetweeting going on
Federico Marini (05:45:00): > AFAIK there’s no hashtag
Kevin Rue-Albrecht (05:57:26): > No worries, thanks
Federico Marini (06:02:50): > pics or didn’t happen
Federico Marini (06:02:53): - File (JPEG): 20180828_112237.jpg
Aaron Lun (06:04:42): > Man I had so much hair back then.
Federico Marini (06:05:18): > sunset boulevard is coming:smile:
Federico Marini (06:05:33): > when was the pic taken?
Aaron Lun (06:05:53): > 3 years ago. When I was young and pretty.
Federico Marini (06:06:53): > when john picked you for the looks. it all figures
Federico Marini (06:07:06): > now he can appreciate you for the brainz
Aaron Lun (06:07:37): > lol
Aaron Lun (10:02:27): > <<-EVIL
Aaron Lun (10:09:28): > I don’t think we have anything to worry about.
Federico Marini (10:11:57): > does it work for you on safari?
Federico Marini (10:12:04): > doesn’t for me
Aaron Lun (10:17:47): > Not really following the thing.
Aaron Lun (10:18:32): > But fuck, it’s complicated.
Federico Marini (17:45:36): > @Charlotte Sonesonand@Aaron Lun: should we sum up in an issue the content/outcome of the quick discussion today? more for thescripta manent
Kevin Rue-Albrecht (17:46:17): > now you got me curious:slightly_smiling_face:
Federico Marini (17:48:15): > nothing transcendental Kev:smile:
Federico Marini (17:48:39): > still, more like plans on what we could, should, and should not do
Kevin Rue-Albrecht (17:49:36): > like.onLoad .... {suppressPackageStartupMessages({require(evilR)})}?
Federico Marini (17:49:42): > being one of the relevant points: for every relevant dataset we get to analys e and publish, put an instance of iSEE available - at best with tour trying to go 1:1 (as much as possible) wit h the paper figures
Aaron Lun (17:54:38): > And@Charlotte Sonesonwill try to get the genomic viewer into iSEE.
Kevin Rue-Albrecht (17:55:42): > Yeah - i was super close to packaging a data set in review as an iSEE app. However, with our active development of the package, it was a bit too much of a moving target
Kevin Rue-Albrecht (17:57:10): > Nice if she can do the genomic viewer. I had a play with it (the original package), and couldn’t figure how it even work itself, let alone integrate it within iSEE.
Aaron Lun (17:57:33): > This’ll be gviz-powered.
Kevin Rue-Albrecht (17:57:39): > oh, that i like
Kevin Rue-Albrecht (17:59:01): > I started going down the Gviz road with my TVTB package, but got interrrupted before I could properly get there. That was for variant exploration at the time
Kevin Rue-Albrecht (17:59:53): > I’d probably be ashamed of going back in that code, in comparison with iSEE ^^
Kevin Rue-Albrecht (18:01:21): > I can’t remember what I read about why Shiny doesn’t/cannot support keyboard shortcuts, but that would have been on my wish list, especially with a genome browser
Federico Marini (18:03:02): > that’d be handy
Federico Marini (18:03:16): > open an issue with the RSTudio guys?
Federico Marini (18:04:18): > Just as an info: I got to insert the deployment of an iSEE for two upcoming/submitted projects, for which we requested funding
Kevin Rue-Albrecht (18:04:26): > I’m sure I’ve seen an issue about it somewhere, probably more than one, too
Federico Marini (18:04:32): > putting nicely a 6mo deliverable:party_parrot:
Kevin Rue-Albrecht (18:08:47): > … if we pull of another one like the lasso selection, we might just get Shiny job offers:wink:
Aaron Lun (18:11:20): > I’m a little concerned about what Simon Anders presented today. Not too much, as it sure involved a lot of coding for an exploratory analysis, but it is a lot more customisable… but then again, if you want full customisability, you might as well useshinydirectly!
Kevin Rue-Albrecht (18:11:59): > oh right, i meant to ask, after looking at the program. He had a talking about interactive exploration
Aaron Lun (18:12:21): > Think you can see it at ander-stats/rlc?
Kevin Rue-Albrecht (18:12:22): > Was it Shiny-based, or ‘only’ JS ?
Aaron Lun (18:12:27): > JS
Aaron Lun (18:12:46): > https://github.com/anders-biostat/rlc - Attachment (GitHub): anders-biostat/rlc > linked-charts from R. Contribute to anders-biostat/rlc development by creating an account on GitHub.
Kevin Rue-Albrecht (18:13:21): > Ahhh the R linked charts! I heard that before
Kevin Rue-Albrecht (18:13:40): > Wasn’t it mentioned/presented at EuroBioc2017 ?
Aaron Lun (18:13:40): > Yes, he presented this at the last euro-bioc meeting.
Aaron Lun (18:13:45): > At which I thought… just use shiny!
Kevin Rue-Albrecht (18:14:47): > Exactly. I can’t remember exactly what I thought, but I have a vague memory thinking about limited use for the general audience
Kevin Rue-Albrecht (18:15:14): > I’m browsing the repo.. it doesn’t come with a demo?
Aaron Lun (18:15:35): > tests?
Kevin Rue-Albrecht (18:16:03): > I meant an online one. (I’m installing the package)
Aaron Lun (18:16:23): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (18:16:48): > I genuinely hope we’ll kick a trend of uploading interactive demo on F1000 ^^
Kevin Rue-Albrecht (18:17:35): > “there is no package called ‘JsRCom’” and “package ‘JsRCom’ is not available (for R version 3.5.1)”:sob:
Kevin Rue-Albrecht (18:17:54): > Sorry Simon. I’ll try another day again..
Kevin Rue-Albrecht (18:18:26): > We have our wonderful, 1st of the name, “group retreat” tomorrow ..
Kevin Rue-Albrecht (18:19:29): > :grimacing:
Federico Marini (18:23:10): > no demo, butlook anders-biostat/rlc_tutorial
Kevin Rue-Albrecht (18:25:35): > Oh cool. I get the gist from the “presentation_steps.txt” thanks
Aaron Lun (18:28:10): > https://github.com/anders-biostat/JsRCom - Attachment (GitHub): anders-biostat/JsRCom > An R package to exchange commands between R and JavaScript - anders-biostat/JsRCom
Aaron Lun (18:28:25): > A package to exchange messages between R and javascript. Hold on… that’s shiny!
Federico Marini (18:42:28): > :dealwithit-parrot:
Federico Marini (18:42:53): > Aaron, do you happen (totally unrelated) to know Tash Sultana?
Federico Marini (18:43:01): > not in person, of course
Federico Marini (18:45:07): > asking for myself, I’m in total trippy mood when I hear that voice:slightly_smiling_face:
Aaron Lun (18:47:09): > nope
2018-08-29
Kevin Rue-Albrecht (02:34:20): > Well spotted Aaron. Obviously I was not in a particular efficient mood last night ^^
2018-08-30
Federico Marini (03:39:19): > Morning:slightly_smiling_face:One more thought off the top of my head, about advertising iSEE: we should not underestimate the relative wide reader base of the RNA-seq blog
Kevin Rue-Albrecht (04:47:30): > I was reminded about that too a couple of days ago - a friend sent them an email about her later paper, and they made a blog post of it
Kevin Rue-Albrecht (04:50:13): > although I also want to reiterate a warning about how they handle this kind of email: they often grab the email verbatim, and barely make any change when they turn it into a blog post format, so we need either 1) to send them a very general email asking them how they prefer things being sent to them, or 2) make sure that we’re happy with whatever we send them
Kevin Rue-Albrecht (04:50:34): > Note also that they can edit the post afterwards if there’s really an issue
Kevin Rue-Albrecht (04:51:56): > For instance, here’s the post of my friend:https://www.rna-seqblog.com/rna-sequencing-rna-seq-reveals-extremely-low-levels-of-reticulocyte-derived-globin-gene-transcripts-in-peripheral-blood-from-horses-equus-caballus-and-cattle-bos-taurus/She was a bit disappointed as they really just grabbed the content of her email to make the blog post, hence the small image - Attachment (rna-seqblog.com): RNA Sequencing (RNA-Seq) Reveals Extremely Low Levels of Reticulocyte-Derived Globin Gene Transcripts in Peripheral Blood From Horses (Equus caballus) and Cattle (Bos taurus) > RNA-seq has emerged as an important technology for measuring gene expression in peripheral blood samples collected from humans and other vertebrate species. In particular, transcriptomics analyses of whole blood can be used to study immunobiology and develop novel biomarkers of infectious disease.
Charlotte Soneson (04:53:10): > Knowing this, I guess the best would be to write the blog post ourselves and send to them. Easy for them, safe for us…
Kevin Rue-Albrecht (04:53:25): > Yep
Kevin Rue-Albrecht (04:55:04): > They also have the sexy feature of quoting authors by the way.. Check out this guy who didn’t know about it:https://www.rna-seqblog.com/goexpress-visualize-rna-seq-and-microarray-data-using-gene-ontology-annotations/ - Attachment (rna-seqblog.com): GOexpress - visualize RNA-Seq and microarray data using gene ontology annotations > GOexpress accepts gene expression datasets obtained from both microarray and RNA-seq platforms formatted in the recommended Bioconductor “ExpressionSet” container, to evaluate the power of each feature expressed in the dataset to cluster biological samples according to known experimental factors. In
Federico Marini (04:56:59): > that one I know
Kevin Rue-Albrecht (04:57:50): > They literally grabbed one sentence from my email and featured it as a quote
Kevin Rue-Albrecht (04:58:15): > unfortunately that doesn’t give reputation points on the Bioc support website:stuck_out_tongue:
Kevin Rue-Albrecht (05:02:53): > On a more serious note - are we ready to bump up the devel version of iSEE?
Kevin Rue-Albrecht (05:04:11): > or does anyone have anything to throw in before?
Kevin Rue-Albrecht (05:04:17): > oh wait Travis fails
Kevin Rue-Albrecht (05:06:25): > Well I’ve got some notebooks to run again to update poster figures, and a trip to the dentist this afternoon (fillings:sob:) , but I should be able to look at this in the evening.
Kevin Rue-Albrecht (05:07:04): > Please have a play with thenewcolorsbranch, and see if anything screams out to be fixed
Kevin Rue-Albrecht (05:07:19): > @Federico Mariniwhat’s the status withtitlelogo?
Charlotte Soneson (05:11:58): > I used thenewcolorsbranch for the demo in Heidelberg this week, and nothing broke during the presentation at least:slightly_smiling_face:
Kevin Rue-Albrecht (05:14:46): > Cool. I haven’t tested it since Aaron implemented the custom text field memory, but I think I still have to look at certain action buttons that may need to be deactivated/reactivated in certain cases, like i did for the custom argument text fields
Federico Marini (05:15:05): > titlelogo is on hold, AFAIK
Federico Marini (05:15:34): > I wanted a proof of principle that we can indeed put a logo in the title, and see that it doesn’t suck with the new skin we picked
Federico Marini (05:15:42): > we do need to finalize the logo indeed
Charlotte Soneson (05:15:42): > we never really made it very far into the logo discussion in Heidelberg…
Federico Marini (05:15:59): > aaron said he’d like it more colourful
Kevin Rue-Albrecht (05:16:13): > fede, did you tell them about my ‘prism’ idea?
Federico Marini (05:16:23): > but we shouldjustpick two plot types
Federico Marini (05:16:27): > didn’t get there yet
Federico Marini (05:16:31): > we put that as item 1
Federico Marini (05:16:40): > we did never touch it really:disappointed:
Kevin Rue-Albrecht (05:17:42): > I know the proof of concept looked crap, but in short, I thought the “seeing” and the color coding of our panels would fit well with a prism that receives data (instead of light), and breaks it into plot panels (of different colors)
Kevin Rue-Albrecht (05:18:15): > However, that’s all a bit ambitious to fit in a small hexagonal sticker
Kevin Rue-Albrecht (05:19:27): > andmy proof of concept picture shamelessly ‘borrowed’ the prism logo from GraphPad
Aaron Lun (06:07:42): > Let’s just get what we promised to get done first.
Kevin Rue-Albrecht (15:59:58): > @Aaron Lunthe reason why Travis fails the unit test is the custom panel parameter"VisibleArgs"initialized toNA. Is it important that it is not""like the"Argument"parameter?
Kevin Rue-Albrecht (16:00:51): > Because at the moment, testing Arguments==VisibleArgs returns NA , which makes anifstatement unhappy
Kevin Rue-Albrecht (16:02:14): > A simple fix would be to initializeVisibleArgsto"", likeArguments, but I don’t want to break anything isNAis important
Kevin Rue-Albrecht (16:08:16): > Ah. I got why NA is important. This is necessary to set VisibleArgs<-Arguments on startup if the latter is preconfigured
Aaron Lun (17:17:32): > We can’t set the default to""as that would tell the app to show nothing in the text box.
Kevin Rue-Albrecht (17:48:32): > yup, thanks for confirming. Made sense when I stopped to think about it twice
2018-08-31
Aaron Lun (09:16:16): > are we good to merge the PR?
Kevin Rue-Albrecht (09:26:42): > Seems like coverage is unaffected, and i’ve checked that the argument textbox worked as expected after my fix last night, so I’d say yes. That said, I haven’t extensively checked other aspects of the app
Kevin Rue-Albrecht (09:28:06): > Do we want to version-bump and push upstream btw?
Aaron Lun (09:28:09): > Yes.
Kevin Rue-Albrecht (09:34:50): > Ok then. I can push the version bump, but then I’ll have to disappear for a couple of hours (dentist in 30min). So there will be everyone’s last chance to check it and/or throw last minute changes in before I merge the PR and push upstream.
Aaron Lun (09:35:05): > :+1:
Kevin Rue-Albrecht (09:37:34): > (e.g. updating the NEWS file if anyone’s really bored)
Kevin Rue-Albrecht (09:41:24): > pushing
Aaron Lun (09:41:25): > Chance to rack up those commits before the end of the month, people.
Kevin Rue-Albrecht (09:41:32): > that’s the spirit
Federico Marini (09:56:26): > :slightly_smiling_face:
Federico Marini (09:57:07): > how sweet it is to be updating websites, written pre-markdown era
Federico Marini (09:57:18): > :face_vomiting:
Kevin Rue-Albrecht (13:40:36): > Merged PR to master. I’ll build one last time locally to play with it a bit, and then push upstream
Aaron Lun (13:51:40): > Thanks. Stuck in a code dogfight over here.
Kevin Rue-Albrecht (13:53:39): > Getting my sensations back in my left jaw over the past couple of hours… I discovered today that I don’t like having it anaesthetised:face_with_head_bandage:
Kevin Rue-Albrecht (14:11:11): > Alright, can’t see anything wrong. I just pushed upstream
2018-09-01
Federico Marini (18:10:42): > for the ones who might want to peek:https://federicomarini.github.io/GMDS2018/
Federico Marini (18:10:51): > here are the slides for the conf next week
Federico Marini (18:11:12): > Aaron, if you need me to change pic, I need a recent one:slightly_smiling_face:
Federico Marini (18:11:29): > but I also would take the chrM- one:smile:
2018-09-03
Aaron Lun (05:10:14): > chrM?
Federico Marini (05:11:22): > mitochondria:smile:
Aaron Lun (05:12:06): > my browser froze upon trying to open your preso, so I don’t get the joke.
Federico Marini (05:12:36): > I put your old photo the one with hair
Federico Marini (05:13:01): > not having access to the one where you were dressed up like a cell
Aaron Lun (05:13:12): > ah right
Aaron Lun (05:55:19): > @Kevin Rue-AlbrechtWe’re errorring on Windows check, file path test failing foronLoad.
Kevin Rue-Albrecht (05:57:41): > Yep. I saw that. Was wondering whether to remove the file path test or figure out whetherfile.pathcan return Windows-style paths
Federico Marini (05:57:48): > do we need a call tonormalizePath?
Kevin Rue-Albrecht (05:57:52): > I think testing the path is over the top anyway
Kevin Rue-Albrecht (05:59:00): > However, I have to send an updated version of my poster to Steve and collaborators today, before I do anything else.
Federico Marini (06:00:03): > a.k.a. buying yourself your free time:slightly_smiling_face:
Kevin Rue-Albrecht (06:00:50): > I couldn’t have put it better ^^
Aaron Lun (13:22:28): > ARGH. Just wasted the last 5 days trying to improve my normalization method.
Aaron Lun (13:22:54): > 20 commits later, I ended up with almost exactly the same code I started with.
Aaron Lun (13:23:08): > The only change was to increase the pool sizes by 1.
Aaron Lun (13:23:18): > grumble
Kevin Rue-Albrecht (13:28:21): > To be fair, that just sounds like everyday for our wet lab friends:wink:
Aaron Lun (13:29:57): > But the problem here is that I feel I should be able to improve it.
Aaron Lun (13:30:09): > When wetlab experiments fail, people just shrug and move onto the next reagent or protocol.
Aaron Lun (13:30:28): > But for me, the solution is so close yet so far.
Kevin Rue-Albrecht (13:31:12): > I was thinking more of the dilution series and test infections my sweetie does before the main experiment
Kevin Rue-Albrecht (13:33:18): > Anyway. You probably just need to do something else for a bit and come back to it in a few days
Aaron Lun (13:34:58): > There is no solution. I’ve tried for years.
Aaron Lun (13:35:05): > on and off and on and off
Kevin Rue-Albrecht (13:35:09): > ah.
Aaron Lun (13:35:11): > It started in 2013.
Aaron Lun (13:35:30): > On the plus side, I now have this really sweet code for fitting a truncated normal distribution.
Kevin Rue-Albrecht (13:35:51): > Does anyone have a Windows machine that is capable of running R unit tests? I have no way of testing whethernormalizePathcould solve the problem on a Windows machine. I could set up a virtual machine, etc.. but.. if there’s an easier solution I’m all ears. > Maybe for the time being I can really just disable theexpect_identical
Aaron Lun (13:36:34): > ¯*(ツ)*/¯
Aaron Lun (13:36:50): > expect_true(Windows == losers)
Kevin Rue-Albrecht (13:39:11): > hehehe that became apparent whenUnsupportedPlatforms: winappeared in.BBSoptions
Aaron Lun (15:57:02): > if you’re lucky, you’ll see a brand new message on your slack loading screens.
Kevin Rue-Albrecht (16:16:22): > looking forward!
2018-09-04
Aaron Lun (04:56:13): > Didn’t see it today. Oh well
Kevin Rue-Albrecht (05:07:07): > Me neither. I got Levi’s BiocManager message
Kevin Rue-Albrecht (06:52:47): > Can we set the seed on login?:stuck_out_tongue:
Aaron Lun (06:54:01): > lol
Aaron Lun (06:54:02): > ¯*(ツ)*/¯
Federico Marini (11:12:41): > iSEE team, what is the plan for eurobioc?
Federico Marini (11:13:02): > it is clear that we want to present:slightly_smiling_face:
Federico Marini (11:13:47): > who can make it/will make it? registration is open from now
Federico Marini (11:14:06): > but i guess we can’t all be listed as presenters?
Kevin Rue-Albrecht (11:17:11): > Argh. Steve is away until the 12th. I’ll ping him a message to see whether I can get the time and travel expenses.
Kevin Rue-Albrecht (11:17:33): > I was hoping to put him in a good mood with the poster I’ve been preparing
Kevin Rue-Albrecht (11:18:36): > In any case, even if he sends me, he’ll probably request that I present something from the group.
Charlotte Soneson (11:18:39): > My plan was to go, it’s close by:slightly_smiling_face:have to check with the new place though
Federico Marini (11:19:55): > Shall we propose two talks? One on the technical aspects, the other one on the experience of getting together? I recall a couple of years ago Rory Stark gave a talk on “when to increase the X in the x.y.z in the version number”:stuck_out_tongue:
Federico Marini (11:20:28): > or well. we can do
Federico Marini (11:20:33): > isee standalone
Kevin Rue-Albrecht (11:20:34): > hahaha sounds like a fun talk
Federico Marini (11:20:50): > and isee as the big hammer to deal with any dataset
Federico Marini (11:21:19): > which means showcasing some apps reproducing cool sets and the figures of publications
Federico Marini (11:23:35): > (i want to hold the attendance streak uninterrupted:smile:)
Kevin Rue-Albrecht (11:44:07): > Street fighter super combo?
Kevin Rue-Albrecht (11:44:53): > Anyway. Anything to add before merging the PR that brings our Windows friends back in the game?
Aaron Lun (11:46:30): > FINISH HIM:-1:
Aaron Lun (13:38:35): > I’ve finally given up on my normalization thing.
Aaron Lun (13:38:56): > I’ve realized that my feelings aboutcomputeSumFactorsis a bit like what a successful marriage would feel like.
Aaron Lun (13:39:03): > At first, I thought it was the best.
Aaron Lun (13:39:18): > Then over time, I started to see more and more flaws, so I tried to get something better.
Aaron Lun (13:39:28): > But now I’ve realized that there is nothing better, and it’s the best I’m going to get.
Kevin Rue-Albrecht (13:42:11): > :rolling_on_the_floor_laughing:
2018-09-05
Federico Marini (03:26:41): > I’ve seen a postcard last week in HD in the hostel
Federico Marini (03:27:00): > “marriage is not that bad. There’s sex every now and then, but apart from that, it’s all good”
Kevin Rue-Albrecht (04:41:05): > Anyone has an idea why iSEE is still listed among the packages that have a mention ofbiocLiteorBiocInstaller?https://gist.github.com/LiNk-NY/44cc844a169d5d96c777a69037dae653I’ve grep’ed and can’t find any mention myself
Charlotte Soneson (04:43:42): > There is a mention ofbiocLitein the NEWS file…:shrug:
Kevin Rue-Albrecht (04:43:50): > argh
Kevin Rue-Albrecht (04:44:17): > Must be it. Thanks!
Kevin Rue-Albrecht (04:44:24): > (my bad, that mention)
2018-09-06
Aaron Lun (03:49:55): > My dog was incontinent last night.
Aaron Lun (03:50:07): > Well, my landlady’s dog.
Federico Marini (03:56:00): > shit
Aaron Lun (03:56:17): > Yes. I was tired of dealing with her shit.
Aaron Lun (03:56:21): > Literally.
Aaron Lun (03:56:26): > So I moved her out of my room.
Federico Marini (03:56:30): > liTTerally:smile:
Aaron Lun (03:56:31): > And she kept on crying.
Aaron Lun (03:56:38): > So I went to sleep with earplugs.
Federico Marini (03:56:58): > have you tried telling her “get yo shit together maaaan”?
Aaron Lun (03:57:12): > should say bitch, that’s technically correct.
Federico Marini (03:57:23): > iSEEwe are going off topic quite a bit
Federico Marini (03:57:25): > :slightly_smiling_face:
Federico Marini (03:57:50): > off the top of my head, so that i don’t forget it
Federico Marini (03:57:59): > here in written form
Federico Marini (03:58:20): > what about pres1: iSEE in general, also as a collaborative experience
Federico Marini (03:59:12): > and pres2: leveraging Bioconductor’s infrastructure at its full:ExperimentHubto put the preprocessed data, and iSEE to have the tour-guided explorer
Aaron Lun (03:59:50): > I don’t know what this is for.
Federico Marini (04:02:04): > the second presentation you mean?
Federico Marini (04:02:48): > as an example of: data is out there, we do a small package for curating it and preprocessing, the data is linked to ehub
Federico Marini (04:03:20): > and iSEE picks the SE object plus the tour and settings and we BOOM deliver a browser for the cool dataset
Aaron Lun (04:04:02): > No, where is this?
Aaron Lun (04:04:06): > what conference is this for?
Federico Marini (04:04:23): > aaah
Federico Marini (04:04:24): > sorry
Federico Marini (04:04:26): > eurobioc
Federico Marini (04:04:30): > I thought it was clear
Federico Marini (04:04:35): > munich, dec 2018
Federico Marini (04:04:42): > registration just opened
Federico Marini (04:04:54): > and to put it like steven tyler, I don’t wanna miss it:smile:
Aaron Lun (08:40:45): > @Kevin Rue-AlbrechtDid 116 get pushed?
Kevin Rue-Albrecht (08:49:35): > yup
Aaron Lun (08:50:20): > okay.
Kevin Rue-Albrecht (08:50:56): > I was half tempted to pile on a 117 to drop the biocLite ‘issue’, but then I found my sanity again and decided that could be a tiny commit part of another release
Aaron Lun (08:51:15): > ¯*(ツ)*/¯
Aaron Lun (08:51:25): > I’ve bumped for less than that.
Aaron Lun (08:51:37): > E.g., when beachmat updates, I have to bump all dependent packages to trigger their recompilation
Aaron Lun (08:51:48): > otherwise things break in weird fucked up ways.
Kevin Rue-Albrecht (08:52:32): > It’s only an ‘issue’ in Marcel’s gist, not anywhere else. But yeah, it itches me enough to push it upstream before the next bioc release
Kevin Rue-Albrecht (08:53:11): > and now that you say it, I remember your recompilation issues from somewhere (Slack or mailing list)
Kevin Rue-Albrecht (09:11:10): > alright, opening a PR for 117
Aaron Lun (09:20:05): > let’s get off the gist of shame
Kevin Rue-Albrecht (09:22:14): > I was so proud of being among the first ones to get off of it that Ihad tomention it in the NEWS file… and get right back on it, “do not collect £200”-style:unamused::face_palm:
Federico Marini (09:27:47): > same as the line in postcards “don’t write below this line” - A friend of mine once wrote “fine, I won’t do that”
Kevin Rue-Albrecht (09:28:07): > I assume the card never arrived?
Federico Marini (09:28:16): > it was not meant for me
Federico Marini (09:28:24): > but I guess not?
Federico Marini (09:29:06): > how flexible and tolerant italian post officers could be, that I think you already experienced ?:stuck_out_tongue:
Kevin Rue-Albrecht (09:30:57): > Ahhh i’ve had the experience of italian post officers a couple of years ago in Pisa. The missus warned me, I insisted, they quickly demonstrated how painful they can be when they just don’t want to deal with something
Federico Marini (09:31:19): > yop
Kevin Rue-Albrecht (09:31:50): > In a different scene, i still regret listening to the missus and buying stamps in a mexican pharmacy last summer. Those postcards clearly never arrived.
Federico Marini (09:32:45): > Reporting from the PM I had with Charlotte, quick feedback post-conference: > - the crowd was relatively off, yet > - got 4-5 questions in the session itself, plus someone interested afterwards > - weird question award: “how can you guarantee your project will still live in 5 or more years from now”
Federico Marini (09:33:12): > to which I wanted to reply: is your code/software for eternity as well:stuck_out_tongue:
Kevin Rue-Albrecht (09:35:02): > “off” as in “postprandial somnolence”, “not interested”, “not impressed”?
Federico Marini (09:36:05): > off in “off cause it is a german conference and even a keynote was held in german”
Federico Marini (09:36:33): > and BTW they put a social activity like the city tour parallel to my session. Well done, orga team!
Kevin Rue-Albrecht (09:36:48): > lol
Federico Marini (09:37:41): > l’insulte en plus de la blessur -> I even paid for signing up to that
Federico Marini (09:38:19): > if that’S a correct translation - otherwise, to add insult to injury
Kevin Rue-Albrecht (09:38:50): > between that and the conferences in ski resorts.. i get the “informal networking” part, but it doesn’t really give credit the scientific program
Federico Marini (09:39:29): > as long as it is one after the other, I am quite fine abt that
Kevin Rue-Albrecht (09:39:41): > (god I’ll have to google the french translation, I’m not sure anymore whether we have an exact equivalent:rolling_on_the_floor_laughing:)
Kevin Rue-Albrecht (09:40:21): > Exactly, pre- or post-conference activity is fine. Anything else you hurt someone’s feelings.
Kevin Rue-Albrecht (09:40:40): > They wouldn’t do it for a keynote speaker, why would they do it to any other speaker?
Aaron Lun (09:41:35): > Speaking of Europeans: Did you guys hear the news about the ex-EBI guy?
Kevin Rue-Albrecht (09:41:36): > here we go: “tourner le fer dans la plaie”, or I’m even more familiar with “tourner le couteau dans la plaie”
Kevin Rue-Albrecht (09:41:44): > (turn the knife in the wound)
Kevin Rue-Albrecht (09:42:04) (in thread): > eh… no
Kevin Rue-Albrecht (09:42:19) (in thread): > what did he do/was done to him?
Aaron Lun (09:43:18) (in thread): > https://www.cambridge-news.co.uk/news/cambridge-news/novere-babraham-court-child-sentence-15111018
Aaron Lun (09:44:57) (in thread): > pretty fucked up right
Aaron Lun (09:45:06) (in thread): > and he has such a high h-index! Crazy.
Federico Marini (09:45:10) (in thread): > didnt hear
Federico Marini (09:45:16) (in thread): > I’mma check
Federico Marini (09:45:22) (in thread): > WUT
Federico Marini (09:45:28) (in thread): > i know that name
Federico Marini (09:45:30) (in thread): > WTF
Federico Marini (09:45:35) (in thread): > this is waaaay fucked up
Kevin Rue-Albrecht (09:45:46) (in thread): > Nicolas Le Novère?? I’m familiar with the name. I’d have to check, but I think one of my friend did a placement with him years ago
Kevin Rue-Albrecht (09:47:29) (in thread): > yup, first name here:http://precedings.nature.com/documents/6363/version/1 - Attachment (precedings.nature.com): The System Biology Format Converter : Nature Precedings > The System Biology Format Converter
Federico Marini (09:49:37) (in thread): > when did that happen?
Aaron Lun (09:55:35) (in thread): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (09:56:13) (in thread): > there are a few dates stated in Aaron’s link
Kevin Rue-Albrecht (09:56:46) (in thread): > the news article itself was written a couple of days ago
Federico Marini (09:58:21) (in thread): > his h10 index did somewhat already peak
Federico Marini (09:58:33) (in thread): > maybe it dates back to 2017 already?
Kevin Rue-Albrecht (09:59:23): > 1…2..3 I’m merging the PR is there’s nothing else that requires immediate attention?
Aaron Lun (09:59:43) (in thread): > There was an email on the EBI mailing list assuring people that the EBI network was not used for this stuff.
Aaron Lun (09:59:50): > FINISH HIM:-1:
Federico Marini (10:00:00) (in thread): > uh
Federico Marini (10:00:20): > BIOC-LITY
Federico Marini (10:00:37): > sorry, that was too good to not say out loud
Kevin Rue-Albrecht (10:02:00): > why did i not understand that one?
Aaron Lun (10:03:12) (in thread): > so typical, covering their own arse
Kevin Rue-Albrecht (10:04:02) (in thread): > hehe.. well they have to..
Kevin Rue-Albrecht (10:05:50): > anyway, 117 pushed up’
Federico Marini (10:33:51): > biocLite + fataLITY
Federico Marini (10:34:08): > if said out loud it would have worked out better
Kevin Rue-Albrecht (10:35:04): > oww right, I wouldn’t have guessed that:sweat_smile:
2018-09-07
Federico Marini (15:52:52): > call it the krishnaswamy effect
Federico Marini (15:52:53): > https://arxiv.org/abs/1802.04927
2018-09-08
Aaron Lun (06:39:00): > <!channel>I propose a feature freeze on the current version, given how close we are to the next release. Our main work now should be to test and create instances as we discussed in HD.
Kevin Rue-Albrecht (07:57:46) (in thread): > I just learned about the existence of the “Stanford bunny” - File (PNG): Pasted image at 2018-09-08, 1:57 PM
Kevin Rue-Albrecht (08:09:49): > Feature-freeze is a good idea right now. > What sort of instances did you have in mind? Full data set style? I was recently wondering whether it’d be didactic to have a small set of modest apps with 1-2 panels to demonstrate individual features in lightweight apps that boot up more rapidly that big “full-feature” ones.
Federico Marini (09:12:15): > we were planning to have the panel somewhat oriented to a minibrowser, which charlotte wanted to draft out
Federico Marini (09:12:31): > but I guess time is even scarcer on her side in the next weeks
Federico Marini (09:12:41): > so I guess feature freeze seems good
Federico Marini (09:13:16): > on my side, I am preparing the whole steps to try and mimicry the blood paper data I was checking
Federico Marini (09:13:33): > (http://www.bloodjournal.org/content/early/2018/03/23/blood-2017-12-821413?sso-checked=true)
Federico Marini (09:13:52) (in thread): > eheheh
Federico Marini (09:18:49): > @Aaron Lun: I was expecting some answer from you on this ->https://support.bioconductor.org/p/112815/#112853
Aaron Lun (09:19:22): > ?
Federico Marini (09:19:54): > you have been almost too kind to him
Aaron Lun (09:20:45): > I was going to be extremely harsh, but then there was an integer overflow.
Federico Marini (09:20:51): > I liked his start with “i read2-3 times”
Federico Marini (09:21:40): > sometimes you have to hit them politely:stuck_out_tongue:
Charlotte Soneson (09:27:22): > Yes - I started making the browser panel, but I don’t think it will be in good enough shape for this release. Right now I’m mostly wondering how much restrictions we can put on the input GRanges object - for example, Gviz likes one column to be called “transcript” so that it can bind together the exons properly, but Ensembl gtfs call this column “transcript_id”…so there may have to be a bunch of selections to make in the panel
Kevin Rue-Albrecht (09:46:36) (in thread): > Maybe we could have a character vector inconstant.Rthat list column names to consider for transcript id, by decreasing order of preference?c("transcript", "transcript_id", ...)
Kevin Rue-Albrecht (09:48:10): > on a separate note, I was also just thinking about other types of assays: DNA-seq, WGBS, … that we could demonstrate the compatibility with iSEE
Charlotte Soneson (09:49:04) (in thread): > yes, might be a possibility. I’ll have to see how many of these issues there are.
Aaron Lun (09:49:08) (in thread): > This seems like a call argument rather than an UI argument.
Aaron Lun (09:49:23) (in thread): > Hard to imagine someone wanting to change the exon groupings on the fly.
Charlotte Soneson (09:49:47) (in thread): > yes, you’re probably right
Kevin Rue-Albrecht (09:51:31) (in thread): > oh yeah, clearly not changing on the fly, but I can see how users might want to change it in their call toiSEE. Could be stored in themetadata()slot of these
2018-09-09
Federico Marini (08:09:46): > are these also SE objects Kev? I am not familiar with the data type
Federico Marini (08:10:13): > or would that rather be something like RaggedExperiment
Charlotte Soneson (11:10:54): > I just pushed a branchfixedcolwith two small changes to the tables: > (1) fix the leftmost column (row names) so that it is always visible even if one scrolls horizontally. > (2) Add a button to select which columns are visible. > Now, I don’t know if we want to keep these additions, but I thought I’d try to see what happens. (1) can make the table look kind of weird if the panel is not wide enough (see Row statistics table screenshot), but it looks ok if you make it a bit wider (sometimes you have to do something to the table, like sort or so, to trigger the re-rendering). However, the tables already look a bit weird without the fixed column when the panel is not wide enough (titles and columns are not aligned, Column statistics table screenshot). - File (PNG): Screen Shot 2018-09-09 at 16.45.29.png
Charlotte Soneson (11:10:58): - File (PNG): Screen Shot 2018-09-09 at 16.47.58.png
Aaron Lun (11:11:58): > be careful with what UI elements you choose to add, as these need to have their ownmemorystores.
Charlotte Soneson (11:12:20): > yeah, I was thinking about that too. The visibility is not remembered.
Charlotte Soneson (11:14:01): > I did not make it optional to fix the first column. If we want to keep these features we probably want to think if there are better ways of doing it.
Aaron Lun (11:15:04): > Can’t you just fix the row names?
Aaron Lun (11:15:15): > that would be the logical choice of fixative.
Charlotte Soneson (11:15:24): > Yes, that’s what I did (which is the first displayed column)
Aaron Lun (11:15:52): > Then that’s fine.
Aaron Lun (11:16:11): > Visibility should be in a parameter box somewhere.
Kevin Rue-Albrecht (11:46:55) (in thread): > Try that: > > library(ExperimentHub) > library(snpStats) > library(iSEE) > > # Fetch data ---- > > ehub <- ExperimentHub() > > EH550 <- ehub[["EH550"]] > f <- openfn.gds(EH550$filename) > > (node <- index.gdsn(f, "genotype")) > genotype <- read.gdsn(node) > > (node <- index.gdsn(f, "sample.annot/population")) > population <- read.gdsn(node) > > (node <- index.gdsn(f, "sample.annot/gender")) > gender <- read.gdsn(node) > > closefn.gds(EH550) > > # Random sampling of variants ---- > set.seed(1234) > idx <-[sample.int](http://sample.int)(nrow(genotype), 1E6) > > # PCA ---- > > snps.class <- new("SnpMatrix", t(genotype[idx, ])) > xxmat <- xxt(snps.class, correct.for.missing=FALSE) > evv <- eigen(xxmat, symmetric=TRUE) > pcs <- evv$vectors[,1:5] > > # iSEE ---- > > sce <- SingleCellExperiment( > assays=list(genotype=genotype[idx, ]), > colData=DataFrame( > population=population, > gender=gender > ), > reducedDims=list(PCA=pcs)) > > initialPanels <- DataFrame( > Name=c("Reduced dimension plot 1", "Column data plot 1", "Feature assay plot 1", "Row statistics table 1", > "Column statistics table 1", "Heat map 1"), > Width=c(4L, 4L, 4L, 4L, 4L, 4L), > Height=c(500L, 500L, 500L, 500L, 500L, 500L) > ) > > rd <- redDimPlotDefaults(sce, 1) > rd$ColorBy <- "Column data" > rd$ColorByColData <- "population" > > iSEE(sce, redDimArgs = rd, initialPanels=initialPanels, sampAssayMax=0) >
Kevin Rue-Albrecht (11:48:28) (in thread): > It’s very prototype, but it does prove the concept, while giving a use case of a discrete assay type (our heat map panel clearly doesn’t like it yet)
Kevin Rue-Albrecht (11:54:03) (in thread): > But to answer the question, theVCFobjects of the VariantAnnotation package, and the BSSeq object of the bsseq packages are also derived from SE.
Kevin Rue-Albrecht (11:55:20) (in thread): > so, these are two cases already that we could look into
Kevin Rue-Albrecht (11:58:04) (in thread): > RaggedExperiment and MultiAssayExperiment could be something else, a tad more complex that we could into another time
2018-09-10
Federico Marini (11:22:23): > anyone of you presenting something else than iSEE?
Federico Marini (11:22:50): > sorry for bugging you again:stuck_out_tongue:
Charlotte Soneson (11:23:00): > Don’t know yet, still need to confirm that I can go
Federico Marini (11:23:31): > how mature is the JCC work of yours Charlotte?
Kevin Rue-Albrecht (11:23:57): > I know Steve’s general policy is that I’d present something from the group, if I were to go
Aaron Lun (11:24:17): > I’m not going.
Charlotte Soneson (11:24:19): > Well, it’s on bioRxiv, the paper is submitted and there is an R package.
Federico Marini (11:25:01): > already on bioc or github only?
Charlotte Soneson (11:25:16): > Nah, GitHub only so far.
Federico Marini (11:25:22): > ok
Charlotte Soneson (11:25:29): > https://github.com/csoneson/jcc - Attachment (GitHub): csoneson/jcc > R package to calculate junction coverage compatibility (JCC) scores - csoneson/jcc
Federico Marini (11:25:58): > how are you holding up with the whole data/skills/duty transfer?
Charlotte Soneson (11:26:18): > So far, so good…at least on my side
Kevin Rue-Albrecht (11:26:42): > btw, I like how the iSEE page has: “Suggests Me DuoClustering2018” Nice getting the thing going, Charlotte!
Kevin Rue-Albrecht (11:26:57): > Haven’t checked when it got there
Federico Marini (11:27:33): > I am soon to put up a repo with the instance of the paper in blood i was mentioning
Federico Marini (11:27:47): > still did not get to the 1:1 plots I would like to have
Charlotte Soneson (11:27:55): > :slightly_smiling_face:not too long ago, package was accepted a couple of weeks ago
Kevin Rue-Albrecht (11:28:38): > I thought i couldn’t have missed a neat reference like this for long:slightly_smiling_face:
Federico Marini (12:08:28): > (I was asking because I didn’t want to push myself and grab the spotlight again)
Federico Marini (12:09:16): > the 2-sided presentation idea was to keep two slots to propose, one tool-centered, the other one ideally is we replicate a paper’s set of figures
Kevin Rue-Albrecht (12:10:57): > from my view grab the spotlight as much as you like: the nice thing about being 4 on the project is that we can each advertise it wherever we go
Federico Marini (12:12:05): > yep- this conf is with limited numbers, otherwise I would drop by like in HD:smile:
Kevin Rue-Albrecht (12:19:59): > Looks like you’re the most sure about going, in that case I’d suggest definitely consider taking the spotlight with an abstract about the tool aspect. > If a second of us decides to go, we can submit another abstract about the replication aspect. > At that point, I’m pretty sure the organizers won’t mind if the two speakers swap their talks ^^
Kevin Rue-Albrecht (12:20:48): > (not everyday one can see two speakers capable or presenting each other’s talk ^^)
Federico Marini (12:56:36): > quick shoutout here as well
Federico Marini (12:57:01): > I recall a tool/addin for checking the packages where funcs in a script/code are taken, and prepended
Federico Marini (12:57:11): > do any of you recall its name/location?
Federico Marini (13:13:23): > @Aaron Lun@Charlotte Soneson@Kevin Rue-AlbrechtI sent you an invitation on the private repo where we could store the different instances of iSEE on public/cool data
Federico Marini (13:13:55): > as of now this is private as we might want to do some experimental stuff, but I can set it public anytime if you prefer
Kevin Rue-Albrecht (13:38:24) (in thread): > reminds me of that recent package that was picking up code conflicts
Kevin Rue-Albrecht (13:38:42) (in thread): > at least conflicts between package functions
Kevin Rue-Albrecht (13:39:46) (in thread): > this one:https://github.com/r-lib/conflicted - Attachment (GitHub): r-lib/conflicted > An alternative conflict resolution strategy for R. Contribute to r-lib/conflicted development by creating an account on GitHub.
Federico Marini (18:07:46) (in thread): > close but not that one
Federico Marini (18:08:10) (in thread): > use case: “scan” a file and pick up exactly where each func is coming
2018-09-11
Federico Marini (04:25:35): > Folks, I did have a long waiting time at the bus stop
Federico Marini (04:25:40): > so I started typing
Federico Marini (04:25:50): > the draft for the rnaseq blog post
Federico Marini (04:25:56): > I came up with
Federico Marini (04:26:02): > > Data exploration is nowadays critical to the comprehension of large biological datasets obtained by high-throughput assays such as sequencing. > Prior to rigorous statistical analyses, novel data-driven hypotheses can be generated, enabling diagnosis of potential problems in the multi-faceted analysis workflow that follows, from quality control to downstream interpretation of the results, e.g. after detecting differentially expressed genes in the experimental conditions of interest. > Visualization and processing of the data in an intuitive and interactive interface is crucial. However, most existing tools for interactive visualization are limited to specific assays or analyses, and lack support for reproducible analysis. > > A group of researchers > ( > Federico Marini, Institute of Medical Biostatistics, Epidemiology and Informatics (IMBEI) and Center for Thrombosis and Hemostasis (CTH), Mainz; > Aaron Lun, Cancer Research UK Cambridge Institute, University of Cambridge; > Charlotte Soneson, Institute of Molecular Life Sciences, University of Zurich and SIB Swiss Institute of Bioinformatics > Kevin Rue-Albrecht, Kennedy Institute of Rheumatology, University of Oxford > ) > from different institutes gathered together at the Bioconductor European Developers' Conference. Their common aim of was to build a general tool to deal with a broad variety of datasets and data types, including large-scale bulk RNA-seq, single-cell RNA-seq and mass cytometry, just to name a few. Fast forwarding a few months, the iSEE package (Interactive SummarizedExperiment Explorer,[https://bioconductor.org/packages/iSEE](https://bioconductor.org/packages/iSEE)) was created, and is recently published at F1000Research ([https://f1000research.com/articles/7-741/v1](https://f1000research.com/articles/7-741/v1)). > Implemented in the Shiny framework, key features include: > - required input: any object based on the SummarizedExperiment class, fundamental for the whole Bioconductor infrastructure > - dynamically linked charts, transmitting the information across panels via brushing > - guided tours of the web applications, to learn step-by-step the salient features of the user interface and of the data > - automatic storage of the exact R code to generate every plot > > The utility and flexibility of this package can be demonstrated by deploying an instance of the tool to showcase salient features of a dataset at hand, ideally generating the figures for the publication with the tool itself, and building on top of that a guided tour, so that researchers can explore themselves the multiple facets omics data can offer. > Example instances are provided in the GitHub repository for the manuscript ([https://github.com/LTLA/iSEE2018/](https://github.com/LTLA/iSEE2018/)), can be run in the browser ([https://marionilab.cruk.cam.ac.uk/iSEE_allen](https://marionilab.cruk.cam.ac.uk/iSEE_allen),[https://marionilab.cruk.cam.ac.uk/iSEE_tcga](https://marionilab.cruk.cam.ac.uk/iSEE_tcga),[https://marionilab.cruk.cam.ac.uk/iSEE_pbmc4k](https://marionilab.cruk.cam.ac.uk/iSEE_pbmc4k),[https://marionilab.cruk.cam.ac.uk/iSEE_cytof)](https://marionilab.cruk.cam.ac.uk/iSEE_cytof)), or also retrieved as Docker images. Additional instances on recently published datasets can be found at[https://github.com/federicomarini/iSEE_instances/](https://github.com/federicomarini/iSEE_instances/), where Rmarkdown documents document the entire preprocessing steps and setup parameters. > > iSEE is publicly available as R package from the open-source Bioconductor project, as well is actively developed at[https://github.com/csoneson/iSEE/](https://github.com/csoneson/iSEE/) >
Federico Marini (04:26:30): > We could add something like we make a interview-like statement, maybe@Aaron Luncan take the mic for this?
Aaron Lun (04:26:41): > “Data exploration is now a solved problem”.
Federico Marini (04:27:17): > cocky but I’d agree:smile:
Federico Marini (04:27:53): > of course the list of our names can be shuffled
Charlotte Soneson (04:27:58): > cool! Shall we have a Google Doc or something where we can collaboratively edit?
Federico Marini (04:28:06): > I just followed the vignette and copied over
Federico Marini (04:28:10): > good point ma’am
Charlotte Soneson (04:28:35): > You mean “Dr”, right?
Federico Marini (04:28:48): > dr ma’am
Federico Marini (04:29:46): > title: data exploration made easy and reproducible?
Aaron Lun (04:31:07): > Data exploration for dummies.
Aaron Lun (04:31:20): > Biological data exploration for dummies.
Federico Marini (04:31:41): > Explore your data like a pro even if you’re dumb:stuck_out_tongue:
Federico Marini (04:31:43): > https://docs.google.com/document/d/11D01Nz3GisFnUUnEbCoJgIKjfF27a0_i9ue2QAV3cQs/edit
Federico Marini (04:31:51): > whoops
Federico Marini (04:31:56): > I think that should work
Aaron Lun (04:35:48): > That’s COMMANDER Lun to you, scum.
Aaron Lun (04:35:52): > Or fleet admiral.
Aaron Lun (04:36:00): > Man, government forms are hilarious, they’ve got all the titles.
Aaron Lun (04:36:09): > If I were a priest, you’d have to call me Father.
Aaron Lun (04:36:13): > Oh yeah…
Federico Marini (04:36:17): > well I can be that
Federico Marini (04:36:29): > ze fazer
Federico Marini (04:36:53): > I always come up with that voice when i usetar
Federico Marini (04:37:10): > for those of you who also forget the options for that:
Federico Marini (04:37:34): > Read it like a german would > - zip up:czf: compress ze file
Federico Marini (04:38:07): > - decompress:xzfxtract ze file
Federico Marini (04:43:55): > heading to a series of meetings till lunch
Federico Marini (04:44:11): > if you want to continue, the doc is now open to you as well to edit
Federico Marini (04:50:37) (in thread): > Aaron get your suit on and get Kev and Anna married:wink:
Aaron Lun (04:53:01) (in thread): > Oh yeah, I could do that.
Aaron Lun (04:53:05) (in thread): > If I were a priest.
Federico Marini (04:53:21) (in thread): > let the formula be
Federico Marini (04:53:49) (in thread): > “If reviewer #2 has something to add, speak up now or be silent forever”
Aaron Lun (04:55:52) (in thread): > lol
Kevin Rue-Albrecht (06:47:09): > Alrighty. I’ve put my edits in there
Kevin Rue-Albrecht (06:48:41): > I’ve explicitly listed all the web links to Docker images.. Looks overkill, so feel free to delete or reshape
Kevin Rue-Albrecht (06:48:54): > Nice job starting it though@Federico Marini
Federico Marini (06:49:19): > Merci
Federico Marini (06:49:48): > I’d say overkill for the blog post: the links to the demos can also be overkill, but at least they serve the “i click, i do right away” purpose
Federico Marini (06:49:56): > while docker imgs do not
Kevin Rue-Albrecht (06:50:25): > yup
Federico Marini (06:50:30): > (I felt it came out naturally while waiting for the damn bus)
Federico Marini (07:00:32): > @admiral@Aaron Lun, wanna take care of the final edits as closest to native?
Federico Marini (07:00:45): > not going to say the O word for Kev’s spot:slightly_smiling_face:
Aaron Lun (07:00:52): > I’ll deal with it at 1 pm precisely.
Federico Marini (07:01:09): > way to go
Aaron Lun (08:19:11): > Okay, I’m pretty happy with it. Just waiting for K’s edits.
Kevin Rue-Albrecht (08:19:49): > Should we include a screenshot within the blog post, or are the interactive apps enough?
Aaron Lun (08:20:31): > Guess you could throw in Figure 1.
Kevin Rue-Albrecht (08:22:33): > i’ve wiped out the last bit about the docker hub, those are linked anyway from the iSEE2018
Kevin Rue-Albrecht (08:30:58): > Alright. Figure 1 added, with a trimmed-down version of the original figure legend. Any edits?
Aaron Lun (08:40:00): > Fire away.
Aaron Lun (08:54:50): > Did someone send this?
Kevin Rue-Albrecht (08:55:03): > Not yet. (speaking for myself)
Charlotte Soneson (08:55:38): > No. But it looks good to me.
Kevin Rue-Albrecht (08:58:31): > For the record, I’ve pasted the screen capture copied from the iSEE readme file, itself pointing the f1000 article. Point is, the resolution looks fine as it is, but we might want to attach the original file from herehttps://github.com/LTLA/iSEE2018/blob/master/manuscript/pics/Fig1.svgto the email - Attachment (GitHub): LTLA/iSEE2018 > Code and manuscript files for the iSEE paper. Contribute to LTLA/iSEE2018 development by creating an account on GitHub.
Aaron Lun (09:00:32): > ¯*(ツ)*/¯
Aaron Lun (09:00:39): > Your effort, your call.
Kevin Rue-Albrecht (09:51:26): > Alright. Finally happy with my poster (I’m sure Steve and co-authors will have something to say still), but I can get onto the RNA-seq blog email if no one else has yet.
Federico Marini (09:51:48): > I was drafting the email as you write
Federico Marini (09:52:01): > is the addressadmin@rna-seqblog.com?
Kevin Rue-Albrecht (09:52:12): > oh right keep going then
Federico Marini (09:52:18): > I was picking it from your email you forwarded me
Federico Marini (09:52:43): > I would just write in the name of all of us, then paste 1:1 the content of the google doc
Kevin Rue-Albrecht (09:52:47): > I was going to check but if it’s in my earlier email it must be it
Kevin Rue-Albrecht (09:53:02): > I’ve posted the address somewhere in this channel too i think
Federico Marini (09:54:14): > Should i addiSEE:to the current title
Federico Marini (09:54:15): > ?
Kevin Rue-Albrecht (09:54:28): > here we go again - Attachment: Attachment > speaking of PR (public relation), Federico and I were wondering yesterday whether we should wait to see if the RNAseq blog picks up iSEE by themselves, or if our corresponding author should contact them to suggest the story (https://www.rna-seqblog.com/contact/)
Federico Marini (09:54:34): > it is always good to have the name, and not just the cheeky description
Kevin Rue-Albrecht (09:55:07): > good point
Kevin Rue-Albrecht (09:55:32): > I think BAN format and BMC bioinfo recommend the software name in the title
Federico Marini (09:56:18): > Ok, I see no disadvantages in “iSEE: Data exploration made easy and reproducible”
Kevin Rue-Albrecht (11:28:23): > have you sent it, or you’ll cc us in it?
Federico Marini (11:36:01): > sent it already
Federico Marini (11:36:23): > forgot to cc you. but I did mention you all in the text before pasting the draft
Aaron Lun (11:37:10): > :party_parrot:
Aaron Lun (11:37:18): > Keep us in the:loop:
Federico Marini (11:39:22): > you have my email FWD now
Federico Marini (11:40:05): > I never thought I would have said that but hey
Federico Marini (11:40:13): > I like writing the early drafts of proposals
Federico Marini (11:40:30): > especially in applied fields more than the technical one
Federico Marini (11:41:16): > and as a nice side effect, I showed those damn wetlab people that INDEED google docs is a huge advantage when drafting such collaborative things in restricted time window
Federico Marini (11:41:17): > s
Aaron Lun (11:41:38): > yeah, those fools and their dropboxes.
Federico Marini (11:52:33): > oh that would be already 2.0
Federico Marini (11:52:37): > I live in middle age
Federico Marini (11:52:54): > FILENAME_revFM_myrevisiondate
Federico Marini (11:53:10): > FILENAME_revFM_myrevisiondate_OTHER_AUTHOR_V2
Federico Marini (11:53:24): > parallel to that, FILENAME_revFM_myrevisiondate_AUTHOR3_v1
Federico Marini (11:54:09): > but hey, once you force/convince them to try, guess whether they go back to their original method?
Kevin Rue-Albrecht (12:30:55) (in thread): > keep us in the ‘hoot’ ?
Kevin Rue-Albrecht (12:31:10) (in thread): > :owl::loop:
2018-09-12
Federico Marini (16:33:42): > the trend looks promising
Federico Marini (16:33:42): > http://bioconductor.org/packages/stats/bioc/iSEE/
Federico Marini (16:33:48): > for september at least
Federico Marini (16:33:55): > we did have some drop in summer
Federico Marini (16:34:18): > but sep looks good to catch up with nicer hits
Kevin Rue-Albrecht (16:51:21): > nothing back from the blog yet?
Federico Marini (16:53:27): > nope
Kevin Rue-Albrecht (16:54:54): > bah. it’ll happen soon enough
Aaron Lun (17:41:17): > I think you should DEMAND an explanation.
2018-09-13
Federico Marini (05:21:05): > no need now:https://www.rna-seqblog.com/isee-an-interactive-shiny-based-graphical-user-interface-for-exploring-data-stored-in-summarizedexperiment-objects/ - Attachment (rna-seqblog.com): iSEE - an interactive Shiny-based graphical user interface for exploring data stored in SummarizedExperiment objects > Data exploration is critical to the comprehension of large biological datasets obtained by high-throughput assays such as sequencing. Interactive exploration fosters the generation of novel data-driven hypotheses prior to rigorous statistical analysis, enables diagnosis of potential problems during
Federico Marini (05:21:14): > they did some sliiight editing, but hey, we are in
Federico Marini (05:35:40): > Should I start submitting the abstract (or do they just want a title) for Munich eurobioc or do you guys want to do it?
Aaron Lun (05:36:12): > Knock yoruself out.
Aaron Lun (05:36:17): > I just got an email query about iSEE.
Aaron Lun (05:36:20): > So +1
Federico Marini (05:36:23): > YEAH
Charlotte Soneson (05:36:32): > Fine with me. I have gotten an OK to go, but I’ll talk about something else.
Federico Marini (05:36:35): > them good ol’ blog
Federico Marini (05:37:04): > we can still crowdsource the title:stuck_out_tongue:
Federico Marini (05:38:25): > for the abstract there is no limit of words, but I’ll put together 2 paras
Kevin Rue-Albrecht (05:58:22) (in thread): > User query? Reasonable or wish upon a star?
Aaron Lun (08:11:02) (in thread): > Reasonable, just asking for customization details.
Aaron Lun (08:11:09) (in thread): > This guy would be a power user if he gets on board.
Federico Marini (08:39:18) (in thread): > name?
Federico Marini (08:39:43) (in thread): > plus: good he got to us - was it via blog?
Aaron Lun (08:40:29) (in thread): > someone at the sanger
Aaron Lun (08:40:38) (in thread): > name is too polish to spell out, but it’s like christoff
Federico Marini (08:41:51) (in thread): > then again, very good:slightly_smiling_face:
Federico Marini (15:44:57): > (ps we’re doing quite good on the blog, among retweets and likes we got a sweet spot)
Federico Marini (15:45:28): > ((plus, we were the last post, i.e. we are pinned up for free til the next come))
Federico Marini (15:45:31): > :stuck_out_tongue:
Federico Marini (15:56:36): > a jackpot for advertisement would be to pick up also Stephen Turner
Federico Marini (15:56:43): > let’s see if he hops on
2018-09-14
Kevin Rue-Albrecht (06:50:42): > speaking of RNA-seq blog, did you guys see this one?https://www.rna-seqblog.com/single-cell-rna-seq-gives-scientists-unprecedented-view-of-brain-development/
Kevin Rue-Albrecht (06:51:12): > “Cell Seek Facilitates Interactive Exploration of the Developing Murine Cerebellum”
Kevin Rue-Albrecht (06:53:31): > I wonder how long it takes them to generate that Monocle2 pseudotime ‘on-the-fly’
Federico Marini (08:27:28): > I guess a lot?
2018-09-15
Aaron Lun (12:11:01): > Ugh. Hardcore bugfixes infindMarkers.
Aaron Lun (12:11:08): > Been debugging for three hours straight now.
Kevin Rue-Albrecht (13:19:06): > scran, I assume
Aaron Lun (13:19:54): > It’s done.
Aaron Lun (13:20:07): > Going for a walk now.
Kevin Rue-Albrecht (13:20:13): > I’ve been looking at the README of iSEE for a bit. I was about to add more unit tests, but then I noticed that the readme is still very much “we will” while in fact “we have”
Aaron Lun (13:20:50): > Knock yourself out. When I come back from my walk, it’s time for some C-level HDF5 debugging… yay.
Kevin Rue-Albrecht (13:21:20): > ouch
Kevin Rue-Albrecht (13:21:39): > haven’t taken the time to learn any C yet, am I missing out?
Kevin Rue-Albrecht (14:05:32): > alright. the name of the branch is clearly wrong, but I was too lazy to rename it once I started editing the readme:testthat
Kevin Rue-Albrecht (14:05:49): > check outhttps://github.com/csoneson/iSEE/tree/testthatto see how it looks now - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (14:09:54): > I think I’m going to update the panel types in bold rather than italic
Kevin Rue-Albrecht (14:10:21): > and i forgot to mention the custom panels. But after those I’ll stop, because I already find the readme rather long like this
Aaron Lun (15:21:01): > looks fine to me. I was going to suggest keeping theREADMEminimal in the same style thathttps://github.com/drisso/SingleCellExperimentuses, but this is fine. - Attachment (GitHub): drisso/SingleCellExperiment > Clone of the Bioconductor repository for the SingleCellExperiment package, see https://bioconductor.org/packages/devel/bioc/html/SingleCellExperiment.html for the official development version. - dr…
Kevin Rue-Albrecht (15:22:29): > oh I remember the “Project Status” from somwhere. We should get “hyperactive”:wink:
Kevin Rue-Albrecht (15:28:32): > If we want to trim the readme, I wouldn’t have a problem dropping sections from “Sample-level visualization” to “Miscellaneous” (basically the list of panel types). > i.e. keeping up to the main “Functionalities” and “Want to try iSEE?” to still give a bit of visibility and incentive to take the package for a spin
Aaron Lun (18:11:58): > Check out the hardcore HDF5 debugging at#bigdata-rep.
Kevin Rue-Albrecht (19:04:01): > Ouch. I read the whole thing, but I stopped understanding about 2 paragraphs in
2018-09-16
Kevin Rue-Albrecht (11:45:12): > Unit testing when “horizontal violins receive a select by restriction effect”https://codecov.io/gh/csoneson/iSEE/src/0036e4ca184744ffd2b2b9ddca0766b2a43a415c/R/plotting.R#L567:dealwithit-parrot:
Aaron Lun (11:57:24): > I don’t know what I’m looking at here.
Kevin Rue-Albrecht (11:57:56): > I’m tackling the 100 lines that are not yet covered by unit tests in plotting.R
Aaron Lun (11:58:19): > okay
Kevin Rue-Albrecht (11:59:50): > Here you’re looking at.choose_plot_typein a special case. It’s not necessarily obvious from the linked snippet, butexists("plot.data.all", envir)is TRUE when a selection to restrict is active, and theifblock that’s not covered is specifically for horizontal violins (thanks for leaving the comment there!)
Kevin Rue-Albrecht (12:00:51): - File (PNG): Screenshot 2018-09-16 17.00.39.png
Kevin Rue-Albrecht (12:01:03): > anyway, I just finished dealing with the heatmap code
Aaron Lun (12:01:07): > Yeah… nudging them up.
Aaron Lun (12:01:15): > Okay, I gotta go to someone’s farewell party
Aaron Lun (12:01:23): > see you in a few hours
Kevin Rue-Albrecht (12:01:26): > now, plotting.R is the last big chunk, which should bump us near 98%
Kevin Rue-Albrecht (12:01:39): > enjoy then!
Kevin Rue-Albrecht (12:02:06): > btw, I’ll be at the Wellcome Genome Campus Conference Centre from Monday to Thursday this week
Aaron Lun (12:03:00): > okay. I’ll probably be at Hinxton on Tues, but we could pub around if you’re in the city.
Kevin Rue-Albrecht (12:04:33): > I’ll have to check out the program again and let you know when the conference dinner and “unmissable stuff” is, but happy to organise something
2018-09-17
Kevin Rue-Albrecht (03:49:38): > btw,https://coursesandconferences.wellcomegenomecampus.org/wp-content/uploads/2018/08/Genome-Informatics-Aug-18-V4.pdfpage 9-10 (today), “Session 1: Data Curation, Integration, and Visualization”
Aaron Lun (05:41:56): > hm.
Aaron Lun (05:42:05): > Guess I’ll see you tomorrow then.
Kevin Rue-Albrecht (05:55:18): > Hopefully I can evade after my poster session
Kevin Rue-Albrecht (05:56:39): > It’s running 18:15-19:30:sleepy:
Kevin Rue-Albrecht (05:56:55): > But still a better shot than the conférence dinner Wednesday evening
Kevin Rue-Albrecht (08:03:27): > we’re officially in the last 10 lines that can be covered by unit tests (Current state: 99.24%, iSEE_extras requires a Shinysessionand I’ll point out that annotations.R requires an internet connection to pass) - File (PNG): Screenshot 2018-09-17 13.01.15.png
Aaron Lun (08:05:11): > God, the conference must be boring.
Kevin Rue-Albrecht (08:05:12): > let’s try to push unit tests concurrently with code, at least on master
Kevin Rue-Albrecht (08:05:29): > the welcome lunch starts only now
Kevin Rue-Albrecht (08:09:22): > But the bus oxford London, and train London Cambridge gave me time, tbh
Federico Marini (10:45:19): > As promised, here’s a draft of what I would submit in the name of the 4 of us:wink:
Federico Marini (10:45:27): > > Data exploration is crucial for the comprehension of large biological datasets obtained by high-throughput assays such as sequencing. > Interactive exploration fosters the efficient generation of novel data-driven hypotheses prior to rigorous statistical analysis, enables diagnosis of potential problems during quality control and facilitates interpretation of the results in the context of a specific scientific question. > Most existing tools for intuitive and interactive visualization are limited to specific assays or analyses and lack support for reproducible analysis. > > Following a fruitful collaboration initiated at #EuroBioc2017, we have built a general-purpose tool, iSEE - Interactive SummarizedExperiment Explorer, designed to accommodate any experimental data (bulk RNA-seq, single-cell RNA-seq, mass cytometry, ...) and/or associated metadata stored in an instance of a SummarizedExperiment container. > We illustrate its salient features, including: > - customizable interface with different panel types, linked among each other via user selection > - automatic tracking, storage, and rendering of the exact R code to generate all visible plots > - interactive tours to showcase datasets and findings, with step-by-step description of relevant publication-ready plots > > Example applications are available online to demonstrate the interactive exploration of the TCGA RNA sequencing data ([https://marionilab.cruk.cam.ac.uk/iSEE_tcga](https://marionilab.cruk.cam.ac.uk/iSEE_tcga)), single-cell RNA sequencing data ([https://marionilab.cruk.cam.ac.uk/iSEE_allen](https://marionilab.cruk.cam.ac.uk/iSEE_allen),[https://marionilab.cruk.cam.ac.uk/iSEE_pbmc4k](https://marionilab.cruk.cam.ac.uk/iSEE_pbmc4k)), and mass cytometry data ([https://marionilab.cruk.cam.ac.uk/iSEE_cytof](https://marionilab.cruk.cam.ac.uk/iSEE_cytof)). >
Federico Marini (10:46:32): > Title-wise, I am torn between > 1 - something a laiSEEing is believing > 2 - iSEE: data exploration made easy & reproducible
Kevin Rue-Albrecht (10:53:40): > Unrelated: nice plots inclustreepackage by Luke Zappia
Kevin Rue-Albrecht (10:54:01): > @Federico MariniDraft for what again?
Federico Marini (10:54:11): > eurobioc2018
Kevin Rue-Albrecht (10:55:30): > ah right of course
Kevin Rue-Albrecht (11:03:48) (in thread): > Should put it on Google Drive if you’d like us to suggest edits
Kevin Rue-Albrecht (11:04:05) (in thread): > More legible too
Federico Marini (11:05:16) (in thread): > didn’t want to bother you too much - but happy to hear out feedback
Federico Marini (11:06:43): > ->https://docs.google.com/document/d/1PtADHs_vJagv24vHt3Kvxi3JjOA1ifv6Uz_jfR5a2Dg/edit
Kevin Rue-Albrecht (11:06:50): > https://www.ebi.ac.uk/gxa/homeapparently ~53k cells from ~43 experiments across ~9 species available there - Attachment (ebi.ac.uk): Home > EMBL-EBI Expression Atlas, an open public repository of gene expression pattern data under different biological conditions
Federico Marini (11:06:55): > here’s the googledoc as suggested by kev
Kevin Rue-Albrecht (11:07:00): > could be a good playground for iSEE demos
Federico Marini (11:07:30): > uh, they did some massive restyling
Federico Marini (11:07:46): > I recall visiting that website and it was looking pretty 90s-style:smile:
Kevin Rue-Albrecht (11:09:51): > single cell gene expression by microarray (?!)https://www.ebi.ac.uk/gxa/experiments/E-GEOD-18617/Results - Attachment (ebi.ac.uk): Experiment > EMBL-EBI Expression Atlas, an open public repository of gene expression pattern data under different biological conditions
Kevin Rue-Albrecht (11:11:14) (in thread): > no bother, I can’t focus on it right now, but we can have a look and suggest small edits here and there if you like
Federico Marini (11:11:46) (in thread): > sure. I’d like to submit by this friday, so that we’re in “early”
Kevin Rue-Albrecht (11:14:54): > my god I’m so addicted to iSEE now: with that website, they distribute the data in ExpressionSet objects, I literally just loaded the RData file and callediSEE()on it. > See RData download link there:https://www.ebi.ac.uk/gxa/experiments/E-GEOD-18617/Downloads?geneQuery=%5B%7B%22value%22:%22Gpr126%22%7D,%7B%22value%22:%22Gfap%22%7D,%7B%22value%22:%22Nefl%22%7D,%7B%22value%22:%22Glial%20fibrillary%20acidic%20protein%22%7D,%7B%22value%22:%22Actb%22%7D,%7B%22value%22:%22growth%20factor%22%7D,%7B%22value%22:%22Lgi4%22%7D,%7B%22value%22:%22GPCR%22%7D,%7B%22value%22:%22Slc1a3%22%7D,%7B%22value%22:%22Plp1%22%7D,%7B%22value%22:%22actin%22%7D,%7B%22value%22:%22Actg%22%7D,%7B%22value%22:%22Ppp1ca%22%7D,%7B%22value%22:%22Fabp7%22%7D,%7B%22value%22:%22Sept4%22%7D,%7B%22value%22:%22Cyp26b1%22%7D,%7B%22value%22:%22Ank2%22%7D,%7B%22value%22:%22ankyrin%202%22%7D,%7B%22value%22:%22Nrxn3%22%7D,%7B%22value%22:%22Dlc1%22%7D,%7B%22value%22:%22notch%22%7D,%7B%22value%22:%22Gdf10%22%7D,%7B%22value%22:%22brain%20lipid%20binding%20protein%22%7D,%7B%22value%22:%22GLAST%22%7D,%7B%22value%22:%22Mtap2%22%7D%5D - Attachment (ebi.ac.uk): Experiment > EMBL-EBI Expression Atlas, an open public repository of gene expression pattern data under different biological conditions
2018-09-18
Aaron Lun (04:09:40): > Talk to me.
Aaron Lun (04:09:55): > Should we meet up during your lunch?
Kevin Rue-Albrecht (04:13:59): > Hey. If lunch is possible for you it’s probably easiest
Kevin Rue-Albrecht (04:16:26): > In a previous message you mentioned pubs if I’m in the city, but everything is on site and I don’t really know my way around so that would be less likely I imagine
Aaron Lun (04:18:16): > yeah okay. I guess you’d have to take a taxi in and out if you went to the city.
Aaron Lun (04:18:39): > Are you taking a train back to London? Because then we could have a quickie on Thursday.
Kevin Rue-Albrecht (04:18:41): > yeah, shuttle times don’t sound like an option ^^
Kevin Rue-Albrecht (04:19:24): > Actually, I was thinking about Thursday as well, although the conference ends at lunch time and I don’t know how that would fit with your own schedule
Kevin Rue-Albrecht (04:19:59): > They’ve organised shuttles back to Cambridge Rail Station after lunch at 13:30 on Thursday
Aaron Lun (04:20:01): > I can pub at any time.
Kevin Rue-Albrecht (04:33:06): > So I’d imagine being at Cambridge train station around 2pm-ish. If you want we can do 2-3pm(ish)
Kevin Rue-Albrecht (04:33:30): > but if you’re also up for lunchtime today, let me know if you prefer one, the other, or both
Aaron Lun (04:39:04): > yeah, when’s your lunch?
Kevin Rue-Albrecht (04:52:16): > 13:10 – 14:30
Aaron Lun (06:24:26): > you’re at the conference center, right?
Kevin Rue-Albrecht (06:37:11): > Yep
Kevin Rue-Albrecht (06:37:43): > We have a break til 10:50 (probably too late to be relevant now )
Kevin Rue-Albrecht (06:37:59): > AViDE - File (JPEG): Image from iOS
Aaron Lun (06:45:04): > I also have a poster for SCG. I’ll let you know when it’s finished.
Kevin Rue-Albrecht (06:46:50): > Ok
Federico Marini (07:25:51): > wow. they cite shiny not as the package but rather from a book about it?
Federico Marini (07:27:38): > no repo available?
Aaron Lun (08:14:31): > I’m comming
Kevin Rue-Albrecht (08:27:46): > We just sat down for lunch across the confèrence centre)
Kevin Rue-Albrecht (08:28:10): > When/where would you like to meet ?
Aaron Lun (08:29:08): > yeah, I just realized that.
Aaron Lun (08:29:13): > Couldn’t insert myself into the room.
Aaron Lun (08:29:23): > But if you’d like, I can get you a coffee later
Kevin Rue-Albrecht (08:29:31): > I can do a speedy lunch. Did you have yours yet?
Aaron Lun (08:30:12): > Yeah, I’ve had mine. Let me know when you’re about to get out.
Aaron Lun (08:30:33): > I’ll meet you in Murray’s (the big cafeteria on the hill-ish part)
Kevin Rue-Albrecht (08:31:49): > Ok. I can probably be out in 20 min and find the cafeteria 10 min later (?) That would give us 30 min before the session although as I said I might skip a couple of talks if we want the time
Aaron Lun (08:32:23): > okay
Aaron Lun (08:51:03): > Going to the toilet, back in a bit.
Aaron Lun (08:52:16): > toilets were full so I’ll just hold it.
Kevin Rue-Albrecht (08:55:46): > Hahahaha
Federico Marini (08:56:54): > almost as good as live tweeting
Federico Marini (08:56:55): > :smile:
Kevin Rue-Albrecht (08:56:58): > I’m passing the Sanger, now i remember the place from a previous quick visit
Kevin Rue-Albrecht (08:57:34): > @Federico Marinifull traceability, the iSEE Spirit:stuck_out_tongue_winking_eye:
Aaron Lun (08:58:03): > okay, I’m coming down
Federico Marini (08:58:41): > trust me I would not be that much intobrushingin this case
Kevin Rue-Albrecht (11:10:13): > I’m just reminded of the small rate of no-show posters at conferences: I should have brought a spare iSEE poster incognito:sunglasses:
Kevin Rue-Albrecht (11:10:52): > « And the poster prize goes to … this unregistered one »:stuck_out_tongue_winking_eye:
Federico Marini (11:22:20): > ahah
Federico Marini (11:22:34): > I once won a poster prize for an interactive one, done with paper
Federico Marini (11:23:12): > it was about pcaExplorer, and looked like an advent calendar - each function with a panel, to lift, and below there was a sample output for that functionality
Federico Marini (11:23:25): > and below one panel I had a goat
Kevin Rue-Albrecht (11:46:49): > goat? that’s diabolical:wink:
Kevin Rue-Albrecht (11:50:39): > just to follow up on the chat with Aaron (and a couple of messages with Charlotte before), we’re basically there for the next release (end of October), I just need to bump up the version number and merge the new unit tests.
Kevin Rue-Albrecht (11:51:34): > Here is the working example for the “DNA-Sequencing dataset from the 1000 Genomes Project”, from the ExperimentHub
Kevin Rue-Albrecht (11:51:42): > > library(ExperimentHub) > library(snpStats) > library(iSEE) > > # Fetch data ---- > > require(ExperimentHub) > ehub <- ExperimentHub() > > EH550 <- ehub[["EH550"]] > # closefn.gds(EH550) > # f <- openfn.gds(EH550$filename) > > (node <- index.gdsn(EH550, "genotype")) > genotype <- read.gdsn(node) > > (node <- index.gdsn(EH550, "sample.annot/population")) > population <- read.gdsn(node) > > (node <- index.gdsn(EH550, "sample.annot/gender")) > gender <- read.gdsn(node) > > closefn.gds(EH550) > > # Random sampling of variants ---- > set.seed(1234) > idx <-[sample.int](http://sample.int)(nrow(genotype), 1E6) > > # PCA ---- > > require(snpStats) > snps.class <- new("SnpMatrix", t(genotype[idx, ])) > xxmat <- xxt(snps.class, correct.for.missing=FALSE) > evv <- eigen(xxmat, symmetric=TRUE) > pcs <- evv$vectors[,1:5] > > # iSEE ---- > > sce <- SingleCellExperiment( > assays=list(genotype=genotype[idx, ]), > colData=DataFrame( > population=population, > gender=gender > ), > reducedDims=list(PCA=pcs)) > > initialPanels <- DataFrame( > Name=c("Reduced dimension plot 1", "Column data plot 1", "Feature assay plot 1", "Row statistics table 1", > "Column statistics table 1", "Heat map 1"), > Width=c(4L, 4L, 4L, 4L, 4L, 4L), > Height=c(500L, 500L, 500L, 500L, 500L, 500L) > ) > > rd <- redDimPlotDefaults(sce, 1) > rd$ColorBy <- "Column data" > rd$ColorByColData <- "population" > > app <- iSEE(sce, redDimArgs = rd, initialPanels=initialPanels, sampAssayMax=0) >
Kevin Rue-Albrecht (11:52:11): > it includes the computation of reduced dimensions from the SNP data
Kevin Rue-Albrecht (11:53:20): > (Note that it’s using a subset of SNPs, to save computation time)
Federico Marini (11:55:31): > want to make it an extraiSEE_instance?
Federico Marini (11:56:17): > doesn’T have to be coolest cutting edge stuff, as long as it is a working demo / the code to set it up
Kevin Rue-Albrecht (11:56:32): > ultimately yes, but my expertise of SNP analysis is relatively limited, so I’m afraid of hurting people’s feeling with a botched demo
Federico Marini (11:56:55): > it is a private repo, so we would be failing in a safe environment:smile:
Kevin Rue-Albrecht (11:57:01): > true
Kevin Rue-Albrecht (11:57:58): > and@Aaron Lun, the heatmapPlot panel fails for character matrix assays, e.g. > > assayMatrix <- genotype[idx, ] > storage.mode(assayMatrix) <- "character" > > with error:Discrete value supplied to continuous scale
Kevin Rue-Albrecht (12:00:24): > Doesn’t sound like the most difficult error to fix. We just missed a switch somewhere for discrete assays, I assume
Kevin Rue-Albrecht (12:10:49): > On a separate note, from both the chat with Aaron and a few messages with Charlotte, the Gviz browser feature is a considerable undertaking and we’re all fairly short on time these days, so it’s good to keep thinking about it for a future release
Kevin Rue-Albrecht (12:11:10): > Especially if we’re thinking about supporting external files.
Kevin Rue-Albrecht (12:12:44): > In terms of self-contained (at least in annotation packages), I wanted to share those prototype functions to handle GeneRegionTrack, both “reference” ones from Ensembl, and taken from the SCErowRanges: > > EnsemblGeneId <- function(feature.name, orgdb, feature.type="ENSEMBL") { > if (feature.type != "ENSEMBL") > if (!feature.name %in% keys(orgdb, feature.type)) { > feature.name <- NA > } else { > feature.name <- mapIds(orgdb, feature.name, "ENSEMBL", feature.type) > } > feature.name > } > # EnsemblGeneId("Axdnd1", org.Mm.eg.db, "SYMBOL") > > EnsemblVisibleRange <- function(ensId, ensdb) { > > queryGeneRange <- range(subset(transcripts(ensdb), gene_id == ensId)) > > queryGeneRange > } > # EnsemblVisibleRange("ENSMUSG00000026601", EnsDb.Mmusculus.v79) > > EnsemblGeneRegionTrack <- function(range, ensdb) { > > annotationFilterList <- AnnotationFilterList( > GRangesFilter(range, feature = "gene", type = "within") > ) > > exonsGRangesList <- exonsBy( > ensdb, by="tx", > columns=c( > listColumns(ensdb, "exon"), > listColumns(ensdb, "tx2exon"), > "symbol"), > filter=annotationFilterList) > exonsGRanges <- unlist(exonsGRangesList) > > GeneRegionTrack( > range=exonsGRanges, > transcript=exonsGRanges$tx_id, > showId=TRUE, geneSymbol=TRUE, symbol=exonsGRanges$symbol, > name="Ensembl") > } > # EnsemblGeneRegionTrack(GRanges("1", IRanges(156323509, 156421159)), EnsDb.Mmusculus.v79) > > SceGeneRegionTrack <- function(range, se) { > > exonsGRangesList <- rowRanges(subsetByOverlaps(sce, range)) > exonsGRanges <- unlist(exonsGRangesList) > > GeneRegionTrack( > range=exonsGRanges, > transcript=exonsGRanges$tx_id, > showId=TRUE, geneSymbol=TRUE, symbol=exonsGRanges$symbol, > name="SummarizedExperiment") > > } > # SceGeneRegionTrack(GRanges("1", IRanges(156323509, 156421159)), se) > > make_GvizPlot <- function(se, feature.name, orgdb, ensdb, feature.type="ENSEMBL") { > > queryGeneId <- EnsemblGeneId(feature.name, orgdb, feature.type) > > queryGeneRange <- EnsemblVisibleRange(queryGeneId, ensdb) > > egrt <- EnsemblGeneRegionTrack(queryGeneRange, EnsDb.Mmusculus.v79) > sgrt <- SceGeneRegionTrack(queryGeneRange, se) > > plotTracks(trackList = list(egrt, sgrt), sizes = c(1, 1)) > } > make_GvizPlot(sce, "Axdnd1", org.Mm.eg.db, EnsDb.Mmusculus.v79, feature.type="SYMBOL") >
Kevin Rue-Albrecht (12:12:57): > (it works for theallendataset
Charlotte Soneson (12:15:20): > Right now it works quite ok via a custom panel (with the external files), we could provide an example of this somewhere to show that it can be done? - File (PNG): Screen Shot 2018-09-15 at 11.20.24.png
Kevin Rue-Albrecht (12:17:00): > Indeed, just like@Federico Marinisuggested the repo for more demos, we should probably also set up something to demonstrate ideas of custom panels. I think the custom panels are still very abstract in the way we describe them
Kevin Rue-Albrecht (12:18:46): > Users would probably get a better idea of all the wonderful things they can do, if we give them a few examples like this
Kevin Rue-Albrecht (12:19:07): > Even better if we had a few screenshots/thumbnails
Kevin Rue-Albrecht (12:19:48): > It could be like the BiocStickers: the (small) screenshot could link to the code of the custom panel
Federico Marini (15:22:26): > i just logged in and saw your msg kev:stuck_out_tongue:
Federico Marini (15:24:00): > plus, Marcel put -> I don’t always useSummarizedExperiment, but when I do,iSEEit:party_parrot:
Kevin Rue-Albrecht (17:44:41): > Yeah I saw marcel’s one a few days ago and have been working on mine ever since:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (18:15:59): > ehm guys.. completely randomly, I talked with a certain Jonathan Manning who developed this fancy packagehttps://github.com/pinin4fjords/shinyngs - Attachment (GitHub): pinin4fjords/shinyngs > Shiny apps for NGS etc based on reusable components created using Shiny modules - pinin4fjords/shinyngs
Kevin Rue-Albrecht (18:16:21): > he came at my poster, but we ended up diverging for some reason..
2018-09-19
Federico Marini (03:20:29): > I know the pkg
Federico Marini (03:20:53): > it is module-based, very interlinked, quite a nice effort if you ask me
Federico Marini (03:21:01): > post analysis as well, if I recall
Kevin Rue-Albrecht (03:31:10): > And apparently done by a single person over the last 2 years. Impressive indeed
Kevin Rue-Albrecht (03:32:29): > That’s another thing we mentioned with Aaron yesterday: whether we could break down the iSEE server function into modules
Kevin Rue-Albrecht (03:33:32): > Although neither of us wrote Shiny modules before. For me it was a couple of years ago that I looked at the doc, but it wasn’t very clear to me
Kevin Rue-Albrecht (03:34:42): > On a completely separate note, my Slack loads too fast: I rarely have long enough to read even the first few words of the message:cry:
Federico Marini (03:45:00): > mine as well. so i logout again, cap the connection, cap the processor performance, and log in again:stuck_out_tongue:
Federico Marini (03:45:49): > re: modules, I never got to use them either
Kevin Rue-Albrecht (03:50:46): > ahhhh nice hack
Kevin Rue-Albrecht (03:52:52): > re: modules, I don’t know how much work that would be (for no apparent change to the behaviour), but I assume there would be some benefits (e.g. to add new panels or something along those lines)
Federico Marini (04:02:58): > Probably- yet, my guess: having this “strongly customised ui” I don’t know if it is really beneficial
Federico Marini (04:03:20): > we have enough of linking everything to everything, handling panel dependencies and more
Kevin Rue-Albrecht (04:05:37): > Rafa’s first point:sunglasses: - File (JPEG): Image from iOS
Kevin Rue-Albrecht (04:08:45): > I like him:grin: - File (JPEG): Image from iOS
Federico Marini (04:12:20): > Did you bring your iSEE banner in the audience?
Kevin Rue-Albrecht (04:12:38): > We need one:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (04:12:55): > And t shirts:grin:
Federico Marini (04:14:56): > slightly unrelated: thanks Kev and Aaron for the feedback. I’ll wait and see if@Charlotte Sonesonhas something to add, then I’ll edit at once
Charlotte Soneson (04:15:32): > oops, sorry, forgot about this. I’ll look now
Charlotte Soneson (04:19:38): > yeah, I think this is good, nothing really important to add
Federico Marini (04:27:21): > ok, I’ll finish up then today
Federico Marini (04:27:41): > no hurry - guess you’re mega busy these days
Federico Marini (06:14:01): > for new instances of iSEE: I took the liberty of building up on your pkg, Charlotte
Federico Marini (06:14:16): > and went on with an Rmd for processing the smartseq data
Federico Marini (06:14:21): > still a WIP
Federico Marini (06:14:48): > but I would like to try to get figures similar to what they show in the paper on biorXiv
Charlotte Soneson (06:16:42): > Ehm, which package?
Federico Marini (07:18:25): > tabulamurisdata
Federico Marini (07:19:07): > I’m grabbing them and preprocessing (norm, red dims, …)
Kevin Rue-Albrecht (07:28:34): > Neat!
Federico Marini (07:34:17): > I did finish up the edits, if you want to have a look. I accepted most of them, indeed it is more readable, so thanks once again
Charlotte Soneson (07:34:45): > Ah, ok. Yeah, I just made the bare minimum in the vignette (but it Suggests iSEE:slightly_smiling_face:)
Federico Marini (07:35:58): > I noticed that sadlyafterwriting 70% of the example:disappointed:
Federico Marini (07:36:30): > good thing we had the same aim
Kevin Rue-Albrecht (07:38:30): > Nice talk from Barbara Engelhardt this morning about GPLVM (nonlinear manifold), and lots of visualisation. A bit disappointed by 3D plots that are so hard to read, so I’ll definitely suggest showcasing the method in iSEE instances:innocent:
Charlotte Soneson (07:42:41) (in thread): > Btw, the package is in Bioc devel now, so maybe the url should point to that page instead.https://bioconductor.org/packages/devel/data/experiment/html/TabulaMurisData.html - Attachment (Bioconductor): TabulaMurisData (development version) > Access to processed 10x (droplet) and SmartSeq2 (on FACS-sorted cells) single-cell RNA-seq data from the Tabula Muris consortium (http://tabula-muris.ds.czbiohub.org/).
Federico Marini (07:43:46) (in thread): > congrats!
Federico Marini (07:44:17) (in thread): > I’ll use the “generic” link
Federico Marini (07:44:18) (in thread): > http://bioconductor.org/packages/TabulaMurisData/ - Attachment (Bioconductor): TabulaMurisData (development version) > Access to processed 10x (droplet) and SmartSeq2 (on FACS-sorted cells) single-cell RNA-seq data from the Tabula Muris consortium (http://tabula-muris.ds.czbiohub.org/).
Charlotte Soneson (07:44:19) (in thread): > Thanks:slightly_smiling_face:doesn’t take so much, there’s almost no code in there
Charlotte Soneson (07:44:24) (in thread): > yes, right
2018-09-20
Kevin Rue-Albrecht (04:54:49): > While a PhD student presents about moss genomics, it just got me to find this “1000 plants project”. Obviously different genomes and datasets, but they could use some visualizationhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC4306014/ - Attachment (PubMed Central (PMC)): Data access for the 1,000 Plants (1KP) project > The 1,000 plants (1KP) project is an international multi-disciplinary consortium that has generated transcriptome data from over 1,000 plant species, with exemplars for all of the major lineages across the Viridiplantae (green plants) clade. Here, we …
Kevin Rue-Albrecht (04:55:46): > “From the data window it is possible to download individual files or perform bulk downloads of multiple files and directories through a Java plugin.”
Aaron Lun (05:10:34): > lol
Federico Marini (05:12:36): > I’m a big FAN of the broad login-before-any-download policy:stuck_out_tongue:
Kevin Rue-Albrecht (05:13:18): > “data is available upon request.. of the password”:wink:
Federico Marini (05:15:33): > the best is “available upon REASONABLE REQUEST”
Federico Marini (05:16:09): > got it from this piece
Federico Marini (05:16:12): > http://www.pnas.org/content/115/11/2584 - Attachment (PNAS): An empirical analysis of journal policy effectiveness for computational reproducibility > A key component of scientific communication is sufficient information for other researchers in the field to reproduce published findings. For computational and data-enabled research, this has often been interpreted to mean making available the raw data from which results were generated, the computer code that generated the findings, and any additional information needed such as workflows and input parameters. Many journals are revising author guidelines to include data and code availability. This work evaluates the effectiveness of journal policy that requires the data and code necessary for reproducibility be made available postpublication by the authors upon request. We assess the effectiveness of such a policy by ( i ) requesting data and code from authors and ( ii ) attempting replication of the published findings. We chose a random sample of 204 scientific papers published in the journal Science after the implementation of their policy in February 2011. We found that we were able to obtain artifacts from 44% of our sample and were able to reproduce the findings for 26%. We find this policy—author remission of data and code postpublication upon request—an improvement over no policy, but currently insufficient for reproducibility.
Federico Marini (05:16:22): > which btw contains other gallery of horrors
Federico Marini (05:16:50): > I’m sorry, but our computer code was not written with an eye toward distributing for other people to use
Aaron Lun (05:22:41): > ha lol
Federico Marini (06:21:07): > “maybe not significant but i do see a trend” ->http://bioconductor.org/packages/stats/bioc/iSEE/
Kevin Rue-Albrecht (11:23:11): > They should put significance stars between consecutive months:wink:
Kevin Rue-Albrecht (11:24:03): > perhaps based on a null distribution that assumes that “most packages don’t change in downloads”
Kevin Rue-Albrecht (11:24:19): > i.e., a trimmed mean of delta-downloads
Kevin Rue-Albrecht (11:26:31): > On a separate note, I rediscovered the GEOquery package:https://www.bioconductor.org/packages/release/bioc/vignettes/GEOquery/inst/doc/GEOquery.html“Getting GSE Series Matrix files as an ExpressionSet” > Not bad considering that they can be converted to SummarizedExperiment
Kevin Rue-Albrecht (11:27:16): > That would be a whole other playground for iSEE demos, on top of the ExperimentHub.
Kevin Rue-Albrecht (11:28:16): > For some reason, the vignette examplegse2553 <- getGEO('GSE2553',GSEMatrix=TRUE)works perfectly, but all the other accession numbers I’ve tried return objects with emptyexprsslot
Kevin Rue-Albrecht (11:32:42): > Anyway, that could be a very convenient workhorse to showcase demos replicating published work… as the data are generally available on GEO
Kevin Rue-Albrecht (11:40:31): > Another set of 43 single-cell data sets:https://www.ebi.ac.uk/gxa/sc/experiments - Attachment (ebi.ac.uk): Experiments > EMBL-EBI Single Cell Expression Atlas, an open public repository of single cell gene expression data
Federico Marini (11:42:27): > Single-cell RNA-seq analysis of 1,522 cells of the small intestinal epithelium -> assays: 1521
Federico Marini (11:42:30): > :smile:
Federico Marini (11:42:43): > but yes, they do put some efforts
Federico Marini (11:42:52): > so these are quite free playgrounds
Kevin Rue-Albrecht (11:43:15): > anyone else notices that they don’t seem to have any interactive interface to visualize the data sets?
Kevin Rue-Albrecht (11:44:05): > i’m saying “user base” …:grimacing:
Federico Marini (11:45:21): > well they do, at least tsne + some interactive plots
Kevin Rue-Albrecht (11:45:52): > oh. did i miss a link? oops
Federico Marini (11:45:54): > pretty limited
Federico Marini (11:45:57): > but still
Federico Marini (11:46:01): > click on any dataset
Kevin Rue-Albrecht (11:46:34): > argh.. sorry.. connection was so slow that I navigated away before it loaded
Kevin Rue-Albrecht (11:46:41): > i’m on the bus back to O-place
Kevin Rue-Albrecht (11:47:49): > I don’t get how Cambridge always seems to have better weather when I visit:neutral_face:
Federico Marini (11:47:55): > I’m pretty done after some lifting done to my other pkgs, cleaning up and stuff like that
Kevin Rue-Albrecht (11:48:19): > sounds like our kind of housecleaning xD
Federico Marini (11:48:36): > and feel quite ashamed if i compare my code from me in solo to the one we got to do the 4 of us with way better “ground rules”
Kevin Rue-Albrecht (11:48:53): > the miss would be happy if I put nearly as much effort in the house as in R packages ^^
Federico Marini (11:49:02): > eheh
Kevin Rue-Albrecht (11:49:15) (in thread): > I know exactly how you feel
Federico Marini (11:49:25): > do worry if she puts pun-stickers on objects
Kevin Rue-Albrecht (11:49:30) (in thread): > I can’t look back at myGOexpressandTVTB
Federico Marini (11:49:44): > the bRoom, the dishwashR, …
Federico Marini (11:50:09) (in thread): > same here
Federico Marini (11:50:10) (in thread): > :smile:
Federico Marini (11:51:44) (in thread): > on top of that
Federico Marini (11:51:52) (in thread): > damn travis and their limit of 45 minutes
Federico Marini (11:52:04) (in thread): > or well, blame me for the many dependencies
Kevin Rue-Albrecht (11:52:21) (in thread): > And I met a guy from Milano, Ottavio something, who wrote a Shiny app for ChIP-seq. The app has some pretty useful functionalities, and it’s a lot of work for someone alone (like ideal and TVTB), which made me feel even worse when I saw how depressed he was when I showed him iSEE:sweat_smile:
Federico Marini (11:52:29): > seen this already? for all the fans of “i see trends”
Federico Marini (11:52:29): > https://xkcd.com/2048/ - Attachment: Curve-Fitting > [Title text] “Cauchy-Lorentz:”Something alarmingly mathematical is happening, and you should probably pause to Google my name and check what field I originally worked in.””
Federico Marini (11:52:53) (in thread): > sorry but that is just good:stuck_out_tongue:
Federico Marini (11:53:16) (in thread): > shows how better things become when good sinergies are in:wink:
Kevin Rue-Albrecht (11:54:50) (in thread): > Yep. I think he cried when I showed him that the app gives the code of the various plots
Kevin Rue-Albrecht (11:55:08) (in thread): > “This is exactly what I was dreaming to do”
Federico Marini (11:57:23) (in thread): > I also have some embarassment in writing up abt the packages and saying “hey hey reproducibility” while I myself was involved in doing somethign that solves that more elegantly
Federico Marini (11:57:24) (in thread): > but hey
Kevin Rue-Albrecht (13:46:41) (in thread): > they probably forgotheader=FALSE:wink:
Federico Marini (14:17:11): > quick on the abstract: I did not see any new edit, so I’d submit as it is now - thank you all for the feedback!
Aaron Lun (14:17:52): > :party_parrot:
Federico Marini (14:19:45): > Aaron, are you planning to join as well?
Aaron Lun (14:22:13): > What. EuroBioC? No.
Kevin Rue-Albrecht (14:25:12): > If anyone wants to have a look or add anything at the PR, I’ll merge this weekend at the latest. Sooner if ‘anyone’ says anything close to “FINISH HIM”:yum:
Federico Marini (15:02:44): > Guess who also uses tours now:https://cells.ucsc.edu/?ds=10xMegacell
Kevin Rue-Albrecht (15:47:40): > is it supposed to do DE on the fly? because the modal is blank when I click on the cluster number
2018-09-21
Federico Marini (04:17:10): > no clue
Aaron Lun (04:20:37): > has anyone tried running Rtsne on a very large data set?
Kevin Rue-Albrecht (06:20:57): > I don’t remember doing it recently, I might have messed with it last summer when there was the 1M neurons buzz, but can’t even remember what happened.
Federico Marini (07:56:00): > I did - takes ages afaiCanRemember
Aaron Lun (08:23:17): > What was the bottleneck? PCA, nearest neighbors, or something else?
Federico Marini (08:27:58): > I never really went in deep to check
Federico Marini (08:28:10): > there might be this other implementation which is faster
Federico Marini (08:28:20): > but i never tried that
Federico Marini (08:28:31): > could be a good reason to give umap a try?
Federico Marini (08:28:40): > (orPHATE)
Aaron Lun (08:29:47): > Switching to a completely different algorithm = giving up.
Federico Marini (08:34:09): > PHATality!
Federico Marini (08:34:28): > is it for the work of yours on PCs?
Aaron Lun (08:38:56): > huh?
Federico Marini (08:41:03): > principal components
Aaron Lun (08:42:05): > huh?
Aaron Lun (08:42:08): > What work?
Kevin Rue-Albrecht (12:07:08): > You know what could be an awesome public instance of iSEE? > ENCODE RNA-seq in tissues for mouse and human > Something a bit more reactive than those two: > -http://promoter.bx.psu.edu/ENCODE/get_human_expr.php?assembly=hg19&gene=BRCA1-http://chromosome.sdsc.edu/mouse/download.html
Kevin Rue-Albrecht (12:07:52): > Note the “Data can be downloaded and viewed in ENCODE website.” in the second link:wink:
Kevin Rue-Albrecht (12:08:38): > Ah no, forget the last joke, I didn’t wait long enough to let the page load. I thought the link was dead
Kevin Rue-Albrecht (12:09:30): > Still.. it points to the ENCODE Portal and the UCSC browser, basically
Kevin Rue-Albrecht (12:11:29): > Anyway. Point being that iSEE could offer a nice view of tissue-specific expression data
Aaron Lun (12:27:44): > Knock yourself out.
Aaron Lun (12:27:53): > I just finished talking to HR:moneybag:
Aaron Lun (12:28:05): > :money_mouth_face:
Kevin Rue-Albrecht (12:28:38): > Damn I should have taken you up on that beer
Kevin Rue-Albrecht (12:28:40): > :stuck_out_tongue:
Kevin Rue-Albrecht (12:29:32): > more seriously, I wasn’t that great after the last conference night at the Red Lion ^^
Aaron Lun (12:34:45): > Yeah, I’ll bet you weren’t.
Aaron Lun (12:34:52): > :tropical_drink:
Aaron Lun (12:35:10): > Were you belting out ABBA hits at 2am?
Kevin Rue-Albrecht (12:36:30): > hahaha no we were rather good school kids, walking back through the gate around 23:30 under the patient eye of the security guy ^^
Kevin Rue-Albrecht (12:37:35): > Still, lots of people looked pretty tired the next day. (conference program was pretty intense too)
Federico Marini (14:06:30): > PEEPS
Aaron Lun (14:06:38): > YES
Federico Marini (14:06:39): > our first citation:love_letter:
Federico Marini (14:06:41): > https://f1000research.com/articles/7-1522/v1 - Attachment (f1000research.com): F1000Research Article: scClustViz – Single-cell RNAseq cluster assessment and visualization. > Read the latest article version by Brendan T. Innes, Gary D. Bader, at F1000Research.
Aaron Lun (14:07:50): > …meh.
Federico Marini (14:07:56): > could be that one of us gets to review it?
Aaron Lun (14:08:35): > I would hope not.
Aaron Lun (14:08:50): > Because I can’t be honest with open review.
Federico Marini (14:08:55): > :stuck_out_tongue:
Federico Marini (14:12:50) (in thread): > I remember such a moment for me too
Federico Marini (14:12:57) (in thread): > it was at a course on site in 2012
Federico Marini (14:13:35) (in thread): > it was awkwardly funny to see loud happy people being shhhhhushed in a sec at the sight of the security man
Kevin Rue-Albrecht (16:56:14): > Not sure in which categories we’re supposed to fall: “[…] existing GUIs, which are either very user-friendly for non-technical users, at the cost of the ability to customize analysis, or very powerful and customizable, at the cost of providing a simple framework for accomplishing a common analysis task.”
Kevin Rue-Albrecht (16:57:14): > I thought we were user-friendly, very powerfulandcustomisable:cry:
Kevin Rue-Albrecht (17:00:22): > NEWS:iSEE version 1.1.8 pushed upstream
Aaron Lun (17:45:58): > We already have a customizable analysis GUI. It’s called R.
Aaron Lun (17:46:11): > Or, for the more n00bish, Rstudio.
2018-09-22
Kevin Rue-Albrecht (08:10:13): > I was wondering if it was conceptually flawed, of if I was missing the way to ‘nicely’ put a dimensionality reduction result for a subset of cells in the parent object that contains all cells, e.g.reducedDim(sce, "A") <- reducedDim(tmpSCE)The first solution I’ve come up with involves making anNAmatrix of the parent size, and populate the relevant cells, e.g.: > > for (tmpName in reducedDimNames(tmpSCE)) { > tmpSource <- reducedDim(tmpSCE, tmpName) > tmpDest <- matrix(NA, nrow=ncol(sce), ncol=ncol(tmpSource), dimnames=list(colnames(sce), colnames(tmpSource))) > tmpDest[colnames(tmpSCE), ] <- tmpSource > reducedDim(sce, paste0("filtered_", tmpName)) <- tmpDest > } >
Kevin Rue-Albrecht (08:11:32): > The idea being ultimately to browse in a single iSEE session the object that includes all cells (e.g. to demonstrate empty droplets, doublets, etc.), but then also have the reduce dim result of the object post-filtering
Aaron Lun (08:12:32): > Yes, that’s correct, but you’ll need to do this before theiSEEcall, obviously.
Kevin Rue-Albrecht (08:15:34): > Sure. Still, makes me wonder whether it’d be worth having arunPCA-style function that can take a subset of sample identifiers and do those few steps to return the matrix with NAs. Although maybe it’s best to leave a bit of work to the user sometimes
Aaron Lun (09:35:51): > ¯*(ツ)*/¯
Aaron Lun (10:04:13): > 25% of the poster is done.
Kevin Rue-Albrecht (10:43:29): > so… four A4 pages?
Aaron Lun (10:50:53): > Yep, doing the next four now.
Kevin Rue-Albrecht (10:52:06): > :writing_hand:
Aaron Lun (10:52:44): > Oh, can you do me a favour and try seeing whether iSEE works with the TCGA data stored as aHDF5Matrix?
Kevin Rue-Albrecht (10:53:55): > side point: I’ve spotted a typo in our README. Shall I bump up the release ‘again’ just to push it upstream for the next release? or should I leave it on our GH only for the time being? it was really just to fix the appearance of our GH readme page
Kevin Rue-Albrecht (10:54:10): > I think you said you’ve bumped for less than that
Aaron Lun (10:54:24): > just GH would be fine, no need for version bup
Kevin Rue-Albrecht (10:54:28): > ok
Kevin Rue-Albrecht (10:57:57): > Alright, time to brush up on my HDF5 skill and figure out how to do the conversion. I haven’t really bothered with the different storage strategies yet.
Aaron Lun (10:59:39): > It should be a simple function;writeHDF5Array. Make sure to set the chunk dimensions to something like 100 x 100.
Kevin Rue-Albrecht (10:59:55): > ah right
Aaron Lun (11:00:12): > As reading is done in terms of whole chunks, regardless of how much of the chunk you actually use.
Aaron Lun (11:00:20): > So extracting a row will require reading of all chunks overlapping that row.
Kevin Rue-Albrecht (11:00:41): > makes sense
Aaron Lun (11:00:52): > You might think to define a row = a chunk, but that has problems if you want to read in a column, as then you will have to read in every. Single. Row. to get the values for that column.
Kevin Rue-Albrecht (11:02:23): > Well now, that’s just up to the chunking strategy, then. Got it.
Kevin Rue-Albrecht (11:30:49): > looks working to me
Kevin Rue-Albrecht (11:31:50): > then, in terms of speed, i don’t see a difference either
Kevin Rue-Albrecht (11:34:24): > now, I’ve put thecountsdata as HDF5Array, and I’ve left thelogcountsas a regular matrix, the the heat map maybe feels a tad slower to switch to the HDF5 than the matrix. That’s for about 60 genes
Aaron Lun (12:02:00): > What about start-up time?
Aaron Lun (12:02:24): > It should be lightning fast to start up if it doesn’t have to load the full matrix into memory. Obviously this only works if both matrices are HDF5’d.
Kevin Rue-Albrecht (12:11:18): > fair enough:Processing block 3/16 ... OK
Kevin Rue-Albrecht (12:11:33): > it doesn’t take long, I’ll tell you in a minute
Kevin Rue-Albrecht (12:13:29): > hm.. maybe I should do it for the reducedDim too, still feels a bit slow
Aaron Lun (12:13:55): > reduced dim shouldn’t matter, it’s only 7000 x 2 or so.
Kevin Rue-Albrecht (12:14:31): > yeah
Kevin Rue-Albrecht (12:16:32): > maybe there are just not enough zeros in the TCGA to fully feel the difference?
Kevin Rue-Albrecht (12:18:40): > > > sum(assay(sce, "counts") != 0) / prod(dim(assay(sce, "counts"))) > [1] 0.8402137 >
Aaron Lun (12:26:32): > Number of zeroe’s shouldn’t make a difference for file-backed data.
Aaron Lun (12:26:46): > What’s thereadRDStime for the dense arrays vs HDF5?
Kevin Rue-Albrecht (12:36:48): > > > microbenchmark({readRDS("log2CPM.rds")}) > Unit: milliseconds > expr min lq mean median uq max neval > { readRDS("log2CPM.rds") } 12.88134 13.09724 13.30238 13.22193 13.34846 15.16495 100 > > microbenchmark({HDF5Array(h5filename, "log2CPM")}) > Unit: milliseconds > expr min lq mean median uq max neval > { HDF5Array(h5filename, "log2CPM") } 11.71695 13.12981 16.6328 15.22173 19.68844 28.46777 100 > - File (PNG): Pasted image at 2018-09-22, 5:36 PM
Kevin Rue-Albrecht (12:37:27): > (100 iterations)
Aaron Lun (12:49:43): > Huh.
Aaron Lun (12:50:08): > Why does the TCGA data take so long to boot up then? Even before the iSEE load screen?
Kevin Rue-Albrecht (12:51:28): > Well, i’ve only tested the log2CPM matrix here, the RDS is 108KB. > The fullsce.rdsis 1.1GB
Kevin Rue-Albrecht (12:52:18): > oh wait a second, i might have to self slap in a minute
Kevin Rue-Albrecht (12:53:01): > i’ve made the log2CPM.rds fileaftermaking it an HDF5 backed array
Kevin Rue-Albrecht (12:55:17): > yeah…log2CPM.rdsis actually 733MB when I stored the real matrix
Aaron Lun (12:55:55): > :man-facepalming:
Kevin Rue-Albrecht (12:57:54): - File (PNG): Pasted image at 2018-09-22, 5:57 PM
Kevin Rue-Albrecht (12:58:47): > We’ll thank Rafa again for his keynote about “look at your data to spot batch effects… [and all sort of mistakes]”
Aaron Lun (12:59:42): > yeah… that’s what I”m talking about.
Aaron Lun (13:00:09): > Probably worth switching TCGA to use HDF5arrays, then. It should make load-up much faster, cut off 10 seconds.
Aaron Lun (13:00:39): > Would this have the same effect for the cytof app@Charlotte Soneson?
Kevin Rue-Albrecht (13:13:38): > You’ll note that under the new situation, I’ve only done 10 iterations of the RDS-backed loading. I’ll still go ahead and say it’s significant
Aaron Lun (13:14:24): > Yep. I think the TCGA loading currently takes ~30 sec, so it would be good to get that down.
Aaron Lun (13:14:45): > I’m going home now, but see if you can update iSEE2018; I’ll push the changes on the server.
Kevin Rue-Albrecht (13:15:03): > yep, I’m trying to sort it as we speak
Charlotte Soneson (14:42:24): > Just tried quickly for the cytof app, andreadRDSis for sure faster (about 30x it seems), but I get an error when I try to run the app with the hdf5 SCE (works fine with the regular one): > > Warning in .coerce_type(xvals, "X", as_numeric = !group_X) : > coloring covariate has too many unique values, coercing to numeric > Warning: Error in .BLOCK_unique: unused argument (nmax = nmax) > 62: unique.DelayedArray > 60: factor > 59: as.factor > 58: eval [<text>#1] > 57: eval > 56: .evaluate_commands [/Users/charlotte/Dropbox/SingleCellBrowser/SEE/R/commands.R#39] > 55: .extract_plotting_data [/Users/charlotte/Dropbox/SingleCellBrowser/SEE/R/plotting.R#499] > 54: .plot_wrapper [/Users/charlotte/Dropbox/SingleCellBrowser/SEE/R/plotting.R#388] > 53: FUN [/Users/charlotte/Dropbox/SingleCellBrowser/SEE/R/plotting.R#210] > 52: server [/Users/charlotte/Dropbox/SingleCellBrowser/SEE/R/iSEE-main.R#359] > Error in .BLOCK_unique(x, incomparables = incomparables, ...) : > unused argument (nmax = nmax) >
Kevin Rue-Albrecht (14:43:33): > Haven’t seen this error when I updated the TCGA. Did you compare with the changes in my PR ?
Charlotte Soneson (14:45:54): > yep, I basically copied your code:slightly_smiling_face:
Kevin Rue-Albrecht (14:49:38): > Hahaha.
Kevin Rue-Albrecht (14:50:19): > Out of curiosity, could you run my updated code? data and app.R ? could be a good independent validation
Charlotte Soneson (14:55:35): > Trying it out now
Charlotte Soneson (14:59:45): > yes, that seems to work
Kevin Rue-Albrecht (15:09:44): > Ok, at least i’m reassured that it wasn’t something with my interactive session
Kevin Rue-Albrecht (15:14:31): > I don’t see thisBLOCK_uniquething in either rhdf5 nor HDF5Array…
Kevin Rue-Albrecht (15:14:53): > although I’ve done the search using the GH search bar within those packages
Kevin Rue-Albrecht (15:15:03): > maybe better to grep after downloading the source
Kevin Rue-Albrecht (15:15:36): > oh no wait, I’m stupid again, it seems to be in DelayedArray
Charlotte Soneson (15:16:34): > Ok. Thanks, I’m gonna have to continue tomorrow
Kevin Rue-Albrecht (15:16:41): > https://github.com/Bioconductor/DelayedArray/blob/1aac67c0761d06949d4f012cd8a9a9f14dcf0692/R/DelayedArray-utils.R#L471 - Attachment (GitHub): Bioconductor/DelayedArray > Delayed operations on array-like objects. Contribute to Bioconductor/DelayedArray development by creating an account on GitHub.
Kevin Rue-Albrecht (15:16:44): > yeah no worry
Kevin Rue-Albrecht (15:17:29): > wild guess: maybe worth trying with different block sizes
Charlotte Soneson (15:22:01): > Yeah, I wonder if it’s because I have only one block in one dimension (there are only 35 markers)
Kevin Rue-Albrecht (15:24:15): > I’ve got no clue what happens when the only block is larger than the dimension
Kevin Rue-Albrecht (15:24:38): > the other thing is that for some reason, the error message refers tofactor(). I don’t see exactly why factors would be involved for assays
Charlotte Soneson (15:25:03): > I could not set the block size larger than the dimension, so I reduced it to 35. I’ll try to reduce it more and see what happens.
Kevin Rue-Albrecht (15:25:15): > ah right
Aaron Lun (15:43:50): > IMO, I suspect it’s becauseX[,1]for a DelayedArrayXdoesnotyield a vector as expected (see a long discussion athttps://github.com/Bioconductor/DelayedArray/issues/6). This means thatunique(probably to get the factors?) calls a DA-specific method rather than using the function from base R, and the former is probably a bit buggy. Perhaps you could hunt down whereuniqueis being called; look upstream in the sequence of function calls; and make sure the column/row subset from the matrix is beingas.vector’d before being passed down. - Attachment (GitHub): Subsetting with drop=TRUE doesn’t behave as expected · Issue #6 · Bioconductor/DelayedArray > As the title suggests: library(DelayedArray) blah <- matrix(runif(1000), ncol=20, nrow=50) X <- DelayedArray(blah) X[,1,drop=TRUE] ## <50 x 1> DelayedMatrix object of type "double&…
Aaron Lun (15:44:17): > Anyway, is the TCGA loading time speed up noticeable? I’m home so cannot test.
Kevin Rue-Albrecht (15:47:16): > well, on my laptop theapp.Rscript runs thereadRDSstatement clearly faster, but do you mean you’ve deployed it to the Marioni server?
Aaron Lun (15:47:49): > Not yet, so that’s why I’d like to know whether I should feel very happy or just merely okay when I do deploy it.
Aaron Lun (15:48:12): > Should I be looking forward to a 10 second speed-up as our test runs suggest?
Kevin Rue-Albrecht (15:48:37): > well, once the app boots up, the plot still take some time to display, but I guess that’s part ggplot
Kevin Rue-Albrecht (15:49:17): > the 10s speed-up should definitely apply, as this is part of the app.R script
Aaron Lun (15:49:18): > It should affect the interval between script running and app initialization.
Kevin Rue-Albrecht (15:49:25): > exactly
Aaron Lun (15:49:34): > okay. Is the 10 s for the log-counts only, or for the counts + log-counts?
Kevin Rue-Albrecht (15:49:34): > then, the app initialization is another story
Kevin Rue-Albrecht (15:49:39): > both of them
Kevin Rue-Albrecht (15:49:45): > oh no
Kevin Rue-Albrecht (15:49:48): > sorry
Kevin Rue-Albrecht (15:49:51): > just the logcounts
Aaron Lun (15:50:49) (in thread): > Incidentally,@Charlotte Soneson, if this is the problem, please chime in on that issue with the fact that this discrepancy in behaviour has practical consequences in a real development environment.
Aaron Lun (15:51:08): > So maybe it’s even better than 10s.
Kevin Rue-Albrecht (15:53:33): > well right nowapp.Rtakes ~3.8s to be sourced
Kevin Rue-Albrecht (15:53:54): > which includes the call toapp <- iSEE(...)
Kevin Rue-Albrecht (15:54:16): > > > microbenchmark({source("app.R")}, times = 10) > Unit: seconds > expr min lq mean median uq max neval > { source("app.R") } 3.923211 4.088842 4.296698 4.141695 4.526641 5.101452 10 >
Kevin Rue-Albrecht (15:55:10): > Anyone saw this app?http://degust.erc.monash.edu^^ - Attachment (degust.erc.monash.edu): Degust > Degust
Kevin Rue-Albrecht (15:55:23): > you can “play with a demo”
Aaron Lun (15:57:36): > Honestly.
Aaron Lun (15:57:46): > It’s all fun and games until the experiment doesn’t work.
Aaron Lun (15:58:52): > Then you can expect people to come to you whining about “batch effects” and “hidden factors of variation” and so on.
Kevin Rue-Albrecht (15:59:36): > :stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (15:59:46): > (but true)
Kevin Rue-Albrecht (16:00:27): > i’d even say R^2 = “rightful rant”
Aaron Lun (16:04:32): > A point and click interface like that kind of defeats the purpose of reproducible analyses.
2018-09-23
Aaron Lun (07:46:11): > Anyway, is there anything docker-related we have to update for the HDF5 as well?
Aaron Lun (07:50:03): > The TCGA app is uploaded. I have1549191029for the file size and8e35f4227b559d28aa3beaf4410c9953for the MD5 sum of the HDF5 file.
Aaron Lun (07:50:20): > Note that it won’t quite work yet because HDF5Array isn’t installed - I’ll have to nag the admin about that tomorrow.
Kevin Rue-Albrecht (08:13:42): > I don’t know about docker-related changes. I’ll have to try updating the docker containers tomorrow from work though: for some reason it takes me 5 min to upload them at work and infinitely long from home
Kevin Rue-Albrecht (08:59:52): > @Aaron LunI almost re-implemented your CUSTOM_DE table panel because I wanted to look at genes that are “broadly DE” between selected cells and others. Point being that I think this exact custom panel probably deserves much more visibility than ‘hidden away’ in the vignette. I’m still trying to think how to best showcase examples of custom panels in a way that users can get a full sense of what’s possible
Aaron Lun (09:21:13): > I would be open to a “common custom panels” set of functions.
Aaron Lun (11:54:46): > Just finished hand-drawing the QR code pointing to our F1000 paper
Aaron Lun (11:54:55): > Now onto the QR code for the Bioc landing page.
Kevin Rue-Albrecht (11:55:02): > noooooo?
Kevin Rue-Albrecht (11:55:25): > pic or didn’t happen?
Aaron Lun (11:55:34): > Will send one once it’s laminated.
Kevin Rue-Albrecht (11:56:27): > Cool. In the meantime, I can imagine coloring a template of QR code, but making one from scratch…. that’s another level
Aaron Lun (11:56:44): > well, I traced it, I’m not that crazy.
Kevin Rue-Albrecht (11:57:10): > ^^
Charlotte Soneson (12:20:55): > Ok - addingas.vector()to the row/column extraction in.make_featAssayPlot()and.make_sampAssayPlot()seems to make it work with the HDF5 matrices also for the CyTOF app. - File (PNG): Screen Shot 2018-09-23 at 18.19.08.png
Kevin Rue-Albrecht (12:21:15): > :party_parrot::tada:
Aaron Lun (12:25:31): > don’t forget to nag herve
Aaron Lun (12:27:36): > top half of the final QR code is done.
Kevin Rue-Albrecht (12:28:47): - File (Shell): Updating Docker containers :)
Kevin Rue-Albrecht (12:29:19): > Alright. I’m all set for future updates:smile:
Aaron Lun (12:33:52): > My hands hurt.
Aaron Lun (12:41:56): > I have defeated the king block
Kevin Rue-Albrecht (12:54:49): > I just realised that iSEE2018 and iSEE_instances didn’t really have direct links to the original package repository. I’ve opened PR’s for that.
Kevin Rue-Albrecht (12:55:29): > I’m just lazy typing ‘isee’ in the address bar to navigate between the various repos. I like links
Aaron Lun (13:10:48): > 75% done
Kevin Rue-Albrecht (13:24:43): > on a completely unrelated note, and not that it means anything: I just realised that we almost caught up with Seurat in our number of commits: 1,730 and 2,015
Aaron Lun (13:32:46): > lol
Aaron Lun (13:32:49): > Anyway
Aaron Lun (13:33:23): > If you think I printed them, I’ll take it as a compliment. - File (JPEG): 20180923_183115.jpg
Aaron Lun (13:34:39): > I’m going home now, because I’m POOPED.
Kevin Rue-Albrecht (13:35:18): > the miniature could have fooled me
Kevin Rue-Albrecht (13:36:27): > did youmicrobenchmarkyourself?
Aaron Lun (13:36:37): > The first one took 3 hours. The second one took 2.
Kevin Rue-Albrecht (13:37:28): > hm.. not much of a confidence interval there.. we probably need another two for the iSEE2018 and Fede’s iSEE_instances to get more data points
Aaron Lun (13:37:50): > you’re more than welcome.
Kevin Rue-Albrecht (13:38:46): > 2h.. I could fit that in our next lab meeting + genomics forum
Aaron Lun (13:39:30): > probably just as productive
Kevin Rue-Albrecht (13:39:47): > oh, and I forgot the journal club
Kevin Rue-Albrecht (13:40:03): > the latter even provides the paper material
Kevin Rue-Albrecht (14:41:16): > Btw, I’ll create the repo to demonstrate a range of custom panels. That’ll nicely complete and balance the set of repositories that we have around iSEE (Charlotte: iSEE, Aaron: iSEE2018, Fede: iSEE_instances, me: iSEE_custom)
Federico Marini (16:00:18) (in thread): > gotta love the commitment
Federico Marini (16:06:41): > you could actually put more n more links in one qr code
Federico Marini (16:06:57): > don’T want to kill your effort tho
Kevin Rue-Albrecht (16:23:48): > fractal QR code? ^^
Federico Marini (16:27:55): > nono
Federico Marini (16:28:00): > just more links
Federico Marini (16:28:18): > as extended text
Aaron Lun (17:22:25): > can your QR scanners read it from the screen?
2018-09-24
Federico Marini (03:04:36): > checkin now
Federico Marini (03:05:21): > yes commander!
Kevin Rue-Albrecht (05:04:07): > both working
Kevin Rue-Albrecht (05:05:29): > Separate note: I’ve just launched the TCGA app from the f1000 article:holy coder of Rit’s fast!
Kevin Rue-Albrecht (05:06:18): > However, am I the only to see the “Feature assay plot 1” sayingError: An error has occurred. Check your logs or contact the app author for clarification.?
Charlotte Soneson (05:06:29): > I see it too
Aaron Lun (05:10:32): > As I mentioned, I need to getHDF5Arrayinstalled on the shiny server.
Kevin Rue-Albrecht (05:11:33): > Ahh, I get it now. Indeed, all the panels that use assay slots are affected.
Kevin Rue-Albrecht (05:11:39): > Sorry I didn’t connect the dots on this one
Federico Marini (08:15:31): > unrelated to iSEE: how many academic-spam emails do you get per day?
Federico Marini (08:15:48): > it is a plague right now in the last weeks
Charlotte Soneson (08:16:59): > so far today 16
Federico Marini (08:17:08): > jesus
Federico Marini (08:17:19): > and they are not filtered by the spam filter
Charlotte Soneson (08:17:30): > some of them
Federico Marini (08:18:10): > crazy shit. I got them in highly increased quantities in the last month
Federico Marini (08:18:20): > but it seems to be similar for many colleagues
Federico Marini (08:18:36): > so, “no, i can’t dream it is because of iSEE”:smile:
Charlotte Soneson (08:18:48): > Don’t think so…
Aaron Lun (09:36:47): > I would be getting them if it were due to iSEE, as I’m corresponding.
Aaron Lun (09:37:57): > Ah, crap, forgot to install the devel version of iSEE.
Federico Marini (09:41:17): > are you also bombarded anyway?
Kevin Rue-Albrecht (09:41:53): > I don’t really see much spam on my side to be honest
Kevin Rue-Albrecht (09:42:52): > but to be fair, I did just find the “feedback on Genome Informatics 2018” email in my junk folder
Aaron Lun (09:50:52): > I find “john.marioni@cruk.cam.ac.uk” in my Junk folder.
Aaron Lun (09:50:55): > Fancy that.
Kevin Rue-Albrecht (09:51:18): > custom rules don’t count:wink:
Aaron Lun (09:51:38): > That’s the thing. I never set these rules!
Aaron Lun (09:51:46): > So the automatic filter on the CRUK webmail does it for me.
Kevin Rue-Albrecht (09:51:51): > (I know, hence the:wink:)
Aaron Lun (09:52:03): > Especially if the email mentions “Sanger” or “Hinxton”.
Aaron Lun (09:52:04): > Dirty words.
Federico Marini (09:52:36): > not to mention the OOOO
Kevin Rue-Albrecht (09:52:44): > That’d be fun, to initialize staff accounts with a rule that filters emails from their boss
Aaron Lun (12:36:11): > FYI, the error is still there after update asdacomphas not been merged and pushed (duh).
Kevin Rue-Albrecht (12:37:40): > I’m sure not what it is.. but funnily enough, when I google “dacomp r”, iSEE is the 6th result ^^ - File (PNG): Screenshot 2018-09-24 17.37.14.png
Kevin Rue-Albrecht (12:38:27): > ah, it’s a branch (merged, pushed) hints…
Aaron Lun (12:38:38): > When I google “Aaron Lun”, I get myself.
Kevin Rue-Albrecht (12:38:43): > damn that’s some pretty efficient indexing from Google
Charlotte Soneson (12:39:14): > Why did you google “dacomp r”?:slightly_smiling_face:
Kevin Rue-Albrecht (12:39:32): > well.. initially, I just googled “dacomp”
Charlotte Soneson (12:39:43): > Ah sorry, I missed Aaron’s message above
Kevin Rue-Albrecht (12:39:44): > I’m tired ok?:stuck_out_tongue_winking_eye:
Charlotte Soneson (12:40:05): > Wondered why you thought of this word out of the blue…
Kevin Rue-Albrecht (12:40:14): > ahh
Kevin Rue-Albrecht (12:40:51): > still.. the speed at which i google stuff before even taking a second to think about it is not really a good sign ^^
Aaron Lun (12:43:45): > Can people check and merge their PRs?
Aaron Lun (12:44:20): > I’ve scanned overdacomp, I think it’s fine.
Kevin Rue-Albrecht (12:47:46): > our PRs? do you mean our tours? or the PR?
Aaron Lun (13:31:17): > well, there’s been a lot of PRs recently so I lost track.
Kevin Rue-Albrecht (13:32:21): > ah rght
Kevin Rue-Albrecht (13:32:41): > well, mine’s merged. I’m just testing Charlotte’s one
Kevin Rue-Albrecht (13:34:10): > …after stupidly running the cytof data.R script in the tcga folder, and then running the tcga script again… from the cytof_hdf5 branch (which doesn’t have my TCGA HDF5 changes):sob:
Kevin Rue-Albrecht (13:36:26): > ..which crashes withError in .BLOCK_unique(x, incomparables = incomparables, ...) :because I haven’t updated to your branch:sob:
Kevin Rue-Albrecht (13:36:31): > breathe Kevin breathe
Kevin Rue-Albrecht (13:37:27): > I’ll pick that up from home. No point in rushing now
Aaron Lun (14:19:05): > FINISH HIM:-1:
Federico Marini (14:31:49): > dacompis the new covfefe
Kevin Rue-Albrecht (14:32:04): > lol, nice one
Federico Marini (14:32:29): > 20.32 CET is the best time for jokes you know:smile:
Kevin Rue-Albrecht (14:36:49): > @Aaron Lun: about the FINISH HIM part.. That awfully looks like it could use a version bump and push upstream. I’ll do the first one and let everyone chime in with any last-minute wish while Travis checks
Kevin Rue-Albrecht (14:37:27): > (I’ve just tested@Charlotte Soneson’s tour with the updated iSEE, it works for me)
Kevin Rue-Albrecht (14:42:06): > Actually, I’ll probably wait for the Docker container to rebuild, then use that to rebuild the tours and check them, before pushing upstream
Kevin Rue-Albrecht (16:41:22) (in thread): > Chiming in late, but I have1917080937d87eb2fdfcc3d5005c9e53file size1549272958
Kevin Rue-Albrecht (16:41:53) (in thread): > Also, I’ve just compared with Charlotte for the cytof tour, and we have different checksums too
Kevin Rue-Albrecht (16:42:17) (in thread): > I’ll re-run both
Aaron Lun (16:44:04) (in thread): > No, that’s okay. I ran out of memory halfway through so I didn’t know if the HDF5 file was fucked up. I guess you can find out once the app is working.
Kevin Rue-Albrecht (16:44:30) (in thread): > Well both apps work for me, but still.. it’s weird
Aaron Lun (16:45:08) (in thread): > probably depends on the spacing the OS adds between chunks.
Aaron Lun (16:45:26) (in thread): > Even gzipping a file on different file systems (but from the same cluster) gives different results.
Kevin Rue-Albrecht (16:45:27) (in thread): > … seriously? OS-dependent?
Kevin Rue-Albrecht (16:45:33) (in thread): > ouch
Aaron Lun (16:45:48) (in thread): > Yes, in terms of exact byte identity.
Aaron Lun (16:46:17) (in thread): > Of course, it doesn’t matter to the algorihtm that ends up using it, it only matters if you’re trying to compare the “same” file constructed on different systems.
Aaron Lun (16:46:30) (in thread): > And obviously it doesn’t matter for downloads because the file was only ever constructed on one system in that case.
Kevin Rue-Albrecht (16:46:44) (in thread): > Alright. Makes sense actually. md5 mostly relies on a file being generated once and then copied after all
Aaron Lun (16:47:03) (in thread): > Okay. With that settled, I’m going home.
Kevin Rue-Albrecht (16:47:32) (in thread): > damn.. i haven’t stayed that late since my PhD era
2018-09-25
Aaron Lun (06:27:37): > Re your email aboutshinyngs. Seems pretty analysis-y to me.
Aaron Lun (06:27:49): > Somewhat unfocused, if I had to say.
Kevin Rue-Albrecht (06:28:02): > yup, kind of tied to a workflow if I got it right
Kevin Rue-Albrecht (06:28:26): > it’s just the first person would told me they looked into Shiny modules
Kevin Rue-Albrecht (06:28:58): > still, sounds like it’s trading off some of the freedom that we used to dynamically link panels
Aaron Lun (06:39:50): > Nonetheless, some thinking about shiny modules would be helpful.
Federico Marini (10:42:36): > hoi team
Federico Marini (10:42:47): > for the advertisement of the pkg
Federico Marini (10:42:54): > we have a veeeery low hanging fruit
Federico Marini (10:43:05): > i.e. the project website done via pkgdown
Kevin Rue-Albrecht (10:43:42): > Interesting. For a second there I thought you meant a GitHub website
Kevin Rue-Albrecht (10:44:08): > But I’d like to learn more of pkgdown too
Kevin Rue-Albrecht (10:45:35): > Or maybe the two are related?
Federico Marini (10:46:47): > strictly
Kevin Rue-Albrecht (10:47:19): > Nevermind: > > If you are using GitHub, the easiest way to make this your package website is to check into git, then go to settings for your repo and make sure that the GitHub pages source is set to “master branch /docs folder”. > fromhttp://pkgdown.r-lib.org/#usage - Attachment (pkgdown.r-lib.org): Make Static HTML Documentation for a Package > Generate an attractive and useful website from a source package. ‘pkgdown’ converts your documentation, vignettes, ‘README’, and more to ‘HTML’ making it easy to share information about your package online.
Federico Marini (10:47:50): > lemme give a run at that, and I’ll either push the branch or just paste a screenshot
Kevin Rue-Albrecht (10:51:59): > Hm… does itreallyhave to be “master branch” ? I’m not so sure it’s a good idea to include the website in the released package source. I would have hoped that thegh-pagesbranch only contains the website files (https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/) - Attachment (help.github.com): Configuring a publishing source for GitHub Pages - User Documentation > You can configure GitHub Pages to publish your site’s source files from master, gh-pages, or a /docs folder on your master branch for Project Pages and other Pages sites that meet certain criteria. I…
Kevin Rue-Albrecht (10:52:33): > You’ll have to ask@Charlotte Sonesonto tweak the repository settings though
Federico Marini (10:52:46): > ok then screenshot will be
Federico Marini (10:52:50): > coming in 1 min
Kevin Rue-Albrecht (10:53:10): > no worries - i’ll dive back in my stuff ^^
Kevin Rue-Albrecht (10:53:26): > :swimmer:
Federico Marini (10:53:59): > whoops
Kevin Rue-Albrecht (10:54:21): > ohh… I was curious about that now:wink:
Federico Marini (10:54:26): > ahaha
Federico Marini (10:54:42): > this one’s better
Federico Marini (10:54:43): > and
Federico Marini (10:54:45): - File (PNG): Screen Shot 2018-09-25 at 16.53.25.png
Federico Marini (10:54:47): > one line away
Federico Marini (10:55:36): - File (PNG): Screen Shot 2018-09-25 at 16.55.07.png
Federico Marini (10:55:44): - File (PNG): Screen Shot 2018-09-25 at 16.54.50.png
Federico Marini (10:56:00): > with slight config editing (tried out for my other pkg) we have a smooth theme, a nice font, and so on
Federico Marini (10:56:25): > say, so.
Federico Marini (10:56:32): - File (PNG): Screen Shot 2018-09-25 at 16.56.07.png
Kevin Rue-Albrecht (10:56:40): > Not bad. Don’t feel shy to put yourself as maintainer though: it’s your email currently in the DESCRIPTION file
Federico Marini (10:58:00): > did not edit anything, all info were picked automaticall
Federico Marini (10:58:00): > y
Federico Marini (10:58:48): > I guess being charlotte the “cre”
Kevin Rue-Albrecht (10:58:50): > Ah. I assume some of it is parsed from the source, while other depends on the repo ownership
Kevin Rue-Albrecht (10:58:57): > ah
Kevin Rue-Albrecht (10:58:59): > my bad
Kevin Rue-Albrecht (10:59:44): > it’s two different things then, the “cre” tag, and the email declared as maintainer (I think the latter is what Bioconductor considers)
Federico Marini (10:59:48): > I’ll post soon the one forpcaExplorerso you can also explore it and have more a feeling than just a static screenshot
Federico Marini (10:59:53): > yop
Kevin Rue-Albrecht (11:00:24): > Cool though the idea of a website. Especially if automatically generated without much extra effort
Federico Marini (11:01:01): > As I said, one line
Federico Marini (11:01:06): > pkgdown::build_site()
Kevin Rue-Albrecht (11:01:10): > lol
Federico Marini (11:01:30): > plus some tweaking in _pkgdown.yml, but you can all benefit of my first steps:smile:
Kevin Rue-Albrecht (11:02:33): > it does remind me that we’ve kinda split the information across multiple repositories, and that we should be careful about not confusing users where to find information
Federico Marini (11:18:10): > (if we switch to news.md, the changelog is also gratis included)
Kevin Rue-Albrecht (11:19:58): > Eh.. I think that’s alright. Can’t remember why we pickedinst/NEWS.Rd. Probably my fault:wink:
Federico Marini (11:20:32): > noone’s, I guess. We sticked to the “old” rules
Federico Marini (11:20:43): > now it seems news in markdown is quite common
Kevin Rue-Albrecht (11:20:52): > devtools::use_news_mdputsNEWS.mdat the root of the package nowadays
Federico Marini (11:20:55): > plus, accepted without even a note in CRA_N
Kevin Rue-Albrecht (11:21:45): > Feel free to reformat and move the news:innocent:
Federico Marini (11:22:15): > I use my repo as sandbox so I have limited damages:smile:
Federico Marini (11:31:27): > live here:http://federicomarini.github.io/pcaExplorer/ - Attachment (federicomarini.github.io): Interactive Visualization of RNA-seq Data Using a Principal Components Approach > This package provides functionality for interactive visualization of RNA-seq datasets based on Principal Components Analysis. The methods provided allow for quick information extraction and effective data exploration. A Shiny application encapsulates the whole analysis.
Kevin Rue-Albrecht (11:39:13): > booooo! 34% coverage:stuck_out_tongue_winking_eye:
Federico Marini (11:39:56): > gotta do the walk of atonement here in the club of 99%:disappointed:
Kevin Rue-Albrecht (11:42:49): > or maybe i just need a rehab session/intervention: “why it’s okay to not have 100%”
Federico Marini (11:43:19): > talking about rehab:stuck_out_tongue:
Federico Marini (11:43:32): > and the naming of such events
Federico Marini (11:44:07): > today at the uni there was a meeting of the personnel, and one topic was “sexual harassment at the workplace”
Federico Marini (11:44:23): > withoutthe subtitle “how to avoid it/detect it”
Federico Marini (11:44:58): > someone you know was tempted to add to the poster “a hands-on session” as subtitle:stuck_out_tongue:
Kevin Rue-Albrecht (11:44:59): > Ah. Still sounds pretty serious. Any issue recently that would have prompted the meeting, or “just” a reminder?
Kevin Rue-Albrecht (11:45:07): > looooool
Federico Marini (11:45:59): > no clue
Kevin Rue-Albrecht (11:46:48): > with the “Nicolas Le Novère” story at the EBI, i wouldn’t be surprised other institutions are treading carefully
Federico Marini (11:49:20): > probably. still, sometimes I think Mainz is sadly further away than any PAMPA
Federico Marini (11:49:24): > no offense for pampas
Federico Marini (11:49:34): > zürich does the centre for repro science
Federico Marini (11:49:39): > munich the open science center
Federico Marini (11:49:49): > mainz does cut funding to my institute
Kevin Rue-Albrecht (12:23:53): > One of those days, when I’m thinking “Damn I’d love to brush those points up there” - File (PNG): Pasted image at 2018-09-25, 5:23 PM
Kevin Rue-Albrecht (13:53:53): > Not sure if we already pointed out that we’re also listed here:https://www.scrna-tools.org/tools.html#iSEE - Attachment (scrna-tools.org): scRNA-tools > A catalogue of single-cell RNA-sequencing analysis tools
Kevin Rue-Albrecht (13:55:40): > I found in the FAQs: “scRNA-tools.orgis maintained by the Oshlack lab”
Federico Marini (14:59:13): > yep, that’s why I told you to shake hands with Luke:smile:
Federico Marini (14:59:59): > he was an early adopter as far as I can remember
Kevin Rue-Albrecht (15:45:30): > yeah - the second person to star iSEE
Charlotte Soneson (15:49:44): > And he’s in Zurich right now:slightly_smiling_face:unfortunately I’m off to Belgium…
Kevin Rue-Albrecht (15:52:36): > Nice guy to meet. > Re: iSEE though, he clearly knows it and keeps an eye on it, but it didn’t feel like he was using it a lot I have to say
Kevin Rue-Albrecht (15:52:49): > Probably busy enough with other things, as we all are
Federico Marini (15:53:17): > @Charlotte SonesonI might ask you soon to try to set up the pkgdown based project website
Federico Marini (15:53:31): > since you’re the owner I guess you are the only one who can do
Federico Marini (15:53:56): > I’m working on the branchprojsite
Federico Marini (15:53:59): > soon to be up
Charlotte Soneson (15:54:41): > Ok, let me know what I have to do. I will be here sporadically the coming two days (conference), but I can follow simple instructions:wink:
Federico Marini (15:54:42): > (you can add also your websites/link to your institution homepages, plus your orcid id as comment in the DESCRIPTION)
Federico Marini (15:55:07): > once we merge it in, it is one action
Federico Marini (15:55:16): > I’ll let you know
Kevin Rue-Albrecht (16:07:09): > I’m doing such a spring cleaning in my Docker images
Federico Marini (16:07:18): > projsiteup for you to have a look
Kevin Rue-Albrecht (16:07:31): > I had a few dozens of them dating several months
Federico Marini (16:07:39): > all the magic is in the_pkgdown.yml
Federico Marini (16:08:29): > to explore it offline, open index.html in /docs
Kevin Rue-Albrecht (16:18:13): > Neat. Works for me
Kevin Rue-Albrecht (16:18:43): > Something weird though: > Go to “article” > “an introduction …“, and scroll down
Kevin Rue-Albrecht (16:19:15): > At some point the section “Sidebar” is sort of stuck at the top of screen while you keep scrolling down
Kevin Rue-Albrecht (16:19:26): > Does that happen to any of you?
Kevin Rue-Albrecht (16:19:58): > Looks like this - File (PNG): Screenshot 2018-09-25 21.19.42.png
Federico Marini (16:20:24): > to me as well
Federico Marini (16:20:28): > in safari
Kevin Rue-Albrecht (16:20:36): > specifically that section
Kevin Rue-Albrecht (16:20:50): > and if you scroll down enough, at some point it is ‘released’
Kevin Rue-Albrecht (16:21:04): > for me it’s when I get to “Heatmap”
Federico Marini (16:21:19): > yep
Federico Marini (16:21:22): > same here
Kevin Rue-Albrecht (16:21:33): > oh wait, when I arrive completely at the bottom, there is the “Sidebar” section again
Federico Marini (16:21:36): > and also happening in chrome
Kevin Rue-Albrecht (16:21:40): > some anchoring issue I guess?
Federico Marini (16:21:47): > weird, tho
Kevin Rue-Albrecht (16:22:26): > Still.. pretty amazing the result of that pkgdown
Federico Marini (16:24:49): > I’d say one of the lowest hanging fruit I have ever tried
Kevin Rue-Albrecht (16:45:45): > argh. I finally cleared all the old Docker containers and rebuilt the new ones
Kevin Rue-Albrecht (16:46:21): > made me realise that the containers that rely on HDF5 need to explicitly install the package at runtime
Kevin Rue-Albrecht (16:46:46): > otherwise the panels show:Error: unable to find required package 'HDF5Array'
Kevin Rue-Albrecht (16:48:59): > maybe it’s actually better if we install it straight in the container ofiSEEitself
Kevin Rue-Albrecht (16:49:13): > otherwise it needs to be installed at runtime for every data set
Aaron Lun (16:52:40): > Exactly how much crap (in terms of files) gets dragged in with this site?
Kevin Rue-Albrecht (16:53:29): > As I said, I’m hoping that we can keep that on a side branch. The docs suggest thatgh-pagesshould be ok
Kevin Rue-Albrecht (16:53:39): > Still, a good question for@Federico Marini
Aaron Lun (16:53:43): > Okay, good.
Kevin Rue-Albrecht (17:02:41): > Just as reminder: I haven’t pushed 1.1.9 upstream yet, despite the version bump. > I’ve just pushed it to GH, so that the Docker container automatically built. > I’ve then rebuilt the tours using the updated Docker container (1.1.9) > Having just tested the TCGA tour, I realise that the app crashes because the Docker container does not have theHDF5Arraypackage installed. > So I’m now opening another PR to install this package directly in the Docker container of the iSEE package. This way child containers don’t need to install it at runtime. > The iSEE docker container only build from themasterbranch, so I am going to merge again for another cycle of testing. > Once the containerized tours work again, I’ll push 1.1.9 upstream.
Federico Marini (17:55:30) (in thread): > everything is in docs/
Federico Marini (17:55:36) (in thread): > plus the yaml
Federico Marini (17:55:43) (in thread): > i included both in rbuildignore
2018-09-26
Kevin Rue-Albrecht (05:08:23): > damn.. the HDF5Array installation worked, but now the container crashes because theDelayedMatrixassays point to the HDF5 file pathrelative to my machine: > > > str(assay(sce)) > Formal class 'DelayedMatrix' [package "DelayedArray"] with 1 slot > ..@ seed:Formal class 'DelayedDimnames' [package "DelayedArray"] with 2 slots > .. .. ..@ dimnames:List of 2 > .. .. .. ..$ : chr [1:23368] "1/2-SBSRNA4" "A1BG" "A1BG-AS1" "A1CF" ... > .. .. .. ..$ : chr [1:8411] "TCGA-02-0047-01A-01R-1849-01" "TCGA-02-0055-01A-01R-1849-01" "TCGA-02-2483-01A-01R-1849-01" "TCGA-02-2485-01A-01R-1849-01" ... > .. .. ..@ seed :Formal class 'HDF5ArraySeed' [package "HDF5Array"] with 5 slots > .. .. .. .. ..@ filepath : chr "/Users/kevin/git/iSEE2018/tours/tcga/sce.hdf5" > .. .. .. .. ..@ name : chr "counts" > .. .. .. .. ..@ dim : int [1:2] 23368 8411 > .. .. .. .. ..@ first_val: int 30 > .. .. .. .. ..@ chunkdim : int [1:2] 100 100 >
Kevin Rue-Albrecht (05:08:43): > see@seed@filepath
Kevin Rue-Albrecht (05:10:34): > I probably need to build Docker containers that store thesce.hdf5files in a standard location, and make the relevant tours depend on those “sce.hdf5 containers”
Aaron Lun (05:15:25): > You just need to reset the paths inapp.R.
Kevin Rue-Albrecht (05:15:47): > oh
Aaron Lun (05:16:09): > So when you load the RDS, just reset the path. I think withpath.
Kevin Rue-Albrecht (05:17:11): > Yep,path(assay(sce))works as an accessor at least. I’ll use the setter set
Kevin Rue-Albrecht (05:25:04): > bingo
Kevin Rue-Albrecht (05:25:24): > just gotta do the same for cytof now
Kevin Rue-Albrecht (05:33:42): > argh.. i don’t get it, now the TCGA works, but for some reason the cytof fails with the error that you had previously solved: > > Error in .BLOCK_unique(x, incomparables = incomparables, ...) : > unused argument (nmax = nmax) >
Aaron Lun (05:38:33): > probably a missingas.vectorfloating around somewhere.
Kevin Rue-Albrecht (05:41:46): > More context: > > Warning in .coerce_type(xvals, "X", as_numeric = !group_X) : > coloring covariate has too many unique values, coercing to numeric > Warning: Error in .BLOCK_unique: unused argument (nmax = nmax) > [...] >
Kevin Rue-Albrecht (05:43:32): > Found inplotting.Rlines 475 and 479 (the latter for Y)
Kevin Rue-Albrecht (05:43:52): > ah and 503 for “Color”
Kevin Rue-Albrecht (05:53:54): > it seems to work withas.vector, but I just realised thatas.vectorturns factors into characters
Kevin Rue-Albrecht (05:54:04): > is that safe?
Charlotte Soneson (05:59:08): > I guess this is not a problem with the assays, which I suppose are typically numeric - but maybe the rowData and colData will also be Delayed* at some point?
Charlotte Soneson (06:00:46): > Btw, I thought I fixed exactly those lines-and you merged the dacomp thing, didn’t you? They don’t have eduroam in this conference place so I can’t check on my laptop right now.
Kevin Rue-Albrecht (06:22:41): > I’ve mergeddacompin PR 238. I see 4 edits, all of the styleplot.data$X <- as.vector(assay(...
Kevin Rue-Albrecht (06:23:38): > Now the error appears to come from one of 3more_data_cmds <- .coerce_type((lines listed above)
Charlotte Soneson (06:24:36): > Yes, that’s what it told me as well before I think.
Charlotte Soneson (06:28:10): > This is the error I got before: - File (JPEG): Image from iOS
Kevin Rue-Albrecht (06:33:21): > This is the full error that I get. I don’t get the line numbers like you do, but it doesn’t look like it fails exactly in the same place > > Warning in .coerce_type(xvals, "X", as_numeric = !group_X) : > coloring covariate has too many unique values, coercing to numeric > Warning: Error in .BLOCK_unique: unused argument (nmax = nmax) > 57: unique.DelayedArray > 55: factor > 54: as.factor > 53: eval > 52: eval > 51: .evaluate_commands > 50: .extract_plotting_data > 49: .plot_wrapper > 48: FUN > 47: server > Error in .BLOCK_unique(x, incomparables = incomparables, ...) : > unused argument (nmax = nmax) >
Kevin Rue-Albrecht (06:34:19): > or maybe it does. hang on, i confused myself, thecoerce_typeis only a warning that we’re used to
Kevin Rue-Albrecht (06:36:28): > Uh. In fact, it looks like it’s only an issue in the Docker container: I’ve just rebuiltmasterof iSEE and ran data.R and app.R in RStudio, it works fine
Kevin Rue-Albrecht (06:36:43): > it’s only when packaged in the Docker container that it seems to fail
Kevin Rue-Albrecht (06:36:53): > I’ll look into it
Kevin Rue-Albrecht (07:42:50): > I honestly don’t get it: I’ve cleared the containers, rebuilt them, and TCGA doesn’t have a problem but not cytof does (same error as I just posted above)
Kevin Rue-Albrecht (07:44:36): > Problem is that the error message doesn’t appear in a panel as usual, here it’s looks like this - File (PNG): Screenshot 2018-09-26 12.44.16.png
Kevin Rue-Albrecht (07:44:43): > The error message is only in the terminal where I run the container from
Kevin Rue-Albrecht (07:45:12): > Anyway, I’m going to go for lab meeting now. I’ll have to leave this for later
Charlotte Soneson (08:32:05): > Just heard a talk abouthttps://orange.biolab.si. Shares some of our ideas - Attachment (orange.biolab.si): Orange – Data Mining Fruitful & Fun > Orange Data Mining Toolbox
Charlotte Soneson (08:34:27): > Single cell toolbox is not released yet, but under development
Aaron Lun (08:34:40): > hm.
Aaron Lun (08:34:47): > Looks pretty analysis-y to me.
Charlotte Soneson (08:37:22): > Oh yes, it’s doing all analysis inside.
2018-09-27
Aaron Lun (10:29:34): > @Kevin Rue-Albrechtisapp.Rthe only thing that changed w.r.t. shiny server functionality?
Aaron Lun (10:29:56): > yes, as it turns out.
Kevin Rue-Albrecht (10:30:11): > yep
Kevin Rue-Albrecht (10:31:09): > the only thing that changed in data.r is the extra hdf5
Kevin Rue-Albrecht (10:31:35): > btw, candy to whoever figures why the cytof tour failsspecificallyin the Docker container
Aaron Lun (10:31:53): > wait, I thought you changed thepath.
Aaron Lun (10:31:58): > Was that only in the docker file?
Aaron Lun (10:33:12): > It’s likely that allapp.Rwill require a path reset.
Kevin Rue-Albrecht (10:34:03): > why all of them? the RDS file didn’t move
Kevin Rue-Albrecht (10:34:17): > it’s only the tours that use hdf5 that require a path reset no?
Aaron Lun (10:34:29): > Yes, I mean “all” as in “all for TCGA”.
Kevin Rue-Albrecht (10:34:40): > ah
Aaron Lun (10:34:41): > but now I realize you haven’t merged it in yet.
Aaron Lun (10:34:48): > which is why I wasn’t seeing it.
Kevin Rue-Albrecht (10:35:23): > Ah right. Yeah, after my path mistake from yesterday, I thought I’d leave the PR open for your to check (and me with fresh eyes)
Aaron Lun (10:36:24): > I wonder if it would be better to reset the path indata.R.
Aaron Lun (10:36:41): > to a relative path
Kevin Rue-Albrecht (10:36:53): > Also, I’ve got a PR open on iSEE itself. A small docfix and i’ve disabled the unit tests that require an internet connection (annotation.R), because they caused crashes recently on Travis
Kevin Rue-Albrecht (10:37:04): > oh
Kevin Rue-Albrecht (10:37:17): > i didn’t bother trying a relative path
Aaron Lun (10:37:23): > Might be risky.
Aaron Lun (10:37:34): > Though I guess it shouldn’t matter if we’re assuming locality of the run script.
Kevin Rue-Albrecht (10:37:53): > yes, they’re just tours anyhow
Kevin Rue-Albrecht (10:37:58): > so long as they work
Kevin Rue-Albrecht (10:38:19): > then, when users do their own stuff, it’s their job to decide which way to go
Aaron Lun (10:52:12): > You could afford to use more newlines.
Kevin Rue-Albrecht (10:53:30): > uh.. where? I’m usually the one who uses them excessively (think lines of code broken after every parenthesis, or our “a phrase per line rule” that took me time to implement)
Aaron Lun (10:53:50): > In the spaces between code blocks, see my latest commit tohdf5paths.
Aaron Lun (10:54:07): > Pretty hard to readapp.Rin particular when everything was compacted together.
Aaron Lun (10:55:07): > Also, one possible cause for the CyTOF failure was that the chunks spanned all rows.
Aaron Lun (10:55:29): > Any attempt to obtain one row would require loading of the chunks overlapping that row.
Aaron Lun (10:55:43): > This would be effectively equivalent to loading of the entire data set into memory.
Aaron Lun (10:55:51): > Not sure whether that causes memory issues.
Kevin Rue-Albrecht (10:55:51): > thanks for the code update.
Kevin Rue-Albrecht (10:56:55): > Ok, i can try smaller blocks and see if that helps. It has to be some technical thing like that, because the TCGA tour works just fine with hdf5 on Docker
Aaron Lun (10:57:37): > tryc(1, 1000).
Aaron Lun (11:12:39): > Any luck?
Kevin Rue-Albrecht (11:12:56): > same error:face_with_raised_eyebrow:
Aaron Lun (11:13:07): > hm.
Aaron Lun (11:13:23): > What do you have to do to generate this error?
Aaron Lun (11:13:35): > cmdline
Kevin Rue-Albrecht (11:13:48): > docker run -i --rm -p 1234:1234 kevinrue/isee_cytof:latest
Aaron Lun (11:14:25): > where do I do this?
Aaron Lun (11:14:29): > anywhere?
Kevin Rue-Albrecht (11:14:35): > and again, I precise that it works just fine if I runapp.Rfrom an R console
Kevin Rue-Albrecht (11:15:07): > yes, anywhere, but let me clarify
Kevin Rue-Albrecht (11:15:35): > just like this, it will download it from my Docker Hub (https://hub.docker.com)
Aaron Lun (11:16:36): > Can’t even get to the download step: > > docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post[http://%2Fvar%2Frun%2Fdocker.sock/v1.27/containers/create](http://%2Fvar%2Frun%2Fdocker.sock/v1.27/containers/create): dial unix /var/run/docker.sock: connect: permission denied. >
Kevin Rue-Albrecht (11:17:10): > if you want to try and debug the situation, you would have to build the image yourself: > > docker pull csoneson/isee:master > docker build -t <your_username>/isee_cytof /Users/kevin/git/iSEE2018/tours/cytof > > and thendocker run -i --rm -p 1234:1234 <your_username>/isee_cytof:latest
Aaron Lun (11:17:30): > ah wait, it’s that sudo nonsense.
Aaron Lun (11:19:44): > Okay, it’s downloaded, but I’m not even getting a browser open.
Aaron Lun (11:20:05): > Won’t let me cntrl c out, either.
Aaron Lun (11:21:20): > Well, this is just fucked up.
Kevin Rue-Albrecht (11:22:06): > wait wait
Kevin Rue-Albrecht (11:22:10): > docker ps
Kevin Rue-Albrecht (11:22:16): > (in a different terminal)
Kevin Rue-Albrecht (11:22:24): > thendocker stop <ID>
Kevin Rue-Albrecht (11:22:35): > you’ll have to wait a few seconds
Kevin Rue-Albrecht (11:22:55): > it will effectively Ctrl-C your other terminal
Aaron Lun (11:23:15): > The sudo thing is a show stopper for me.
Kevin Rue-Albrecht (11:23:28) (in thread): > did you not get a messageListening onhttp://0.0.0.0:1234in the first terminal?
Aaron Lun (11:23:53): > I mean, I have sudo access, but I can’t imagine this is easier to deploy in general.
Kevin Rue-Albrecht (11:23:54) (in thread): > it doesn’t open the browser by itself, but all I have to do is navigate to that page in my browser
Aaron Lun (11:24:27) (in thread): > That was… not obvious.
Kevin Rue-Albrecht (11:24:49) (in thread): > Sorry:sweat_smile:
Aaron Lun (11:25:06): > Okay, I’m getting the.block_UNIQUEerror.
Kevin Rue-Albrecht (11:25:18) (in thread): > I suffered through a few Jupyter notebooks for a few weeks, so I got used to that sort of things
Kevin Rue-Albrecht (11:26:03): > Yeah - nice things about the Docker containers: they make everything perfectly reproducible, including the errors
Aaron Lun (11:26:21): > Well, this seems pretty obvious; is Docker using iSEE 1.1.9?
Kevin Rue-Albrecht (11:26:22): > don’t help much in the way of debugging though
Kevin Rue-Albrecht (11:27:04): > it should - it uses the last version pushed tomaster
Kevin Rue-Albrecht (11:27:33): > (which is not synonym to the one on bioc-devel, to be clear)
Aaron Lun (11:27:45): > Seems like something to double-check.
Kevin Rue-Albrecht (11:28:31): > if you really have time on your hands, your can compare with the TCGA tour by simply editing thedocker runcommand toisee_tcga
Kevin Rue-Albrecht (11:28:47): > it’s just that the container is ~1.5G
Aaron Lun (11:28:50): > The thing is that CyToF triggers a warning about too-many-factors.
Aaron Lun (11:29:13): > And TCGA doesn’t, which is likely the cause of the difference between the two.
Kevin Rue-Albrecht (11:29:58): > ah. maybe it’s the coercion that creates an issue somehow then
Aaron Lun (11:30:32): > Indeed,plotting.Rhas a few lines that callassay()but withoutas.vector()protection.
Aaron Lun (11:30:51): > in.define_colorby_for_column_plot.
Aaron Lun (11:31:01): > and.define_colorby_for_row_plot.
Aaron Lun (11:31:42): > Probably time to rant about this onhttps://github.com/Bioconductor/DelayedArray/issues/6. - Attachment (GitHub): Subsetting with drop=TRUE doesn’t behave as expected · Issue #6 · Bioconductor/DelayedArray > As the title suggests: library(DelayedArray) blah <- matrix(runif(1000), ncol=20, nrow=50) X <- DelayedArray(blah) X[,1,drop=TRUE] ## <50 x 1> DelayedMatrix object of type "double&…
Kevin Rue-Albrecht (11:32:18): > hm.. I’m just thinking that an indirect way of testing this would be to comment out the cytof app preconfiguration inapp.Rand see if avoiding the suspicious lines makes the app run
Kevin Rue-Albrecht (11:36:15): > … it’s not crashing… but it’s not showing any plot either XD - File (PNG): Screenshot 2018-09-27 16.35.53.png
Aaron Lun (11:36:44): > Well, I’m having trouble getting the TCGA example to run on the shiny server.
Aaron Lun (11:36:54): > I think it’s failing to find the right paths.
Kevin Rue-Albrecht (11:37:14): > maybe relative paths are better then
Aaron Lun (11:37:24): > Wait, hold on.
Aaron Lun (11:37:49): > It seems that no matter what you do indata.R,path<-WILL convert it into an absolute path for you.
Kevin Rue-Albrecht (11:38:16): > ah I wasn’t completely nuts then
Kevin Rue-Albrecht (11:38:59): > still, it work in the TCGA container when I give the relative path, so I suppose the setter just callsnormalizePath()on whatever is given
Aaron Lun (11:39:28): > well, you were overwriting it with an absolute path,file.path(getwd(), "sce.h5").
Kevin Rue-Albrecht (11:42:37): > oops yeah
Kevin Rue-Albrecht (11:43:08): > news on the cytof side
Kevin Rue-Albrecht (11:43:47): > it works (i.e. displays panels) if i comment out the custom configuration inapp.R
Kevin Rue-Albrecht (11:44:11): > so yeah, it must be one or more of those unprotectedassaycalls
Aaron Lun (11:56:03): > Finally, the TCGA example works.
Aaron Lun (11:56:05): > God.
Aaron Lun (11:58:16): > Anyway, can you verify that the missingas.vectors are the problem? I will complain on the DA GH issues.
Kevin Rue-Albrecht (11:59:33): > Sure, but it’ll have to be later this evening, I’m trying to finish some qc plots to make sure a couple of genes arenotcorrelated
Kevin Rue-Albrecht (12:00:02): > it’s fantastic:wink:
Aaron Lun (12:28:48): > BTW the TCGA heatmap crashes if there’s no samples (i.e., empty selection).
Aaron Lun (12:29:06): > That should probably not happen.
Kevin Rue-Albrecht (12:29:55): > eh.. do you mean that’s a heat map issue more than a TCGA-specific issue?
Aaron Lun (12:30:34): > Probably.
Aaron Lun (14:35:05): > See updates tohttps://github.com/Bioconductor/DelayedArray/issues/6. - Attachment (GitHub): Subsetting with drop=TRUE doesn’t behave as expected · Issue #6 · Bioconductor/DelayedArray > As the title suggests: library(DelayedArray) blah <- matrix(runif(1000), ncol=20, nrow=50) X <- DelayedArray(blah) X[,1,drop=TRUE] ## <50 x 1> DelayedMatrix object of type "double&…
Aaron Lun (14:35:27): > Try again with updated DA from GH. If it works, please stripas.vectors away.
Kevin Rue-Albrecht (14:43:30): > Ok. I’ll have to put the laptop away for the time of dinner though if I want to avoid trouble:wink:Just thinking out loud in the meantime: I’ll have to add thedevtools::install_github("Bioconductor/DelayedArray")in theapp.Rsourced at runtime by the container, for the purpose of testing. I’m not sure how long it’ll take for the newDelayedArraydevel version to propagate to the Bioconductor devel container
Kevin Rue-Albrecht (14:44:27): > If it works, perhaps the compromise in the meantime would be to add theinstall_githubstatement in the iSEE container. This will avoid the need to install it at runtime in the individual tours
Federico Marini (16:29:18): > small tip, no need of “full devtools”
Federico Marini (16:29:27): > you can use the same function inremotes
Kevin Rue-Albrecht (16:31:38): > oh right, i didn’t haveremotes. Apparently i never installed it before
Kevin Rue-Albrecht (16:34:30): > actually, turns out that devtools is already installed in the container while remotes isn’t
Kevin Rue-Albrecht (16:34:50): > any drawback of “full devtools”?
Federico Marini (16:35:23): > oh, none if already there
Kevin Rue-Albrecht (16:35:39): > omg omg omg:tada::tada::tada::tada::tada::tada:
Federico Marini (16:35:43): > remotes is just the focused on installing remote repos
Federico Marini (16:35:57): > what what what?
Kevin Rue-Albrecht (16:36:31): > the newDelayedArray(0.7.44) makes the cytof tour work great again!
Federico Marini (16:36:43): > eheheh
Kevin Rue-Albrecht (16:36:56): > i would even add covfefe!
Federico Marini (16:37:22): > the newDelayedArraymakes the cytof tour laughwithus
Kevin Rue-Albrecht (16:38:04): > alright, so battle plan:
Kevin Rue-Albrecht (16:42:11): > 1) for the time being, I’ll adddevtools::install_github("Bioconductor/DelayedArray")to the iSEE container (throughinst/install_iSEE.R). > 2) whenDelayedArraybecomes available on Bioc-devel, it should get installed by the existingBiocManager::install("iSEE", version = "devel")as a dependency. At that point we can remove the extradevtoolscall (even though it won’t hurt more than ~30s build time for the iSEE Docker container, and has no effect otherwise)
Kevin Rue-Albrecht (16:42:55): > ah and 0) I’ll remove theas.vectorthat we’ve recently injected in the iSEE source
Kevin Rue-Albrecht (16:55:40): > Actually@Aaron Lun, I just realised that theinstall_iSEE.Rscript that is called to build the Docker image each time we merge to master runs the commandBiocManager::install("iSEE", version="devel"), which means that it wasn’t running 1.1.9 - Attachment: Attachment > Well, this seems pretty obvious; is Docker using iSEE 1.1.9?
Kevin Rue-Albrecht (16:57:46): > i.e. each time we merge to master, it still install the iSEE currently available from bioc-devel, which is at best 2-3 days old
Kevin Rue-Albrecht (17:04:17): > @Aaron Lun: I’ve pushed a new commit ondocfix, but I need to merge tomasterto propagate to thecsoneson/iseeDocker container. Do you want to have a look over the PR to spot any potential issue?
Kevin Rue-Albrecht (17:07:38): > And as a side note, we should also edit DESCRIPTION toDelayedArray (>= 0.7.44)as soon as it’s available on bioc-devel (not before to avoid build errors on bioc-devel I guess)
Aaron Lun (17:09:28): > looks fine to me.
Kevin Rue-Albrecht (17:09:44): > Perhaps it’s just better to wait a couple of days that DelayedArray (>= 0.7.44) becomes available, we’re not particularly pressed for time, and it could avoid a mess with the devtools/BiocManager switch, or?
Aaron Lun (17:10:47): > ¯*(ツ)*/¯
Aaron Lun (17:10:56): > That’s probably true to some extent.
Kevin Rue-Albrecht (17:11:21): > hm.. actually, on second thought, thereissomething that I’d like to test, thedevtools::install(build_vignettes=TRUE)instead ofBiocManager::install("iSEE", version = "devel")
Kevin Rue-Albrecht (17:11:41): > because the former (current version) is just wrong
Kevin Rue-Albrecht (17:12:17): > the Docker image is triggered by the merge tomaster, thus should be based on the iSEE source code ofmaster, not by whatever is available from bioc-devel that day
Kevin Rue-Albrecht (17:13:29): > let’s merge and see what DockerHub thinks of it - worse case i fix it this weekend
Kevin Rue-Albrecht (17:16:56): > uh-oh.. looks like removing theas.vectorbroke something
Kevin Rue-Albrecht (17:17:18): > https://travis-ci.org/csoneson/iSEE/builds/434324050?utm_source=github_status&utm_medium=notification
Kevin Rue-Albrecht (17:19:12): > nevermind. I forgot to update the unit test
Kevin Rue-Albrecht (17:22:08): > as painful as they are to write, those unit tests really are life savers
Kevin Rue-Albrecht (17:27:55): > Any preference or magic number to chunk the cytof data set by row? > Current GH: 35 (all rows in one chunk), > Recent suggestion: 1 > Naive suggestion: .. say 5, to make a few chunks, but not quite 1 by row > ?
Aaron Lun (17:28:39): > make them longish row-wise.
Aaron Lun (17:28:51): > Say, about c(1, 2000)
Kevin Rue-Albrecht (17:29:19): > I don’t really have a sense to optimise hdf5, so I’m all ears
Kevin Rue-Albrecht (17:29:26): > for the record: > > > dim(bcrxl) > [1] 35 172791 >
Kevin Rue-Albrecht (17:29:44): > so it’s better to stick with 1 row-wise?
Kevin Rue-Albrecht (17:30:56): > From a quick division: > > > 172791/35 > [1] 4936.886 >
Kevin Rue-Albrecht (17:31:11): > makes sense to putc(1, 5000)in this case? (it would be the equivalent of a ‘square’)
Kevin Rue-Albrecht (17:55:38): > Ok, there’s an expected issue now with the Docker build. I need to move the Dockerfile back to the root of the package, and Charlotte needs to reset the default configuration on DockerHub. This is becausedevtools:install()needs to be run from the root of the package, and right now it’s being run from the location of the Dockerfile, which isinst/extdata. Hence the error here:https://hub.docker.com/r/csoneson/isee/builds/bienjuarxesax9yxg3hijqj/Could not find package root.
Aaron Lun (17:57:08): > Re chunk size: that sounds fine.
Aaron Lun (17:57:31): > Don’t make the chunks too long as that will penalize sample assay plots (should anyone be silly enough to construct them).
Aaron Lun (17:57:43): > Re: docker. Sounds easier to just leave it be.
Kevin Rue-Albrecht (17:59:33): > Easier, but not correct. I don’t know what went through my mind when I wrote the install_iSEE.R script that installs the latest bioc-devel version every time we push to master. No wonder we’re confused why the container behaves differently than the code we just pushed.
Kevin Rue-Albrecht (18:00:44): > We still have a decent amount of time before the next release so that I can fix that. Plus it’s invisible from the point of view of package users. It’s only really affecting the Docker containers which I’m probably the only one in the world to care about
Aaron Lun (18:09:18): > Well, just have it pull the Bioc-devel version.
Kevin Rue-Albrecht (18:11:59): > Fair enough, we can always throw in some dummy commits on master to pull a new bioc-devel version if we need to
Aaron Lun (19:21:12): > Or you could justdevtools::installfrom Charlotte’s repo.
Aaron Lun (19:21:25): > No need to change directory.
2018-09-28
Kevin Rue-Albrecht (05:32:50): > what the hell:object 'int_metadata' is not exported by 'namespace:SingleCellExperiment'https://hub.docker.com/r/csoneson/isee/builds/baanxdvvjmn2uyfnkczsyso/
Kevin Rue-Albrecht (05:33:39): > a bit higher in the report, it only saysinstalling **source** package 'SingleCellExperiment' ..., but not the version
Kevin Rue-Albrecht (05:35:43): > ah no, here we gohttps://bioconductor.org/packages/3.7/bioc/src/contrib/SingleCellExperiment_1.2.0.tar.gz
Aaron Lun (06:44:16): > Aaron’s sweet-ass job hunt self-promotion. Find yourself! - File (PDF): overview.pdf
Kevin Rue-Albrecht (06:46:49): > :slightly_smiling_face:
Kevin Rue-Albrecht (06:47:31): > iSEE under “scRNA-seq”? I thought we were all-encompassing:stuck_out_tongue_winking_eye: - File (PNG): Pasted image at 2018-09-28, 11:47 AM
Aaron Lun (06:47:49): > Yeah, couldn’t be bothered making a separate category.
Kevin Rue-Albrecht (06:51:37): > on a separate note, neitherdevtools::install(local master branch) nordevtools:install_github("csoneson/iSEE")can be used yet in the docker container because something somewhere still throws an error'BiocInstaller' must be installed to install Bioconductor packages
Kevin Rue-Albrecht (06:52:18): > I guess that might only be solved by/after the next Bioc release in October
Aaron Lun (06:52:59): > ¯*(ツ)*/¯
Federico Marini (09:19:01): > isn’t kmknn already out?
Federico Marini (09:19:16): > Laleh’s stuff, isnt it?
Aaron Lun (09:20:04): > No, that’s something totally different.
Aaron Lun (09:20:06): > Laleh’s is MNN.
Aaron Lun (09:20:29): > kmknn is just a fast way of doing exact nearest neighbors. Nothing to do with single-cell, but I just put it in there as part of the scran infrasturcutre.
Aaron Lun (09:20:37): > Do not confuse with the method from Sarah T’s lab.
Federico Marini (09:21:16): > ok, now I am confused:smile:
Federico Marini (09:21:54): > but apart from that, are you applying somewhere else already or just putting stuff together?
Aaron Lun (09:22:23): > interview mid next month
Federico Marini (09:22:47): > where if I might ask?
Federico Marini (09:23:54): > I don’t know when the whole process will start here - we need to get the new head of the institute first - but we will open a W2 professorship in bioinfo
Aaron Lun (09:24:03): > Genentech
Federico Marini (09:24:38): > Is Michael Lawrence (still) there, I guess?
Aaron Lun (09:24:53): > yep.
Federico Marini (09:26:12): > can I say they’d be stupid if they don’t want you?:slightly_smiling_face:
Aaron Lun (09:27:12): > I try to suppress those thoughts.
Aaron Lun (09:27:35): > That kind of attitude was probably why I didn’t get a bunch of fellowships.
Federico Marini (09:31:49): > ouh:disappointed:
Federico Marini (09:32:07): > could they somewhat “penalize” too much commitment to open source stuff?
Federico Marini (09:32:19): > btw, what is then “too much”
Aaron Lun (09:32:33): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (09:37:20): > from personal experience, “too much” seems to be when open source projects significantly impact/reduce the time spent on in-house research projects (i.e. those listed in the original grant)
Aaron Lun (14:00:19): > why is bioc stil on 1.1.8?
Aaron Lun (14:00:24): > I thought we pushed?
Kevin Rue-Albrecht (14:01:39): > not yet, that’s what i said a few days ago, I was hoping to clear up the docker situation before pushing upstream, but obviously the docker issues not directly related to the bioc version, so technically we could (push upstream)
Kevin Rue-Albrecht (14:03:07): > I’m just pushing a revert to install from bioc-devel in the docker; devtools has issues related to still using BiocInstaller somehow
Aaron Lun (16:05:07): > FINISH HIM:-1:
Kevin Rue-Albrecht (16:05:46): > as soon as it clears on master
Aaron Lun (16:06:37): > Ah good, someone’s still at work.
Kevin Rue-Albrecht (16:06:51): > multitasking with tv show ^^
Aaron Lun (16:10:45): > What? You’re not in the office?
Aaron Lun (16:10:47): > Boo!
Aaron Lun (16:10:52): > Married life:-1:
Kevin Rue-Albrecht (16:10:58): > hehehe
Kevin Rue-Albrecht (16:29:47): > alright, i’ve just merged, i’ll push upstream now
Kevin Rue-Albrecht (16:30:56): > boom
Aaron Lun (16:32:10): > and the dirt is gone.
2018-09-29
Charlotte Soneson (06:31:13): > I added a check that there is at least one row ofplot.datafor the heatmap, in theemptyheatmapfixbranch. I checked that it works (doesn’t generate a heatmap, gives a message) if the transmitting selection is empty - Aaron, did you have another use case (and can you then check that it works too)?
Aaron Lun (08:42:06): > EmptyDrops MS got rejected from Nat comms.
Aaron Lun (08:42:07): > Fuck.
Aaron Lun (08:42:12): > I don’t want to do any more work on it!
Aaron Lun (08:42:25): > Well, at least this is better than having to respond to a million idiotic comments.
Charlotte Soneson (08:44:04): > Well, you can always send it somewhere else and get other comments…
Aaron Lun (08:46:15): > I mean, the package is complete. There’s nothing to change, so what’s the point of all this extra work?
Charlotte Soneson (08:47:53): > :shrug:“Added value”?
Aaron Lun (08:56:41): > “citations” and “h-index” is the only answer I have.
Federico Marini (09:16:22): > what was the (main) reason for rejection?
Federico Marini (09:16:31): > after one or two rounds? or desk rejection?
Aaron Lun (09:30:33): > one round. Haven’t read the reviews, don’t want to start my WE on a bad mood.
Federico Marini (09:34:16): > :disappointed:
Aaron Lun (10:37:44): > Anyway,@Charlotte Soneson, onto theemptyheatmapfix. I would like us to avoid usingvalidate()statements where possible. In fact I have tried to remove allvalidatestatements in the other plots, as an invalid plot means that point transmission to other plots becomes difficult to define. Rather, I would try to create some sensible plot when there are no samples or features, instead of displaying an error message.
Kevin Rue-Albrecht (10:39:16): > You can check out the empty plot that I wrote for the custom panel vignette. The custom PCA that Aaron wrote
Aaron Lun (10:39:33): > I don’t know what this would mean in the heatmap case. For no features, I can imagine just printing the top bar with sample colourings. For no samples… I guess we should have the top bar that’s empty, and the names of the genes, and an all-white “heatmap” that isn’t visible.
Kevin Rue-Albrecht (10:43:13): > Sorry to hear about the empty drop MS
Aaron Lun (10:48:09): > ¯*(ツ)*/¯
Aaron Lun (10:48:12): > Do not care anymore.
Kevin Rue-Albrecht (14:13:24): > Did we spot this one yet?https://twitter.com/slowkow/status/1042797163309420549?s=12 - Attachment (twitter): Attachment > Check out cellBrowser by Maximilian Haeussler from @ucsc: > > :package: https://github.com/maximilianh/cellBrowser > > You can browse #SingleCell gene expression data across 1.3 million cells from @10xgenomics: > > :globe_with_meridians: https://cells.ucsc.edu/?ds=10xMegacell > > #opensource #python #javascript #bioinformatics #BigData
Aaron Lun (14:14:15): > ¯*(ツ)*/¯
Aaron Lun (14:14:54): > Think this is a CZI output.
Aaron Lun (14:18:16): > Yes, I do recall this being mentioned in the CZI visualization session.
Aaron Lun (14:18:35): > There’s going to be a million different tools for scRNA-seq visualization, I wouldn’t worry about it.
Aaron Lun (14:19:09): > Though I would like to get one up and running for the 1 million cell data set. Any volunteers?
Aaron Lun (14:21:47): > Maybe@Charlotte Sonesonalready has some t-SNE coordinates we can use? Or UMAP. Anything’s fine, really.
Charlotte Soneson (14:22:40): > Not really. But you applied PCA at some point, didn’t you?
Aaron Lun (14:23:31): > Yeah, but getting it tSNE’d is a drag. Check out#bigdata-repfor some comments on how we could speed it up. But it seems I’m the only guy pulling this particular horse.
Aaron Lun (14:23:57): > I’d love to be able to make this work on the 1M data set with a big fuck you to the CZI-funded groups.
Charlotte Soneson (14:24:32): > Ehm - you also got some money from them if I remember right…:thinking_face:
Aaron Lun (14:24:57): > Well, not enough.
Aaron Lun (14:24:59): > NOT ENOUGH.
Aaron Lun (14:25:18): > It barely funds me for half a year.
Kevin Rue-Albrecht (14:57:36): > We just discussed this paper in journal club this week:https://www.cell.com/action/showPdf?pii=S2405-4712%2818%2930235-7
Kevin Rue-Albrecht (14:58:33): > figure 4 says that they can approximate 1.5 days worth of 10x Genomics t-SNE in about an hour
Kevin Rue-Albrecht (14:59:43): > Software availability says “A C++ implementation of net-SNE with example data and scripts are available at:http://netsne.csail.mit.eduandhttps://github.com/hhcho/netsne.” and the Github says that there is an R wrapper here:https://github.com/schwikowskilab/rNetSNE - Attachment (GitHub): Build software better, together > GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects. - Attachment (GitHub): schwikowskilab/rNetSNE > Contribute to schwikowskilab/rNetSNE development by creating an account on GitHub.
Kevin Rue-Albrecht (15:00:28): > what I love about the R wrapper is thatR_netSNE(..., path.netSNE.dir = "/opt/ohpc/pub/libs/net-SNE/20180419/bin/")
Kevin Rue-Albrecht (15:00:59): > that’s what Itrulycall a wrapper:joy:
Federico Marini (15:13:15): > very much WYSIWYG:stuck_out_tongue:
Aaron Lun (16:17:25): > computational methods? in cell? What a joke.
Federico Marini (16:23:48): > cell systems
Aaron Lun (16:25:57): > Hm.
Aaron Lun (16:26:02): > That’s a bit better, I guess.
Aaron Lun (16:26:14): > Bioinformaticsor nothing, that’s my motto from now on.
Aaron Lun (16:58:21): > I”M BCK
Aaron Lun (16:58:23): > BACK
Aaron Lun (17:42:44): > Well, I was going to try to explain the neat speed-up for Monte Carlo p-value calculations inemptyDropsduring the seminar, but I threw it out. Far too technical.
2018-09-30
Federico Marini (04:56:12): > mornin’ everyone
Federico Marini (04:56:44): > just for your info: I’ll be on an island in the nordsee for t he next 10 days
Federico Marini (04:56:57): > our landlord mentioned “some wifi connection”
Federico Marini (04:57:26): > hope he’s right:slightly_smiling_face:
Federico Marini (04:57:45): > otherwise you will see me on a notsoregular basis
Federico Marini (04:58:34): > if anything is pretty urgent, you can try to ping me on the phone and I’ll try to get in:wink:
Charlotte Soneson (05:00:35): > Have fun! Maybe there should be a German “See” in our logo:upside_down_face:
Federico Marini (05:03:06): > oh
Federico Marini (05:03:27): > this will force me to find a fitting picture once I am there
Federico Marini (05:03:34): > :stuck_out_tongue:
Federico Marini (05:03:53): > but anyway, we should probably converge soon on that, if we want stickers to be given around in december
2018-10-01
Kevin Rue-Albrecht (05:24:58): > For those not subscribed to the HCA Slack channel “data-analysis”, they’ve just posted about clustviz. Shame we didn’t think of posting there, now it would sound redundant: > > garybader [2:31 AM] > > Hi everyone - we’ve released an R package designed to make it easier for users to cluster and visualize their scRNA-seq data.https://f1000research.com/articles/7-1522/v1There is also a way to use the package to distribute your data along with a Shiny interface from GitHub e.g.https://github.com/BaderLab/MouseCortex - Attachment (f1000research.com): F1000Research Article: scClustViz – Single-cell RNAseq cluster assessment and visualization. > Read the latest article version by Brendan T. Innes, Gary D. Bader, at F1000Research. - Attachment (GitHub): BaderLab/MouseCortex > R data: Developmental emergence of adult neural stem cells as revealed by single cell transcriptional profiling - BaderLab/MouseCortex
Aaron Lun (05:25:48): > I wouldn’t worry about it.
Kevin Rue-Albrecht (05:46:11): > FYI, next time we merge to master, the iSEE docker container will pick up version 0.7.45 of DelayedArray, which contains the fix implemented in 0.7.44. Currently our container still only includes 0.7.43 which doesn’t contain the fix.
Kevin Rue-Albrecht (05:47:10): > But I’ll be unavailable most of today: I’ve got to prepare for a seminar, and then project meeting.
Aaron Lun (08:30:04): > Boo! Meetings! Boo!
Aaron Lun (08:30:11): > Buy a Nintendo Switch and take it to your meetings.
Kevin Rue-Albrecht (09:27:13): > omg i feel old: what the hell is this thing?! looks cool!
Aaron Lun (12:49:31): > can we mergehdf5paths?
Kevin Rue-Albrecht (13:02:02): > eh.. give me the evening, i’m sure I was thinking about something to do
Kevin Rue-Albrecht (13:02:48): > ah right i got it: good that i opened an issue to remind me adding the version specification to the DelayedArray depedency
Kevin Rue-Albrecht (13:03:27): > oh sorry wait, you meant iSEE2018
Kevin Rue-Albrecht (13:05:06): > I think so. I’ll still have to check on the cytof tour as soon as we trigger a Docker container that includes the new version ofDA
Kevin Rue-Albrecht (16:05:29): > eh…https://twitter.com/bioturing/status/1045098861222539265:cry: - Attachment (twitter): Attachment > Check out #BioTuringBrowser Single-cell add-on, a package that makes access to #singlecell #rnaseq data as easy as reading a paper! > > 3D interactive visualization > cell type prediction > *sub-clustering > > Subscribe to get updated about its release - at http://bioturing.com
Aaron Lun (16:17:46): > blah blah blah
Kevin Rue-Albrecht (16:18:16): > :stuck_out_tongue_winking_eye:
Aaron Lun (18:11:08): > Nonetheless, we should probably try to set up a 1 million cell instance.
2018-10-02
Kevin Rue-Albrecht (05:29:50): > I forgot to ask: how’s the poster coming along?
Aaron Lun (05:34:03): > most of it’s done, just need to finish the mechanisms.
Brendan Innes (11:50:33): > @Brendan Innes has left the channel
2018-10-03
Kevin Rue-Albrecht (03:48:23): > oh we lost a groupie:cry:
Kevin Rue-Albrecht (03:48:42): > anyway, I’m testinghdf5pathsbranch
Kevin Rue-Albrecht (03:48:55): > - cytof: check
Kevin Rue-Albrecht (04:05:03): > - tcga: check
Kevin Rue-Albrecht (04:05:09): > the others haven’t changed.
Kevin Rue-Albrecht (04:05:18): > so it’s good for merge, for me
Kevin Rue-Albrecht (05:25:02): > I think I’ll add an option toiSEEto set the app title width:https://github.com/csoneson/iSEE/blob/master/R/iSEE-main.R#L189 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (05:25:42): - File (PNG): Screenshot 2018-10-03 10.25.22.png
Kevin Rue-Albrecht (05:25:43): > Here’s the last born
Federico Marini (15:49:58): > new instance?
Federico Marini (15:50:13): > greetings from the island of juist, where it is daaaamn windy:smile:
Kevin Rue-Albrecht (15:51:32): > Hey, an instance that currently only exists locally on our cluster. A way to check check out genes that are markers of cell types
Kevin Rue-Albrecht (15:52:27): > still.. at the moment, one needs to know which ones to look at ahead of time.. kind of defeats the purpose, but at least the app took me no time to set up thanks to iSEE
Federico Marini (16:02:00): > gutes im bösen, so to say
Kevin Rue-Albrecht (16:03:50): > “there’s good in bad” ?
Kevin Rue-Albrecht (16:05:58): > ah and the other thing that keeps me from making it a GH-showcase instance is that for this one I downloaded the ~150 CEL files from GEO. If I can find a way to avoid that or make it “more automated”, I’ll consider pushing it to the repo of instances
Kevin Rue-Albrecht (16:06:44): > GEOquerymight solve that. I’ve used it to fetch metadata, but I think it can also get the data in some cases
Aaron Lun (16:10:25): > Incidentally, brushing to color when the plot already has a color has the potential to fuck things up.
Kevin Rue-Albrecht (16:11:10): > really? I thought the only risk was to get confused with the mix of colors
Aaron Lun (16:11:11): > Specifically, when all the brushed points are also all of the highly expressing cells for a gene, and you colour by the expression of that gene, the colour scale seems to reset.
Kevin Rue-Albrecht (16:11:21): > oh
Aaron Lun (16:11:29): > I will try to make a MWE tomorrow.
Aaron Lun (17:40:12): > haha, I saw the iSEE loading message.
2018-10-04
Aaron Lun (07:03:35): > Bug report - Attachment (GitHub): Colour scale is not invariant to selecting on a different color · Issue #243 · csoneson/iSEE > Run example(iSEE) and then redDimPlotArgs <- new('DataFrame', nrows=1L, rownames=paste0('redDimPlot', seq_len(1))) redDimPlotArgs[['SelectByPlot']] <- "Feature as…
Kevin Rue-Albrecht (07:23:24): > :rotating_light:
Aaron Lun (17:51:07): > … HSBC?
2018-10-05
Kevin Rue-Albrecht (03:59:59): > not sure how to take your last message into account (oh-oh)
Kevin Rue-Albrecht (16:42:18) (in thread): > issue fixed
Kevin Rue-Albrecht (16:42:47) (in thread): > https://github.com/csoneson/iSEE/commit/f1d11c29e1f778ef1271403affc4644d5e51d436 - Attachment (GitHub): fixes #243 · csoneson/iSEE@f1d11c2 > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (16:57:04) (in thread): > Just a quick helper script to work on the heatmap > > heatMapArgs <- heatMapPlotDefaults(sce, 1) > > initialPanels <- DataFrame( > Name=c("Reduced dimension plot 1", "Feature assay plot 1", "Heat map 1"), > Width=c(4L, 4L, 4L), > Height=c(500L, 500L, 500L) > ) > > iSEE(sce, heatMapArgs = heatMapArgs, initialPanels=initialPanels) >
Kevin Rue-Albrecht (16:57:15) (in thread): > (afterexample(iSEE))
Kevin Rue-Albrecht (17:08:56) (in thread): > I don’t have an immediate fix for this heatmap issue. > However, I’d like to mergeemptyheatmapfixafter a version bump and push upstream to fix the color scale issue that@Aaron Lunopened in issue #243
2018-10-06
Kevin Rue-Albrecht (08:40:27): > ahh the joy of going through the post-acceptance nested checklists of Nature Comm. ^^
Aaron Lun (12:46:11) (in thread): > ok
2018-10-07
Kevin Rue-Albrecht (04:56:22): > Hi all, I’d like to mergehttps://github.com/csoneson/iSEE/pull/244tomastertoday, and possibly pushupstream(after a version bump). > Charlotte protected the heatmap against restriction to 0 samples, and I’ve fixed@Aaron Lun’s issue about the color scale. - Attachment (GitHub): Emptyheatmapfix by kevinrue · Pull Request #244 · csoneson/iSEE > Candidate for version bump
Aaron Lun (06:08:17): > Good, good.
Aaron Lun (06:08:41): > Is it really that hard to just create an empty heatmap, rather than usingvalidate?
Kevin Rue-Albrecht (06:09:27): > I’ve briefly tried: the main heatmap isn’t the issue, I ran into trouble with the annotation bar at the top
Kevin Rue-Albrecht (06:10:12): > because that one receives a 0-row data.frame, and ggplot2 needs at least one value to map aesthetics to
Kevin Rue-Albrecht (06:10:33): > (it’s not enough to have the right column names on that 0-row data frame)
Aaron Lun (06:11:04): > GURGH
Aaron Lun (06:11:31): > Well, let’s do one at a time. Can we get rid of the validate for the no-gene case?
Kevin Rue-Albrecht (06:12:41): > Perhaps one could still extract all the relevant colData information (given that the heatmap always shows all samples anyway) even when no samples are selected. I just had to dive back into the Nat. Comm. checklist as I only have the weekend to do that.
Kevin Rue-Albrecht (06:12:51): > Not sure how you want to get rid of that validate either
Aaron Lun (06:13:27): > No genes but with samples.
Kevin Rue-Albrecht (06:13:30): > I mean the complexity of the heat map plot panel is that it’s not just one plot: it’s (at least) 3 different plots that we are linked to one another
Aaron Lun (06:13:32): > The annotation bar should be valid.
Aaron Lun (06:13:40): > And the main heatmap should be empty.
Kevin Rue-Albrecht (06:14:11): > the annotation would be valid, but I guess there’ll be an issue about computing the heat map color scale from 0 values
Kevin Rue-Albrecht (06:14:44): > or a big NaN issue if the heat map is scaled (centred too maybe)
Aaron Lun (06:15:38): > Perhaps just create an emptyggplot()to fill the space.
Kevin Rue-Albrecht (06:15:39): > the other thing that we recenty discussing with Charlotte is that the heat map panel is an endpoint anyway (i.e., doesn’t transmit further), how do you imagine it could invalidate the app state in any way?
Aaron Lun (06:16:09): > It won’t. But it would be good pracice to not rely onvalidates if we don’t do it anywhere else.
Aaron Lun (06:16:29): > No other panel produces error messages.
Kevin Rue-Albrecht (06:19:13): > I don’t know how much each of you guys usesComplexHeatmap, but that is usually my go-to choice, as it can display row and column annotations as well as legends as a single plot. It’s notggplotbased, but that could relieve us from all that “3-split plots” issue I’ve just mentioned. (maybe, I haven’t tried implementing that approach yet)
Aaron Lun (06:19:52): > I’m happy to go with that.
Kevin Rue-Albrecht (06:20:25): > point being that in that case, there’d be only one plot to manage at the end, and it’d probably be easier to show a blank plot if any row or column issue arises during the “plot assembly”
Kevin Rue-Albrecht (06:20:44): > (I did write “probably” in the message above)
Kevin Rue-Albrecht (06:21:58): > In the meantime, I’ll bump up the current PR, merge and push upstream. We can’t guarantee that the rest of the heatmap issue can be solved before the next bioc release
Kevin Rue-Albrecht (06:24:01): > that said, just looking at the NEWS file now, did we say something about moving it frominst/NEWS.RdtoNEWS.md?
Kevin Rue-Albrecht (06:26:20): > Looking at core bioc packages, BiocParallel adopted the simplerNEWSformathttps://github.com/Bioconductor/BiocParallel/blob/master/NEWS
Kevin Rue-Albrecht (06:26:45): > Hell you know what, I’ll just stick with what worked for us so far
Aaron Lun (06:46:19): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (06:49:07): > alright 1.1.10 sent upstream
Charlotte Soneson (07:46:01): > I can’t remember exactly, but wasn’t one reason why we went forggplot2heatmaps rather thanpheatmap/ComplexHeatmapthe zooming capabilities?
Charlotte Soneson (07:46:27): > Even if we don’t transmit, we can still interact with the current heatmap
Kevin Rue-Albrecht (08:25:09): > that’s very possible indeed, not sure whether shiny allows interaction with non-ggplot figures
Kevin Rue-Albrecht (08:33:46): > FYI, following the merge today, the Docker container of thecytoftour works with HDF5 now
Kevin Rue-Albrecht (08:34:23): > I’ll have to upload to DockerHub tomorrow from the Institute, as it takes more than forever on my home connection
2018-10-08
Kevin Rue-Albrecht (08:29:58): > https://twitter.com/jamesaeddy/status/981997635790823425with link tohttps://isb-cgc.shinyapps.io/shiny-iatlas/ - Attachment (twitter): Attachment > Lot’s of cool stuff happening today. Here’s a not-so-short summary: (1/n)
2018-10-09
Kevin Rue-Albrecht (08:51:23): > Anyone in favor of a functionavailablePanels()? > I find myself callingiSEE:::translation(triple colon:face_vomiting:) to copy paste the accurate panel names to buildinitialPanels
Kevin Rue-Albrecht (08:52:45): > Plus, we could throw in anavailablePanels(encoded=FALSE)whether to return thenames(colDataPlot`), to help users figure out the names of the UI elements to anchor tour steps
Kevin Rue-Albrecht (08:53:04): > (maybeavailablePanelsTypes())
Aaron Lun (09:10:33): > huh?
Kevin Rue-Albrecht (09:11:33): > well, as developers, we know the panel names pretty well, but I still need to look them up sometimes to avoid typos when I define the initial panels
Kevin Rue-Albrecht (09:14:09): > And I’m usually looking up the panel names inconstants.Rto get them right. I was thinking that for novice users, it’d be nice to have a function to avoid switching to the docs/vignettes/GitHub even if the function only exposes > > availablePanels() <- function() { > translation # i.e., iSEE:::translation > } >
Aaron Lun (09:21:16): > You might as well have that as a internal function as well.
Aaron Lun (09:21:53): > Avoid usingtranslationdirectly in internal code, I mean.
Aaron Lun (09:21:57): > Not that it really matters.
Kevin Rue-Albrecht (10:13:04): > Well I don’t remember who created ‘translation’, but it never bothered me. > That said, I agree that if we make a public function we might as well use it internally. > Which also means that ‘rev.translation’ would need its own, maybe.
Aaron Lun (10:13:22): > Yeah I did it.
Aaron Lun (10:13:33): > Well, probably just KISS it for now.
Kevin Rue-Albrecht (10:13:47): > anyway, I just to sanity check the concept
Kevin Rue-Albrecht (10:14:04): > Is that an acronym? KISS?
Aaron Lun (10:14:05): > most functions will not accesstranslationdirectly, but via the encoding/decoding functions.
Kevin Rue-Albrecht (10:14:17): > Keep in…
Aaron Lun (10:14:17): > Yes. Your fiancee doesn’t have to worry.
Aaron Lun (10:14:34): > Keep it simple stupid
Kevin Rue-Albrecht (10:15:01): > She’ll be relieved:stuck_out_tongue_winking_eye:
Aaron Lun (14:36:00): > Also, to all: I updated the author info inDESCRIPTION. Check out my sweet new email.
Kevin Rue-Albrecht (14:38:28): > nooooooo:rolling_on_the_floor_laughing:
Kevin Rue-Albrecht (14:40:15): > don’t forget@author Aaron Lun \email{alun@@wehi.edu.au}https://github.com/csoneson/iSEE/blob/master/R/iSEE-pkg.R#L13 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (14:41:33): > and you’ve gotemail:aaron.lun@cruk.cam.ac.ukin the vignettes
Aaron Lun (14:43:38): > Ah shit.
Aaron Lun (17:59:03): > Can someone re-fire the travis build for 246?
Kevin Rue-Albrecht (18:05:06): > will do
Kevin Rue-Albrecht (18:05:32): > done
2018-10-11
Federico Marini (04:49:04): > Hi team, I’m back on the main land now:slightly_smiling_face:
Federico Marini (04:49:17): > couple of sparse thoughts on open stuff (minor, mostly):
Federico Marini (04:49:51): > - the dl stats did hop up a little in the last month
Federico Marini (04:50:19): > - logo: if we want it for eurobioc, we probably need to finalize soon
Federico Marini (04:50:36): > - pkgdown: what are the thoughts on that?
Kevin Rue-Albrecht (04:50:50): > true, DL basically doubled from Aug to Sep
Federico Marini (04:51:08): > - conda recipe: as of now it is not possible yet, but I keep it on my todo list once it is possible to add pkgs from the 3.7 release
Kevin Rue-Albrecht (04:51:15): > although we’re a 3rd into Oct now, and only 20 DL reported
Federico Marini (04:51:54): > might have been a spike, but still, it went up nicely
Kevin Rue-Albrecht (04:52:03): > Not sure how much I can help on the logo. Lots going on here, and our Illustrator license expired last week
Kevin Rue-Albrecht (04:52:23): > plus I’m usually better at patching things together thanactuallydrawing
Federico Marini (04:53:06): > about the logo, on the way to the north sea island I had a potential illumination
Federico Marini (04:53:27): > with a lighthouse in the logo, and have iSEE shine light on your data
Federico Marini (04:54:08): > while still having somehow in mind that the haven pkg from hadley already might use the lighthouse
Federico Marini (04:54:12): > and indeed, it does
Federico Marini (04:54:14): > so
Kevin Rue-Albrecht (04:54:33): > like the beam of light from the lighthouse focused on a cluster in a tSNE maybe?
Federico Marini (04:54:53): > I also liked the prism+light idea, but I think, the more minimal we keep it, the better
Federico Marini (04:55:30): > the goggles are quite linked to seeing without too much of mental load
Federico Marini (04:55:52): > and we’d have something with CC license to start on
Federico Marini (04:58:02): > that said, I need to finish wrapping up soon my ms on pcaExplorer, especially after some other group posted the PCAGO preprint (seems also formatted for bioinfo, so double shame/pity if that gets through while mine was rejected back then)
Kevin Rue-Albrecht (05:01:52): > I agree that the prism prototype was overloaded. I was hoping to convey the multiple panel types somehow. > Light house could be nice although there’s a biorXiv out there that used the idea recently:https://www.biorxiv.org/content/biorxiv/early/2018/08/20/395004.1.full.pdf
2018-10-12
Charlotte Soneson (03:44:57): > I was just discussing briefly with Kevin, and I’m starting to agree with him that the Gviz panel would be more suitable as a custom panel. I thought a lot about the organization, and it seems to me that, in contrast to other panels which are very generic, this will be much more specific - for example, if you are working with methylation, you may not want the genes as a track, but rather the CpG islands, or something completely different. Also, the only way I could see that this would transmit to any other panel would be to send a selected range (which would assume that the SCE has rowRanges that can be matched) - we can’t rely on IDs since we don’t know what’s in the SCE. Maybe in this case people could just search for the range in the feature table (don’t know if you can search for start/end position in a range actually, but it would be nice). So with all the specifications that would have to be done, this would lead to a lot of selections and exceptions, and in the end it may be easier to just write a custom panel. It will also mainly be useful for data sets with few samples - otherwise the app will choke on all the coverage tracks. I can set up an example (similar to the one I sent some time ago) to illustrate how this can be done at least for RNA-seq data. Any objections?
Federico Marini (05:38:33): > Sounds very reasonable to me
Aaron Lun (05:39:37): > Okay, no problems.
Aaron Lun (05:39:52): > Well, we should have “prebuilt” custom panel functions, so to speak.
Kevin Rue-Albrecht (06:21:26): > Yeah. I’ll set up a GH repo this weekend, using the examples from the vignette as starter. I just need to decide on a template for each example as a folder like for the tours
Federico Marini (06:23:24): > does it fit under theiSEE_instanceswe already have?
Kevin Rue-Albrecht (13:22:00): > I’d rather keep the « instances » repository for « fully featured » apps which as you originally said attempt to reproduce figures from papers. > I’m thinking of the « custom » repo as a list of thumbnails to single-panel apps (Max two if point selection is relevant).
Kevin Rue-Albrecht (13:24:16): > We’ll want to minimise the risk of confusion by having only the code of the single custom panel in each example app. Otherwise we’ll easily end up with apps that define multiple custom panels, which will confuse novice users about which panel is coded/demonstrated where.
Aaron Lun (13:24:53): > I was thinking of a convenience function that is baked intoiSEE.
Aaron Lun (13:25:13): > Or a number of these for common tasks, e.g., sub-dimensionality reduction, DE detection, and so on.
Kevin Rue-Albrecht (13:25:56): > Maybe that’s where Shiny modules could come in (?)
Aaron Lun (13:26:03): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (13:26:23): > But yeah. I had DE on my mind too
Kevin Rue-Albrecht (19:08:40): > Alright, off to sleep, but here goes a first template:https://github.com/kevinrue/iSEE_custom/tree/master/table_cachedFoldChange
Kevin Rue-Albrecht (19:08:55): > justsetwdto the folder and runapp.R
Kevin Rue-Albrecht (19:10:23): > @Federico MariniI still think it is better to keepinstancesandcustomseparated, so thatinstancesremain more ambitious reproductions of research work, whilecustomshowcases one panel per app
Federico Marini (19:14:42): > fine:wink:
Federico Marini (20:19:37): > trying it out,
Federico Marini (20:19:51): > question: what is transmitted from the row data plot to the table?
Federico Marini (20:20:11): > (might be because it is past 2 am that the question comes)
2018-10-13
Kevin Rue-Albrecht (05:15:56): > the set of genes to show in the table
Kevin Rue-Albrecht (05:16:41): > I haven’t completed the tour yet, because I stopped around midnight, but I’ll finish it today
Kevin Rue-Albrecht (06:29:52): > Tour extended. It should be clearer now
Kevin Rue-Albrecht (08:36:59): > I’ve just made a README, can I please get some opinions?https://github.com/kevinrue/iSEE_customIf you’re happy with it, I’ll make it public already.
Kevin Rue-Albrecht (14:25:40): > availablePanelTypesfunction here:https://github.com/csoneson/iSEE/compare/Available_panels?expand=1I’ve resolved all the code updates for the app to work as before, as far as I can see. - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (14:26:00): > Happy to hear feedback about merging before the Bioc release
2018-10-14
Charlotte Soneson (04:30:41) (in thread): > The readme looks good to me!
Kevin Rue-Albrecht (12:33:54): > @Aaron Lunpush incoming to iSEE repo. I’ve removed the function and exported the named character vectors instead
Aaron Lun (12:52:56): > Good, good… your hate has made you strong.
Aaron Lun (12:53:16): > Now, kill your father, and your conversion to the dark side will be complete!
Aaron Lun (12:53:45): > Or in other words:Yes.
Kevin Rue-Albrecht (12:53:49): > you’re right I might push “master”
Kevin Rue-Albrecht (12:54:16): > after all I want to be a good padawan
Kevin Rue-Albrecht (14:57:05): > I’ll take the updated package for another spin before merging, because I’d like to version-bump and push upstream for the upcoming bioc release, and I wouldn’t like any runtime error that might not be caught by the unit tests
Aaron Lun (15:28:49): > Okay. I’ll probably be out of it for the next week, in SF for the interview.
Kevin Rue-Albrecht (17:30:53): > good luck!
2018-10-15
Federico Marini (03:25:08): > good luck Aaron:wink:
Aaron Lun (08:18:14): > Worth keeping an eye on the#hca_rfa, they’re planning to put something together regarding visualization.
Federico Marini (08:22:47): > Wouldn’t we also be a quite natural fit? I checked briefly the minutes of the latest call
Kevin Rue-Albrecht (14:59:05): > happy to stay in the loop about the HCA RFA
Kevin Rue-Albrecht (15:01:08): > @Aaron LunBtw, I was testing out the Allen instance on the Marioni Shiny server. I tried opening the vignette from there, but it says “not built on this system”, which means it’s runningrunLocal=TRUE(default). I assume those instances are running the code onLTLA/iSEE2018? which means the code should be update? or am i wrong?
Aaron Lun (23:16:37): > ¯*(ツ)*/¯
Aaron Lun (23:24:11): > Not seeing this.
Aaron Lun (23:25:42): > Okay, seeing it now.
Aaron Lun (23:26:07): > Well, it’s because I installed it from Github rather than from BioC.
Aaron Lun (23:26:37): > GH installs don’t have the built vignette.
2018-10-16
Kevin Rue-Albrecht (04:58:46): > Ok, then we probably just need to change theapp.RtorunLocal=FALSE. Or perhapsrunLocal=interactive()
Aaron Lun (05:01:08): > probably
2018-10-17
Federico Marini (04:37:29): > Release is coming soon, so no breaking changes will come up, but yet…
Federico Marini (04:37:47): > what are your thoughts on the open points from last week?
Federico Marini (04:37:59): > (should i open them up as issues on gh for better tracking maybe?)
Kevin Rue-Albrecht (04:38:50): > Argh… you just made me realise that I haven’t updated the NEWS file for release 1.1.11
Federico Marini (04:39:05): > can we still edit?
Federico Marini (04:39:14): > i think we are past the freeze?
Kevin Rue-Albrecht (04:39:41): > I pushed upstream. The version bump took effect, but the Bioc website only links to the NEWS file, not the DESCRIPTION
Kevin Rue-Albrecht (04:40:07): > So we’d have to push a new version upstream just to edit the NEWS file:sweat_smile:
Kevin Rue-Albrecht (04:40:27): > i thought they only froze the RELEASE branch, not the DEVEL (?)
Federico Marini (04:40:35): > may be right
Kevin Rue-Albrecht (04:40:54): > > Monday October 15 > > Stop building Bioc 3.7, current release. Commits to this branch will be disabled. > Start building Bioc 3.9, new devel. >
Federico Marini (04:40:56): > not having anything under review this time, I lost track of what day is what re: schedule
Kevin Rue-Albrecht (04:41:24): > http://bioconductor.org/developers/release-schedule/
Federico Marini (04:44:42): > still safe then
Kevin Rue-Albrecht (04:45:05): > yeah, i’m just tracking down the changes since version 1.1.10
Federico Marini (04:45:31) (in thread): > late reply, but nice usage of the table with the screenshot
Kevin Rue-Albrecht (04:46:49) (in thread): > Thanks. I was tempted to learn about those GIF-style thumbnails, but I didn’t afford the time, and more importantly I think a static screenshot can be used to highlight the “star panel”
Federico Marini (04:47:06) (in thread): > fair enough
Charlotte Soneson (04:47:11) (in thread): > The README contains the sentence “This repository contains the code required to construct the tours described in the iSEE paper by Rue-Albrecht et al. (2018)“, which I guess was copied fromiSEE2018?
Kevin Rue-Albrecht (04:47:25) (in thread): > :grimacing:
Federico Marini (04:47:34) (in thread): > the gif is one click away if users follow the tour, so no need for the gif there
Federico Marini (04:47:41) (in thread): > although, many users might be lazy
Federico Marini (04:48:03) (in thread): > gifs might be good for twitter
Kevin Rue-Albrecht (04:48:16) (in thread): > One other thing, is that the screenshot is almost 1MB. I was wondering if you had any trick/smarter way of having something good quality yet lighter
Kevin Rue-Albrecht (04:48:39) (in thread): > Retina display might be over the top in terms of screenshot
Federico Marini (04:51:44) (in thread): > on a separate repo it is fine
Federico Marini (04:51:49) (in thread): > i like to see nice res
Federico Marini (04:52:07) (in thread): > in the official repo we should keep it small, so no probs
Kevin Rue-Albrecht (04:52:38) (in thread): > Alright. version 1.1.12 is building on Travis
Federico Marini (04:53:37) (in thread): > :thumbsup:
Kevin Rue-Albrecht (04:54:03) (in thread): > Version bump for a NEWS file. Can’t remember last time I did that ^^
Kevin Rue-Albrecht (04:54:47) (in thread): > Can you throw them in this thread again please? I’m too lazy to scroll up
Kevin Rue-Albrecht (04:56:43): > On a separate note, I’ve been wondering for a while now whether we could store DE results (e.g. edgeR) in nestedrowData, and then use aRow data plotpanel to draw MA plots. I just always had something else in the way of trying this. Any taker?
Charlotte Soneson (04:59:20): > We have been experimenting with adding DE results in nested data frames (one df per contrast) and then plotting - works nicely, the only thing is that we effectively lose the nesting immediately here:https://github.com/csoneson/iSEE/blob/918e464bb3ee4fccb6b99df30420f9ddd042d4af/R/iSEE-main.R#L147 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (05:01:30): > Yes. I remember that the DataFrames are “un”nested to allow the UI to display them as tables, but as I remember name clashes are prevented by prefixing the top levels. This makes long column names in the final data frame, but as you wrote, it should work, right?
Kevin Rue-Albrecht (05:01:39): > I didn’t remember that we already tried
Charlotte Soneson (05:01:39): > yes, it works
Kevin Rue-Albrecht (05:01:54): > cool, nevermind then
Charlotte Soneson (05:02:21): > I don’t know if we have discussed it (as in “we - the iSEE team”) - I tried it this weekend for a local application
Kevin Rue-Albrecht (05:03:06): > I was thinking about it when considering again@Federico Marini’s iSEE_instances. Having MA and volcano plots would definitely help in reproducing a whole bunch of papers out there
Federico Marini (05:04:22) (in thread): > Sure:smile:
Federico Marini (05:05:12): > Picking up Kev’s thread reply, here again the open points
Federico Marini (05:05:23): > - logo: if we want it for eurobioc, we probably need to finalize soon
Federico Marini (05:05:28): > - pkgdown: what are the thoughts on that?
Federico Marini (05:05:50): > - conda recipe: hopefully soon it will be possible to have that as well
Federico Marini (05:06:31): > (- once the logo is there, we could put it in the title header of the app?)
Kevin Rue-Albrecht (05:06:37): > The alternative being to have custom panels for that. However, custom panels can’t send point selection downstream, which is a big caveat if they show MA plots without any way of taking “interesting genes” forward
Charlotte Soneson (05:07:14): > agree - I don’t think there is a need for custom panels here, we’re just plotting one rowData variable against another.
Kevin Rue-Albrecht (05:07:17) (in thread): > alright, it’s thread o’clock
Kevin Rue-Albrecht (05:07:32) (in thread): > arghh you didn’t put it all in one message?!
Kevin Rue-Albrecht (05:07:51) (in thread): > - logo: if we want it for eurobioc, we probably need to finalize soon > - pkgdown: what are the thoughts on that? > - conda recipe: hopefully soon it will be possible to have that as well > - once the logo is there, we could put it in the title header of the app
Kevin Rue-Albrecht (05:08:51) (in thread): > Logo: I’d love to help, but we’re out of Illustrator license, and it’s about the only thing I know how to use to design logos
Federico Marini (05:12:10) (in thread): > I think it is more efficient to have the points as GH issues
Federico Marini (05:12:29) (in thread): > guess Aaron is traveling now/soon, and the thoughts will be sparse/lost
Kevin Rue-Albrecht (05:19:07): > @Charlotte SonesonI’ve fixed the README of iSEE_custon:https://github.com/kevinrue/iSEE_custom - Attachment (GitHub): kevinrue/iSEE_custom > Examples of custom panels for the iSEE package (https://github.com/csoneson/iSEE) - kevinrue/iSEE_custom
Aaron Lun (07:59:14): > Ugh. Good morning.
Aaron Lun (07:59:35): > I’m planning to get rid of the dodgy unnesting next release.
Kevin Rue-Albrecht (09:26:23): > go for it:thumbsdown:
Kevin Rue-Albrecht (09:26:56) (in thread): > very true
Kevin Rue-Albrecht (09:27:06) (in thread): > easier to keep track of the conversation on GH
Kevin Rue-Albrecht (09:29:17): > eh actually, I’ll do it, as I need to push it upstream anyway
Kevin Rue-Albrecht (09:32:28): > Done
2018-10-18
Vince Carey (14:31:25): > @Vince Carey has joined the channel
2018-10-19
Federico Marini (05:07:16): > Hi Vince, welcome:wave:
Kevin Rue-Albrecht (05:28:40): > ditto!
Aaron Lun (08:58:53): > My time zones are fucked up, man.
Kevin Rue-Albrecht (09:05:46): > Happy Easter Aaron
Kevin Rue-Albrecht (09:06:01): > How’s it going over there?
2018-10-21
Aaron Lun (12:28:03): > If I provide a RDS file with the 1.3 million cell PCA and t-SNE coordinates, will someone be willing to do the clustering and make an app?
Charlotte Soneson (12:30:25): > I could run FlowSOM like I did here:https://github.com/Bioconductor/TENxBrainAnalysis/blob/master/flowsom.Rmd - Attachment (GitHub): Bioconductor/TENxBrainAnalysis > R scripts for analyzing the 1.3 million brain cell data set from 10X Genomics - Bioconductor/TENxBrainAnalysis
Aaron Lun (12:31:22): > I thought you concluded that flowSOM wasn’t that good for scRNA-seq data.
Charlotte Soneson (12:31:50): > It works fine, as long as you set the number of clusters high enough
Charlotte Soneson (12:32:25): > The problem is that it generates some tiny clusters so if you set 4 clusters when there are actually 4 “large” clusters it’ll probably chop off a few small clusters and merge some of the big ones
Charlotte Soneson (12:33:25): > Don’t know which other algorithm would be happy with 1.3M cells
Aaron Lun (12:36:41): > Hm. Graph building can be done reasonably quickly with approximate nearest neighbours, but the actual clustering via igraph is single-core and will shit itself.
Aaron Lun (13:35:41): > Anyway - poster update - another two panels finished - the first interactive component.
2018-10-22
Kevin Rue-Albrecht (07:17:47): > Packages have until next Monday to ” limit changes to “show-stopper” bugs and documentation improvements.” > I’ve spotted a few things in the basic vignette that could use an update. I’d like to update that in the next couple of days, but I think we could all have an extra look, to make the new “Bioc release” iSEE as self-consistent as possible
Federico Marini (07:38:49): > make an extra branch for this purpose and we can all have a look + solve it quickly?
Kevin Rue-Albrecht (18:43:59): > Done. Off to sleep now. Feel free to complain about the changes:wink:
Aaron Lun (18:44:39): > I will be up all night. Putting this on my todo.
Kevin Rue-Albrecht (18:45:19): > btw, the list of panel types was outdated, I’ve added the missing ones, but now the vignette misses a section about “Columns statistics tables”
Kevin Rue-Albrecht (18:45:45): > which can basically point to the “Row statistics tables” that’s already there
Kevin Rue-Albrecht (18:46:52): > catch up tomorrow - I’ve got an early start helping a colleague of my missus to install and get familiar with… iSEE:sunglasses:
Aaron Lun (18:47:58): > lol
Aaron Lun (18:54:20): > :party_parrot:
2018-10-23
Kevin Rue-Albrecht (05:34:33): > Building up the user base:grin:
Kevin Rue-Albrecht (05:36:43): > But god it’s so weird to use release 1.0.1 again…
Kevin Rue-Albrecht (05:36:55): > Looking forward to the bioc release
Kevin Rue-Albrecht (05:37:41): > “And you can also facet plots… oh wait you can’t”
Aaron Lun (16:03:10): > all 4 interactive panels are done for my poster! at last!
Aaron Lun (16:03:20): > Oh, there’s also a bug with row data plot colouring by sample name.
Kevin Rue-Albrecht (16:03:42): > uh? something that needs a last minute fix?
Kevin Rue-Albrecht (16:05:57): > I’m just going through your comments on the vignette now.
Aaron Lun (16:07:54): > I was going to go home - haven’t slept in >36 hours!
Kevin Rue-Albrecht (16:08:00): > ouch
Kevin Rue-Albrecht (16:08:05): > “was” ?
Aaron Lun (16:08:08): > But I guess I’ll pinpoint this for you.
Kevin Rue-Albrecht (16:08:13): > fire away
Kevin Rue-Albrecht (16:08:35): > or was that all?
Aaron Lun (16:10:32): > See attached for how to generate the app.
Aaron Lun (16:11:02): > Row data plot issue - File (R): slide_generator.R
Aaron Lun (16:11:11): > Now, change the colour to “sample name”.
Aaron Lun (16:11:20): > You’ll notice that the result does not respond to “assay” choices.
Aaron Lun (16:11:34): > We’re probably missing an observer somewhere iniSEE.Rwith respect to the sample name assay choice.
Aaron Lun (16:12:28): > Okay, going home now.
Kevin Rue-Albrecht (16:13:19): > alright, get some sleep, i’ll see what i can do
Aaron Lun (16:18:01): > Also, I need an idea for the last A4 panel of the poster.
Kevin Rue-Albrecht (16:18:19): > wanna share a current pic’ ?
Aaron Lun (16:18:22): > I have 4 intro panels; 4 iSEE layout panels, 4 interactive panels, 2 QR code panels, 1 “this is aaron” panel.
Kevin Rue-Albrecht (16:20:45): > how’bout a panel of iSEE visualizing dimensionality reduction results on similarity between the BiocViews of Bioconductor packages?:grimacing:
Kevin Rue-Albrecht (16:21:40): > given the number of BiocViews vs. packages, it should be a fairly sparse matrix too
Aaron Lun (16:22:04): > Need 1 more panel (have not done the Aaron’s face poster yet). - File (JPEG): 20181023_212018.jpg
Aaron Lun (16:22:10): > @Charlotte Sonesonand@Federico Marini, get your opinions in here as well.
Kevin Rue-Albrecht (16:23:44): > :dizzy_face::+1:
Federico Marini (16:32:53): > I’m just loving it
Federico Marini (16:33:14): > also because I had a similar poster abt 2.5 years ago
Federico Marini (16:33:36): > about pcaExplorer, where I had the panels covered up by a a5 page
Federico Marini (16:33:46): > and you could lift that up and reveal the content
Federico Marini (16:34:06): > so I am totally feeling it
Kevin Rue-Albrecht (16:35:03): > I’m curious how the linked panels work in this one. It seems it’s about opening the sort of shutters
Kevin Rue-Albrecht (16:35:51): > but I dig the modular structure of the poster ^^
Kevin Rue-Albrecht (16:36:27): > “you know what, I’m gonna showcase the custom panel this time ”
Kevin Rue-Albrecht (17:14:03): > alright. rowDataPlot bug fixed:tada:
Kevin Rue-Albrecht (17:14:08): > well spotted
Kevin Rue-Albrecht (17:17:44): > last one before version bump would be to add a short section about the colStatTable in the basic vignette. Only panel type missing.
Kevin Rue-Albrecht (17:18:13): > I’ll give it a fresh look tomorrow morning if no one gets to it first
Aaron Lun (17:26:57): > The linked panels are connected by a string. Opening one will force the other to close, thereby “updating” the selection on the plot.
Aaron Lun (17:27:09): > The next interactive panel has a wheel that you can turn to change the colour scheme.
Aaron Lun (17:27:19): > The final interactive panel has a tab that you can pull to get the code.
Aaron Lun (17:28:15): > Anyway, I still need some ideas for a final panel.
Federico Marini (18:29:43): > my knowledge of gimp is < 0
Federico Marini (18:29:50): > but this would be my draft
Federico Marini (18:30:03): - File (PDF): iSEE.pdf
Federico Marini (18:30:33): - File (PNG): iSEE.png
Federico Marini (18:30:51): > code for that:
Vince Carey (18:30:54): > maybe as a final panel you can have plans for the future
Vince Carey (18:31:11): > do you have any approach to recording the selections in a panel?
Federico Marini (18:31:29): > > library(ggplot2) > library(png) > library(grid) > library(hexSticker) > > ## Settings: > ## Bioconductor blue: #3792AD > ## Bioconductor green: #8ACA25 > > # in honor of the conference where it all began, #euroBioc2017 > col_border <- "#A3C1AD" # Cambridge Blue > col_bg <- "#FFFFFF" ## classic white > col_text <- "#000000" ## black > > ## col_border <- "#8ACA25" ## BioC green > > img_file <- ("./glasses_empty.png") > img_file <- ("./glasses_full_draft.png") > img <- readPNG(img_file) > # img_glasses <- rasterGrob(img, width = 1.0, x = 0.5, y = 0.6, > # interpolate = TRUE) > > sticker(img_file, > package="iSEE", > p_size = 9.3, > p_family = "Aller_Lt", > p_color = col_text, > s_x = 1, > s_y = 0.83, > s_width = 0.95, > # s_height = 1.1, > h_fill = col_bg, > h_color = col_border, > spotlight = FALSE, > url = "[www.bioconductor.org](http://www.bioconductor.org)", > u_color = col_border, > filename="iSEE.png", > ) >
Federico Marini (18:32:35): > do you mean the options of each panel Vince?
Vince Carey (18:32:49): > i can’t well parse the photo of the slides so i don’t really know what is being communicated
Aaron Lun (18:32:55): > Yes, in an indirect way. The selections themselves are not directly recorded, but the code required to define each selection is, so we can reproduce it easily in an R session.
Vince Carey (18:33:46): > hmm. i am thinking of each selection generating an element of a list of indices
Vince Carey (18:34:19): > do you have non-rectangular selections?
Aaron Lun (18:34:24): > Yes.
Aaron Lun (18:34:40): > The problem with a list of indices is that it gets pretty big for large numbers of samples/genes.
Aaron Lun (18:35:40): > In the context of an script, it is also less easy to document where this list comes from in downstream analyses, whereas providing users with the code to define the selection makes it much clearer - as the coordinates of the box or lasso can be directly examined.
Vince Carey (18:36:10): > sounds good
Aaron Lun (18:36:41): > I think I might go with future ideas.
Aaron Lun (18:36:54): > Of which I had a few but have since forgotten.
Aaron Lun (18:37:10): > If we had the stickers@Federico Marini, the last panel would have been a bag with “take some”.
Federico Marini (18:37:35): > that, or some draft of a custom panel?
Aaron Lun (18:37:42): > Or maybe “additional features” would be good as well.
Aaron Lun (18:37:54): > Yeah, some comments on customizability.
Federico Marini (18:37:57): > a la iSEE_custom repo
Federico Marini (18:38:02): > or again
Federico Marini (18:38:22): > “make your paper figures and deploy its interactive browser”
Aaron Lun (18:38:31): > custom set-up
Aaron Lun (18:38:34): > custom panels
Aaron Lun (18:38:38): > custom tours
Aaron Lun (18:38:47): > Yep, that’s enough to fill an A4 page of text.
Vince Carey (18:39:07): > figures of merit for different projections?
Aaron Lun (18:39:21): > huh?
Aaron Lun (18:39:36): > you mean t-SNE and the like?
Vince Carey (18:39:37): > umap is touted for capturing global properties, tsne for local … do we have any basis for preference?
Aaron Lun (18:39:59): > I haven’t played around with umap enough to really say.
Aaron Lun (18:40:16): > Anecdotally… it all looks the same to me.
Aaron Lun (18:40:33): > I was of the understanding that the main difference was speed.
Vince Carey (18:40:40): > maybe it is scope creep but if the tool can do this, help people choose a more informative projection, that could be useful.
Vince Carey (18:40:54): > i have been doing comparisons.
Vince Carey (18:41:07): > the claim that it is just a faster tsne is not correct
Aaron Lun (18:41:26): > We can definitely show all projections side-by-side and link one to each other to see how they interact
Aaron Lun (18:41:41): > in terms of which points in the t-SNE correspond to which points in UMAP/PCA/etc.
Aaron Lun (18:42:18): > Just stuff it into the reduced dimensions of an SCE and fire it up.
Vince Carey (18:42:24): > yes, but how can we go beyond one selection at a time? can you freeze selections? sorry i have not spent too much time with iSEE … it seemed to me that when you did a second selection the first was cleared
Aaron Lun (18:43:01): > Nope, can’t freeze selections.
Aaron Lun (18:43:26): > Not easily done while preserving the sanity of the app.
Vince Carey (18:43:28): > “paint” the dots – should be feasible
Aaron Lun (18:44:17): > The problem is that you need additional infrastructure to remember the brushing history of each panel.
Aaron Lun (18:44:39): > And then extra UI elements to allow users to select particular brushes/lassos.
Vince Carey (18:44:54): > will discuss tomorrow.
2018-10-24
Kevin Rue-Albrecht (03:24:48): > Interesting discussion that I missed last night. > For multiple simultaneous selections the current workaround is to add the set of desired linked panels multiple times in the UI, and have a different selection in each set.
Kevin Rue-Albrecht (03:27:50): > Apps can be preconfigured to launch with the additional panels (any number of any type), even reordered for instance to put tSNE and UMAP side by side, and with the different selections and links already in place. (It is fairly tedious to open the default app and add multiple panels with the UI top left button each time)
Kevin Rue-Albrecht (03:32:46): > The « cytof tour » is a great example of setting up multiple copies of a panel type, as well as selections and links. While it focuses on feature assay plots, the same logic applies to reduced dimensions plotshttps://github.com/LTLA/iSEE2018/tree/master/tours/cytof - Attachment (GitHub): LTLA/iSEE2018 > Code and manuscript files for the iSEE paper. Contribute to LTLA/iSEE2018 development by creating an account on GitHub.
Kevin Rue-Albrecht (04:48:08): > Alright, a screenshot (and a tour) being worth a thousand words, please check out this custom app for an example of “custom multiple redDimPlots” panel:https://github.com/kevinrue/iSEE_custom/tree/Multireddim/plot_multiRedDim - File (PNG): Pasted image at 2018-10-24, 9:47 AM
Kevin Rue-Albrecht (04:49:12): > I couldn’t installumapfor some OS issues, but if you add it to the SCE, it’ll get picked up too. I only used PCA and tSNE for the proof of concept here
Kevin Rue-Albrecht (04:52:31): > This is actually much simpler than the solution that I described earlier. You only need to set up one custom panel and one selection link (not multiple clones of linked plots with a different selection in each)
Kevin Rue-Albrecht (04:54:22): > (and incidentally, this multiplotting capability is one of the many reasons why custom panels have to be end-points, i.e. can’t transmit further)
Kevin Rue-Albrecht (04:56:21): > iSEE, proud partner of comparative dimens’omics:sweat_smile:
Federico Marini (06:37:53): > neat, kevin, neat
Federico Marini (06:37:57): > :slightly_smiling_face:
Federico Marini (06:38:22) (in thread): > we all miss the correlomics era
Federico Marini (06:38:42) (in thread): > who cares about robust methods
Kevin Rue-Albrecht (06:39:15) (in thread): > I like those days when I wake up inspired:slightly_smiling_face:
Federico Marini (06:39:29) (in thread): > when you have singletons and the logFC is the ratio, and the significance is tightly related to the relevance of a gene in the project
Kevin Rue-Albrecht (06:40:00) (in thread): > hahaha I know exactly what you’re talking about
Kevin Rue-Albrecht (06:42:49): > Before merging this custom panel branch to the master iSEE_custom branch, I’ll probably add apattern=custom argument where users can somewhat control the redDimNames shown (rather than all of them). This way one could have e.g. a set ofTSNE_*for a range of perplexities
Kevin Rue-Albrecht (10:47:50): > Just messing around with Powerpoint, I got to this concept of a logo. Could throw in a few points and shapes to suggest plots and tables, and maybe some links between the colored panels? (a bit like Aaron designed Figure 1B) > Otherwise, the glasses and example plots do get the point across. I’m just brainstorming. - File (PDF): SimpleDraft.pdf
Kevin Rue-Albrecht (13:14:43): > On a separate note, anything else for the last PR?
Aaron Lun (13:16:57): > Nope.
Kevin Rue-Albrecht (13:53:55): > Ok. then I’ll just add that tiny missing section about colStatTable in the basic vignette this evening and bump up the version
Kevin Rue-Albrecht (14:10:05): > @Aaron Lun(and everyone), can you please have a look athttps://github.com/csoneson/iSEE/commit/4c7e187026e42af0fb237a34f3859bb12f4cc481? - Attachment (GitHub): add a section for Column statistics tables, and fix occurences of ’ge… · csoneson/iSEE@4c7e187 > …nes’ by ‘features’
Aaron Lun (14:11:44): > Oh, and I do remember something to do. If colouring by feature name in row-based plots or by sample name in column-based plots, make the highlighted point stand out more. It’s really hard to see - probably need to increase the size by several fold relative to the other points.
Kevin Rue-Albrecht (14:31:20): > ah you mean color+ sizefor the selected points?
Aaron Lun (14:38:05): > yes.
Kevin Rue-Albrecht (16:03:05): > @Aaron LunI agree with the benefit of expanding the size of a single point in those plots, but that would require a bit more rewiring than I’d like to do two days before the “show-stopper” deadline > Currently, the size is coded outside the aesthetics, which means the same size is applied to all points (e.g. plotting line 1358 for row plots). > > sprintf( > "geom_point(aes(x=X, y=Y), data=subset(plot.data, ColorBy=='TRUE'), col=%s, size=%s, alpha=1) +", > > Moving it into the aesthetics would then also require adding (in a separate part of the code) an extra column “Selected” to the plotting data.frame. > Then we’d have to add a switch between the two commands, and unit test the two behaviour (x2 row/column plot types)
Kevin Rue-Albrecht (16:04:17): > I’ll give it a brief try, but if I can’t get in for tomorrow, I open and merge the PR without
Kevin Rue-Albrecht (16:12:16): > Actually, I think i just misdiagnosed the issue. It’s almost done.
Kevin Rue-Albrecht (16:14:45): > But it made me run into a last minute bug, can someone replicate the very simple following action: > > colouring by sample name in column-based plots > Do you also seetwopoints highlighted, while there should only be one? > The two points are always at the same Y coordinate for me, which suggests that it’s actually the same point being rendered twice, I just haven’t figured out where the issue is yet.
Kevin Rue-Albrecht (16:18:20): > oh.. I see (thank god for the code tracker)
Kevin Rue-Albrecht (16:22:10): > Not an obvious one, but there are two layers: one for the jittered points and an extra one purely for the selected point. Both apply color to the selected point but the latter does it outside the aesthetic. Not sure which one to “fix”. Perhaps I can subset the first one toColorBy=FALSE > > ggplot() + > geom_violin(aes(x = X, y = Y, group = GroupBy), alpha = 0.2, data=plot.data, scale = 'width', width = 0.8) + > geom_point(aes(y = Y, color = ColorBy, x = jitteredX), alpha = 1, plot.data, size=1) + > labs(x = "", y = "NREADS", color = "SRR2140028", title = "NREADS ") + > coord_cartesian(ylim = range(plot.data$Y, na.rm=TRUE), expand = TRUE) + > scale_color_manual(values=c(`FALSE`='black', `TRUE`="red"), drop=FALSE) + > geom_point(aes(x=X, y=Y), data=subset(plot.data, ColorBy=='TRUE'), col="red", size=5*1, alpha=0.8) + > scale_x_discrete(drop = FALSE) + > theme_bw() + > theme(legend.position = 'bottom', legend.text=element_text(size=9), > legend.title=element_text(size=11), legend.box = 'vertical', > axis.text.x = element_text(angle=90, size=10, hjust=1, vjust=0.5), > axis.text.y=element_text(size=10), > axis.title=element_text(size=12), title=element_text(size=12)) >
Federico Marini (16:24:15): > <!channel>: after couple of fiddling rounds, have a look athttps://github.com/federicomarini/BiocStickers/tree/master/iSEE - Attachment (GitHub): federicomarini/BiocStickers > Stickers for some Bioconductor packages - feel free to contribute and/or modify. - federicomarini/BiocStickers
Federico Marini (16:24:48): > (pdf version looks better, btw ->https://github.com/federicomarini/BiocStickers/blob/master/iSEE/iSEE.pdf) - Attachment (GitHub): federicomarini/BiocStickers > Stickers for some Bioconductor packages - feel free to contribute and/or modify. - federicomarini/BiocStickers
Kevin Rue-Albrecht (16:25:21): > it’s cool:slightly_smiling_face:
Federico Marini (16:28:09): > Kev, for the ordering you need the pdf right?
Federico Marini (16:28:13): > or png?
Kevin Rue-Albrecht (16:28:36): > eh.. can’t remember but I imagine they’d be happier with a PDF
Kevin Rue-Albrecht (16:30:54): > Just to illustrate the issue, the redDim is fine, but the violin plot has two red points at the same Y - File (PNG): Screenshot 2018-10-24 21.30.00.png
Kevin Rue-Albrecht (16:31:27): > and the bug applies to square plots too, for the record (jittered plots, basically)
Kevin Rue-Albrecht (16:32:26): > too much? - File (PNG): Screenshot 2018-10-24 21.32.12.png
Kevin Rue-Albrecht (16:37:57): > @Federico Marinione thing about the sticker: i’d reduce a bit the glasses; I would leave more margin from the border of the sticker
Federico Marini (16:38:35): > that’S codable, so it is an easy change:smile:
Kevin Rue-Albrecht (16:49:26): > alright Aaron’s wish and unexpected bug both solved - File (PNG): Screenshot 2018-10-24 21.48.40.png
Kevin Rue-Albrecht (16:49:50): > I feel the broken unit test now .. check’ing
Kevin Rue-Albrecht (16:50:50): > aaaargh.. i almost forgot to apply the same fix to the square plots! back to the mine
Kevin Rue-Albrecht (16:53:47): > FYI, that’s an expansion factor of 5 between the selected point and the others
Kevin Rue-Albrecht (16:55:28): > Alright, can’t resist one last screenshot with the whole family - File (PNG): Screenshot 2018-10-24 21.54.58.png
Kevin Rue-Albrecht (16:56:32): > note that incidentally, the selected data point is ‘normally’ plotted for violins and squares, that is, at the intersection of the axes
Kevin Rue-Albrecht (17:01:34): > Alright the usual one unit test failed. Trying again.
Kevin Rue-Albrecht (17:02:23): > I’d really appreciate if anyone could pull the branchvignettesonce I push back, as Travis has been fairly confused recently due to the deprecation of BiocInstaller
Kevin Rue-Albrecht (17:13:06): > alright, if anyone on<!channel>could pull branchvignettesrunR CMD checkand confirm that everything’s fine, I merge tomasterand push upstream ASAP
Aaron Lun (17:21:49): > Hold on. Why aren’t the violin and square plots using the jittered coordinates? They should have been doing so before.
Kevin Rue-Albrecht (17:24:12): > ?
Kevin Rue-Albrecht (17:24:28): > you mean for the selected point?
Kevin Rue-Albrecht (17:26:46): > because if you look at the code chunk that I pasted above, yes, they were, but you’ll also notice that the selection adds a second layer with the selected point colored (again) that is not jittered
Kevin Rue-Albrecht (17:27:03): > i decided to the keep the second for the color, which makes sense when you have a single point as the jitter doesn’t mean anything anyway
Kevin Rue-Albrecht (17:30:01): > oh.. and I just realised that the colorpicker isn’t applied -_- the selected point is set to"red"no matter what
Aaron Lun (17:31:42) (in thread): > How is this possible? There should only be one source of coordinate information.
Aaron Lun (17:31:59) (in thread): > Is the code usingjitteredXandjitteredYin theaes?
Kevin Rue-Albrecht (17:33:20) (in thread): > > ## The following list of commands will generate the plots created in iSEE > ## Copy them into a script or an R session containing your SingleCellExperiment. > ## All commands below refer to your SingleCellExperiment object as `se`. > > se <- sce > colData(se)[,"sizeFactors(se)"] <- sizeFactors(se) > colormap <- ExperimentColorMap() > colormap <- synchronizeAssays(colormap, se) > all_coordinates <- list() > custom_data_fun <- NULL > custom_stat_fun <- NULL > > ################################################################################ > ## Column data plot 1 > ################################################################################ > > plot.data <- data.frame(Y = colData(se)[,"NREADS"], row.names=colnames(se)); > plot.data$X <- factor(character(ncol(se))) > plot.data$ColorBy <- logical(nrow(plot.data)); > plot.data[1L, 'ColorBy'] <- TRUE; > plot.data$ColorBy <- factor(plot.data$ColorBy); > plot.data <- subset(plot.data, (X) & (Y)); > > # Saving data for transmission > all_coordinates[['colDataPlot1']] <- plot.data > > # Setting up plot coordinates > plot.data$GroupBy <- plot.data$X; > set.seed(100); > plot.data$jitteredX <- iSEE::jitterViolinPoints(plot.data$X, plot.data$Y, > width=0.4, varwidth=FALSE, adjust=1, > method='quasirandom', nbins=NULL); > > # Creating the plot > ggplot() + > geom_violin(aes(x = X, y = Y, group = GroupBy), alpha = 0.2, data=plot.data, scale = 'width', width = 0.8) + > geom_point(aes(y = Y, color = ColorBy, x = jitteredX), alpha = 1, plot.data, size=1) + > labs(x = "", y = "NREADS", color = "SRR2140028", title = "NREADS ") + > coord_cartesian(ylim = range(plot.data$Y, na.rm=TRUE), expand = TRUE) + > scale_color_manual(values=c(`FALSE`='black', `TRUE`="red"), drop=FALSE) + > geom_point(aes(x=X, y=Y), data=subset(plot.data, ColorBy=='TRUE'), col="red", size=5*1, alpha=0.8) + > scale_x_discrete(drop = FALSE) + > theme_bw() + > theme(legend.position = 'bottom', legend.text=element_text(size=9), > legend.title=element_text(size=11), legend.box = 'vertical', > axis.text.x = element_text(angle=90, size=10, hjust=1, vjust=0.5), > axis.text.y=element_text(size=10), > axis.title=element_text(size=12), title=element_text(size=12)) >
Kevin Rue-Albrecht (17:34:29) (in thread): > there’s only one source of information indeed, but two layers use it
Kevin Rue-Albrecht (17:34:58) (in thread): > we did that thing where one layer shows the unselected points, and vice versa
Aaron Lun (17:38:39) (in thread): > shouldn’t the secondgeom_pointbe usingx=jitteredX?
Kevin Rue-Albrecht (17:42:56) (in thread): > well.. itcould, but why would it?
Aaron Lun (17:44:06) (in thread): > Because then you’d be colouring the point at the same location that it is distributed in the violin.
Kevin Rue-Albrecht (17:44:24): > I’ve almost fixed the observer to the colorpicker for colorBySampleName and colorByFeatName
Aaron Lun (17:44:42) (in thread): > Otherwise: imagine you have a square or violin with only one point. This scheme will showanotherred point rather than overlaying on top of the single point.
Kevin Rue-Albrecht (17:44:49) (in thread): > yes and no, when we’re talking about a single point
Kevin Rue-Albrecht (17:45:40) (in thread): > well it shouldn’t show theuncoloredjittered point
Kevin Rue-Albrecht (17:46:11) (in thread): > we should havedata=subset(plot.data, !ColorBy)for the jittered layer
Kevin Rue-Albrecht (17:46:33) (in thread): > (I’m talking about adding thesubsethere)
Aaron Lun (17:46:49) (in thread): > I don’t understand and I can’t check it out on this computer.
Aaron Lun (17:47:26) (in thread): > But inspection of the code suggests the double plotting problem should go away if you usejitteredXin the secondgeom_pointcall.
Kevin Rue-Albrecht (17:47:57) (in thread): > we’ve done that when we’re coloring multiple points , one layer plotsdata=subset(plot.data, ColorBy)with the selected color, and a second layer plotsdata=subset(plot.data, !ColorBy)in black
Kevin Rue-Albrecht (17:48:50) (in thread): > I don’t know why we haven’t done it here, although it only becomes noticeable when you have few enough points to notice that there’s an extra one
Aaron Lun (17:50:02) (in thread): > I’m understanding even less. 1) Is there a problem? 2) What’s wrong with settingx=jitteredXin the secondgeom_pointcall?
Kevin Rue-Albrecht (17:51:14): > i’ve just pushed a fix for missing observers on the colorpicker always related to the same topic: color by sample name and by feature name
Kevin Rue-Albrecht (17:51:25): > how did we miss those all this time?
Kevin Rue-Albrecht (17:51:37) (in thread): > let me check
Aaron Lun (17:51:40): > Because no one ever user it.
Kevin Rue-Albrecht (17:53:32) (in thread): > I don’t think there’s any problem, of the two possible choices given by the duplication issue, I just decided to use the non-jittered. But I can see now how it makes more sense to use the jittered: at least the extra point is on top of the smaller uncolored one (even though I would argue that the latter shouldn’t even be there, but I don’t want to “fix” that now)
Aaron Lun (17:56:51) (in thread): > Well, you could modify it to not even bother settingColourByin the firstgeom_pointwhen colouring by sample name, and rely fully on the secondgeom_pointto set the colour.
Kevin Rue-Albrecht (17:57:08) (in thread): > ahhhhh no, I got it: the reason why it’s not usingjitteredfor the colored layer is that it’s done in.add_color_to_column_plot, which does not know which kind of plot it’s in. It only takescolorby, param_choicesfor arguments
Aaron Lun (17:57:31) (in thread): > Ah shit.
Aaron Lun (17:58:11) (in thread): > Can we just add a size modifying command rather than a newgeom_point?
Aaron Lun (17:58:37) (in thread): > Or even just piggy-back off the existingaesrather than re-specifying it.
Kevin Rue-Albrecht (17:59:24) (in thread): > that was my first intuition, but it would require a family ofadd_size_to_*_plotanddefine_size_for_*_plot…
Kevin Rue-Albrecht (18:00:53) (in thread): > let me see if I can just use thatsubsetapproach to simply not show the selected point in the jittered uncolored layer..
Aaron Lun (18:01:41) (in thread): > You wouldn’t need a family. Just do it insideadd_color_to_*_plot.
Aaron Lun (18:02:26) (in thread): > And in any case, I thought ggplot would use anaesfrom an earlier command, so why do you need to respecify it?
Kevin Rue-Albrecht (18:04:38) (in thread): > Only anaesdeclared in the mainsggplotcall can be inherited by layers, I believe. Then each layer can override it independently of the others, as far as I know. But that’s not really the issue here I think.
Aaron Lun (18:05:45) (in thread): > The easiest way is to modifyadd_color_*to accept anxargument (defaulting to"x").
Kevin Rue-Albrecht (18:17:56) (in thread): > I’m not sure what “x” here. But indeed, conceptually color and size should both se linked to the same aesthetic, as this is how we describe the entire use case
Aaron Lun (18:20:01) (in thread): > theaes``x="x"
Kevin Rue-Albrecht (18:24:43) (in thread): > ok i think i get you know, i should be able to change it there then
Kevin Rue-Albrecht (18:34:42) (in thread): > testing…
Kevin Rue-Albrecht (18:37:06) (in thread): > ok it shows where it should, but now I have an extra legend. I know how to do that, but I just hope I can put the command in the right place here - File (PNG): Screenshot 2018-10-24 23.35.55.png
Aaron Lun (18:38:03) (in thread): > Kill the TRUE/FALSE completely, I think. Now that you have a secondgeom_point, there’s no need to even haveColorByin the firstgeom_point.
Aaron Lun (18:38:22) (in thread): > Thescale_colorcall would also go.
Kevin Rue-Albrecht (18:42:36) (in thread): > i’m getting tired i think: I can kill the TRUE/FALSE legend easily, but what then isscale_color?
Kevin Rue-Albrecht (18:43:11) (in thread): > sounds like the same thing to me
Kevin Rue-Albrecht (18:44:44) (in thread): > ok look AFA i’m concerned it’s fixed for colDataPlots
Kevin Rue-Albrecht (18:45:29) (in thread): > it’s jittered, it’s colored, the black point is hidden behind the colored bigger point
Kevin Rue-Albrecht (18:47:22) (in thread): > now i just need to apply the same for rowData plots, and square plots
Aaron Lun (18:51:45) (in thread): > scale_color_gradient_in that case is unnecessary.
Aaron Lun (18:52:03) (in thread): > Consider what happens without your secondgeom_pointcall.
Kevin Rue-Albrecht (18:52:41) (in thread): > where do you see a scale_gradient?
Aaron Lun (18:52:58) (in thread): > Fuck this computer sucks
Kevin Rue-Albrecht (18:53:03) (in thread): > ah scale_color_gradient sorry
Aaron Lun (18:53:06) (in thread): > trying to writesale_color_manual.
Aaron Lun (18:53:19) (in thread): > well you know what I mean.
Aaron Lun (18:53:31) (in thread): > Typing faster than the words show up on this machine.
Kevin Rue-Albrecht (18:53:32) (in thread): > ah right
Kevin Rue-Albrecht (18:54:00) (in thread): > well, I still need the scale to control the color chosen by the user, even if we don’t show a legend
Aaron Lun (18:54:24) (in thread): > I thought that was specified ingeom_point.
Aaron Lun (18:54:51) (in thread): > The first one.
Kevin Rue-Albrecht (18:55:42) (in thread): > geom_pointdefines the binding of columns to aesthetics (e.g. color), but the layer doesn’t control the color scale, that’s the job of thescale_*commands
Aaron Lun (18:56:10) (in thread): > But ifscale_*isn’t specified, what does it default to? Black? Nothing?
Kevin Rue-Albrecht (18:56:23) (in thread): > damn, I almost had it right, but I mixes the jittering direction of the square plots I think
Kevin Rue-Albrecht (18:56:33) (in thread): > worse, it goes to the ggplot2 default color scale
Kevin Rue-Albrecht (18:56:57) (in thread): > like pale blue and red
Aaron Lun (18:58:08) (in thread): > Well, we definitely don’t need to specify red inscale_color_manual. Suggest setting all points to a grey =, to make the selected point stand out more.
Kevin Rue-Albrecht (18:59:29) (in thread): > I can be so stupid sometimes… it wasn’t the direction: when I set upjitteredXforadd_points_*, I didn’t think a second of addingjitteredYfor the square plots.
Kevin Rue-Albrecht (19:00:01) (in thread): > Can’t users define the base color of points?
Aaron Lun (19:00:18) (in thread): > Not with the colour-by set to something else.
Kevin Rue-Albrecht (19:00:18) (in thread): > I’m about to check again, hang on
Kevin Rue-Albrecht (19:00:22) (in thread): > ah
Kevin Rue-Albrecht (19:00:34) (in thread): > well, that’ll be for the next devel branch
Aaron Lun (19:01:13) (in thread): > Now, the real fun begins when someone tries to color on select as well.
Aaron Lun (19:01:36) (in thread): > Because we over-plot there as well, I think.
Kevin Rue-Albrecht (19:01:42) (in thread): > I’ll be honest: I’m reading but not processing anymore ^^
Kevin Rue-Albrecht (19:01:55) (in thread): > Just wanna get to sleep xD
Kevin Rue-Albrecht (19:02:30) (in thread): > I think I got it this time. I’ll push in 2 min hopefully
Kevin Rue-Albrecht (19:06:00) (in thread): > argh. It’s ok for all shapes of colData plots but the point doesn’t get highlighted anymore in the rowData plots
Kevin Rue-Albrecht (19:08:37) (in thread): > nah i was just doing something stupid with a restriction effect
Kevin Rue-Albrecht (19:09:59) (in thread): > Pushing. If people can test it early tomorrow, we can merge and push upstream
Kevin Rue-Albrecht (19:20:07): > Yep. My job here is done. Let’s merge tomorrow please.
Kevin Rue-Albrecht (19:20:11): > g’night
2018-10-25
Kevin Rue-Albrecht (04:11:56): > @Federico Marini@Charlotte Sonesonlet’s bring back the conversation(s) here
Federico Marini (04:12:06): > yo
Kevin Rue-Albrecht (04:12:08): > Charlotte had an issue with thevignettesbranch > > ok, I don’t know if something went wrong (I had to explicitly pull down thevignettesbranch to be able to check it out, which I have never needed to to before)
Federico Marini (04:12:31): > the branch issue is solvable if I do it in the rstudio gui
Federico Marini (04:12:37): > weird, yet, …
Charlotte Soneson (04:13:18): > oh, ok. I did a regulargit pullin the RStudio terminal, and it showed me that it pulled downvignettes, but there was no way I could check it out
Charlotte Soneson (04:13:24): > It worked when I pulled it manually
Kevin Rue-Albrecht (04:13:45): > ok I do have the same unit test error as Charlotte reported to me: > > 1. Failure: .define_colorby_for_column_plot handles sample selection (@test_plotting.R#988) >
Kevin Rue-Albrecht (04:14:00): > Forgot to unit test before the last commit last night
Kevin Rue-Albrecht (04:14:53): > I’m gonna have to dash to work now, meeting in 45min and it takes me 30 to get there. If someone wants to try their hands on the unit test, otherwise I’ll pick it up ASAP
Kevin Rue-Albrecht (04:15:45): > I’ll give it a 5 min try now:smiling_imp:
Kevin Rue-Albrecht (04:17:52): > Got it.checking again. Thanks@Charlotte Sonesonfor runningcheckwhen I didn’t:sweat_smile:
Charlotte Soneson (04:18:22): > :+1:running again now
Kevin Rue-Albrecht (04:18:49): > Ok, I’ve pushed while checking, because i’m on 7% battery and the charger’s at work
Federico Marini (04:19:04): > gogogo
Kevin Rue-Albrecht (04:20:32): > daaaaamn, I forgot to roxygenize
Kevin Rue-Albrecht (04:20:49): > still this stupid warning about undocumented arguments
Kevin Rue-Albrecht (04:21:07): > alright, some does that while I go to work:sweat_smile:
Federico Marini (04:21:15): > go, we can handle it
Federico Marini (04:22:55): > plus, it is an internal, and as far as I can remember, we do not track them, do we?
Kevin Rue-Albrecht (04:23:25): > the bioc build shouldn’t spot them, and we don’t push them indeed
Federico Marini (04:23:41): > then it’s good
Charlotte Soneson (04:24:08): > I still get errors in the unit tests after pulling: > > Error in x[[method]](...) : attempt to apply non-function > Calls: <Anonymous> ... <Anonymous> -> o_apply -> lapply -> FUN -> <Anonymous> >
Kevin Rue-Albrecht (04:24:55): > don’t know why, but it seems the unit test still failed with the same error on my side. someone just look at it calmly I really gotta go. I’ll be on my phone though
Kevin Rue-Albrecht (04:27:30): > Weirdly for me, it failed with the same error message as before, line 988
Kevin Rue-Albrecht (04:28:08): > But the test passes when I ran it interactively before pushing. I don’t get it
Charlotte Soneson (04:29:40): > yes, stepping throughtest_plotting.R, everything is fine
Charlotte Soneson (04:29:47): > also the other tests, it seems
Kevin Rue-Albrecht (04:30:05): > Basically the set of colouring commands changed, so to fix the unit test, one only needs to run the content of the test, ’cat(deparse(‘ the new command, and paste it as expected value
Kevin Rue-Albrecht (04:30:46): > Ah ok
Kevin Rue-Albrecht (04:31:29): > Btw have you updated all packages to the latest version?
Charlotte Soneson (04:31:40): > let me check
Charlotte Soneson (04:32:11): > updating now, let’s see if it helps
Kevin Rue-Albrecht (04:33:35): > My devtools check looks nicer anyway:innocent:
Charlotte Soneson (04:50:53): > ok, tests seem to be running fine under R CMD check. I just have the documentation warnings.
Kevin Rue-Albrecht (04:52:22): > .. phone battery died too on the way ..
Kevin Rue-Albrecht (04:52:47): > alright, can go roxygenize and push?
Charlotte Soneson (04:53:25): > @Federico Marini, are things working for you?
Charlotte Soneson (04:54:07): > I still get an error if I try to run only the tests with devtools for some reason.
Federico Marini (04:54:20): > also getting the test not running in my normal ops
Kevin Rue-Albrecht (04:56:51): > Charlotte makes me wonder whether I saw something about devtools’s check recently on a mailing list
Kevin Rue-Albrecht (04:58:01): > not that it matters much, as Bioconductor uses the ‘original’R CMD check
Federico Marini (05:05:17): > well, I also normally use devtools check
Federico Marini (05:05:32): > but now I changed it to regular R CMD check and that goes smooth
Charlotte Soneson (05:06:23): > ok, let’s stick with that then:wink:
Kevin Rue-Albrecht (05:10:37): > merge to master when ready, and i’ll push upstream:slightly_smiling_face:
Federico Marini (05:12:54): > unrelated to devel pure, but sticker-wise: I got a new draft with a for-sure-cc0 image
Federico Marini (05:13:18): > just out of ethics, I felt lil bad not knowing how the one we had was released
Federico Marini (05:34:27): > (updated the main README.md for the sticker collection -> spot us there:smile:https://github.com/federicomarini/BiocStickers) - Attachment (GitHub): federicomarini/BiocStickers > Stickers for some Bioconductor packages - feel free to contribute and/or modify. - federicomarini/BiocStickers
Charlotte Soneson (05:44:38): > Ok, with regular R CMD CHECK I get no warnings. Nothing to push since the internal docs don’t go on GH. Let me know if I should mergevignettesor if anyone wants to check anything else.
Federico Marini (05:45:04): > looks fine to me as well
Federico Marini (05:45:33): > if we agree on the sticker it can go into the readme, but that is not so hurry-ish
Charlotte Soneson (05:45:55): > I like the sticker - but the tSNE looks very pale(?)
Federico Marini (05:48:31): > I can increase the alpha
Charlotte Soneson (05:52:53): > I’d probably prefer a bit less pale, to make sure it’s visible on the sticker when printed (it should stand out from the white background)
Federico Marini (06:02:40): > when I was live in the app it was all more crowded
Federico Marini (06:02:50): > but yes, now wee have more space
Charlotte Soneson (06:03:21): > @Kevin Rue-Albrechtwhat’s actually going on with Travis? Btw, I couldn’t merge either with the current settings - I just edited the branch protection rule to allow “administrators” to do it - I don’t know if that changed anything for you too or just for me:slightly_smiling_face:
Kevin Rue-Albrecht (06:17:43): > indeed, i just roxygenized and it’s not showing me anything new to push
Kevin Rue-Albrecht (06:24:21): > how do you guys getR CMD checkto run again? because I get one of those typical errors: > > * checking for file './DESCRIPTION' ... ERROR > Required fields missing or empty: > 'Author' 'Maintainer' >
Federico Marini (06:25:19): > it is complaining because we are doing with Authors@R
Federico Marini (06:25:30): > you mean now from command line, right?
Kevin Rue-Albrecht (06:26:29): > yep
Kevin Rue-Albrecht (06:26:37): > how did you runR CMD checkwithout raising this error then?
Kevin Rue-Albrecht (06:28:22) (in thread): > Travis is choking on devtools::install of the error: > > Error: (converted from warning) dependencies 'rnaturalearthhires', 'loomR', 'lqa', 'spDataLarge', 'adehabitat', 'rkward', 'glmmADMB', 'Rcampdf', 'tm.lexicon.GeneralInquirer', 'R2Cuba', 'FRB', 'cacheSweave', 'R2wd', 'codetoolsBioC', 'ActuDistns', 'RDCOMClient', 'kmndirs', 'corpus.JSS.papers', 'corpus.useR.2008.abstracts' are not available > > Not sure why
Kevin Rue-Albrecht (06:30:54): > was it this way?https://github.com/hadley/gtable/issues/52 - Attachment (GitHub): Maintainer and Author field missing from Description File · Issue #52 · hadley/gtable > Hi, I am trying to use your code base to run a check for gtable installed on R on Ubuntu 15.04. I get following error : R CMD check gtable using log directory ’/home/jenkins/workspace/Rlang_gtable_…
Aaron Lun (06:31:54): > build first.
Kevin Rue-Albrecht (06:32:18): > ok, that’s what Hadley wrote too:slightly_smiling_face:
Kevin Rue-Albrecht (07:03:27): > I got build and check to pass on my side. I think we’re good to go.
Aaron Lun (07:03:49): > give me some time after lunch.
Kevin Rue-Albrecht (07:04:22): > ok. I’ll give you time to watch the youtube video linked in the PR message too:slightly_smiling_face:
Aaron Lun (08:21:48): > See my simplifications.
Federico Marini (09:00:20): > @Charlotte Sonesonhere’s the new version, not so pale anymore
Federico Marini (09:00:34): - File (PNG): iSEE.png
Charlotte Soneson (09:01:11): > yeah, I like this better
Federico Marini (09:01:21): > me too
Federico Marini (09:01:35): > I did not put if fully opaque to guarantee some shades
Aaron Lun (09:03:08): > Make the reddim a bit larger to fill it up a bit.
Aaron Lun (09:03:23): > remember, it’s going to be even smaller when printed.
Aaron Lun (09:03:34): > And reduce the number of violins so that each violin is visible.
Federico Marini (09:04:15): > huh? they are all in there
Federico Marini (09:04:21): > or do you want them bigger?
Federico Marini (09:04:46): > (and I wanted to leave all the groups in, being all of them TCGA samples)
Aaron Lun (09:04:53): > “visible” in terms of “easy to see”.
Aaron Lun (09:04:57): > Yes, I would like them bigger.
Aaron Lun (09:05:20): > I’m looking at a few stickers I have now. It will be much harder to see both of these things when printed.
Federico Marini (09:05:58): > height is 5 cm, yep
Kevin Rue-Albrecht (09:07:07): > For example, check out the stickers of limma, edgeR, or EGSEA to get an idea for the size of data points on a sticker
Federico Marini (09:15:53): > I made it lil bigger
Federico Marini (09:16:08): > https://github.com/federicomarini/BiocStickers/tree/master/iSEEis the whole material - Attachment (GitHub): federicomarini/BiocStickers > Stickers for some Bioconductor packages - feel free to contribute and/or modify. - federicomarini/BiocStickers
Federico Marini (09:16:17): > gotta go and pick up the lil lady
Federico Marini (09:16:52): > if you want to try out something else, be my guest(s):wink:
Federico Marini (09:17:05): > catch you later:wave:
Kevin Rue-Albrecht (09:49:08): > @Aaron Lunyour update seems to be working for me
Kevin Rue-Albrecht (09:49:27): > i’ve tried column and row based plots, violin and square jitters
Kevin Rue-Albrecht (10:26:14): > @Charlotte Sonesonshall we merge?
Charlotte Soneson (10:27:32): > yep.
Charlotte Soneson (10:28:02): > ok, merged
Kevin Rue-Albrecht (10:28:32): > I’m just on my way back from the post office. I’ll push upstream when I get to my desk
Kevin Rue-Albrecht (10:29:04): > @Aaron Lunany famous last words?
Aaron Lun (10:29:28): > Flying has never been safer!
Kevin Rue-Albrecht (10:30:40): > Good for me
Kevin Rue-Albrecht (10:39:46): > Pushed upstream. Hurray
Charlotte Soneson (10:40:25): > :tada:
Federico Marini (10:42:27): > happy pre-release everyone:smile:
Federico Marini (11:24:09): > any of you with experience in putting a visitor counter on a shiny app website?
Kevin Rue-Albrecht (11:31:22): > not that I remember
Kevin Rue-Albrecht (11:31:27): > yo mean onshinyapps.io?
Kevin Rue-Albrecht (11:31:55): > or on a locally hosted shiny app?
Federico Marini (11:33:20): > locally hosted
Kevin Rue-Albrecht (11:35:50): > you could ask the bioc core team on the general or random channels: how they count downloads for each package. I guess you need to set the webserver to +1 a counter in some kind of database (sqlite would probably do in this case) every time it serves the page
Federico Marini (14:37:07): > seems possible fromhttps://shiny.rstudio.com/articles/google-analytics.htmlandhttps://shiny.rstudio.com/articles/usage-metrics.html
Kevin Rue-Albrecht (17:47:34): > How’s that for a first slide you think? - File (PDF): Presentation1.pdf
Federico Marini (18:27:01): > people, just to be safe
Federico Marini (18:27:17): > we did not increment the version number
Federico Marini (18:27:25): > so I guess the upstream push did nothing?
2018-10-26
Kevin Rue-Albrecht (03:06:02): > We didn’t???
Kevin Rue-Albrecht (03:06:24): > Damn I thought I did a few commits before Aaron’s fixes
Kevin Rue-Albrecht (03:07:02): > To be clear 1.1.13isa version bump over the currently available developer version
Kevin Rue-Albrecht (03:08:26): > I just checked: were fine indeed. Bioc devel landing page says 1.1.12 and I pushed 1.1.13
Kevin Rue-Albrecht (03:08:42): > … don’t scare me like that man:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (03:18:24): > (I also justgit cloned iSEE from bioconductor, and it sends me 1.1.13 back, so they received it)
Kevin Rue-Albrecht (06:34:47): > For those still having our shared Dropbox folder active, I’ve dropped draft slides for a demo that I’ve started to prepare.
Kevin Rue-Albrecht (06:36:19): > Feel free to comment or borrow
Federico Marini (08:09:59): > sorry for the scary effect. was not meant as a preliminary trick or treat
Kevin Rue-Albrecht (08:10:55): > hehehe
Federico Marini (08:11:06): > ok, my bad. Did not see that the bump was already before the latest round of feedback
Kevin Rue-Albrecht (08:11:11): > I’m just as anxious as you to see the build report
Federico Marini (08:11:19): > better safe than sorry
Kevin Rue-Albrecht (08:11:20): > I hate last minute changes
Kevin Rue-Albrecht (15:37:52): > <!channel>:rotating_light:ALERT I’ve just noticed that 1.1.13 has R CMD check issues on Bioc!!!!!
Kevin Rue-Albrecht (15:38:45): > I’m not at a computer right now. Can someone try again, and fix the faulty unit test please?
Elana Fertig (15:39:17): > @Elana Fertig has left the channel
Kevin Rue-Albrecht (15:39:18): > http://bioconductor.org/checkResults/3.8/bioc-LATEST/iSEE/merida1-checksrc.html
Charlotte Soneson (15:56:39): > I pushed this (currently infixtestbranch). Didn’t seem like the.define_colorby_for_column_plotfunction actually returns anything related toscale_size_manualandguides. - File (PNG): Screen Shot 2018-10-26 at 21.54.29.png
Charlotte Soneson (15:59:54): > seems reasonable?
Charlotte Soneson (16:00:12): > the test passes
Kevin Rue-Albrecht (16:01:52): > Yeah. I must have forgotten to run check on Aaron’s chances but that makes sense: my earlier commits added extra commands, and Aaron simplified that
Kevin Rue-Albrecht (16:02:46): > I think it’s ok to have a legend for color even preferable
Charlotte Soneson (16:03:12): > Ok. Travis still fails. I’ll bump the version and force merge then?
Aaron Lun (16:03:58): > That should be correct.
Kevin Rue-Albrecht (16:05:58): > Yep Charlotte, please do merge
Kevin Rue-Albrecht (16:06:14): > I’ll push upstream as soon as I get home
Aaron Lun (16:06:27): > There’s still a few warnings floating about.
Kevin Rue-Albrecht (16:06:43): > Warnings ?
Aaron Lun (16:06:59): > Notes.
Aaron Lun (16:07:13): > x -> x_aes and the like.
Kevin Rue-Albrecht (16:07:16): > Ah I’m used to seeing 2
Aaron Lun (16:07:28): > What’s wrong with Travis anyway?
Kevin Rue-Albrecht (16:08:28): > Something about R packages not being available
Kevin Rue-Albrecht (16:10:57): > And I think Travis is still struggling with the move to biocmanager
Aaron Lun (16:14:12): > Hm.
Charlotte Soneson (16:19:27): > ok, R CMD CHECK passes locally with 3 NOTEs
Charlotte Soneson (16:23:43): > and yes, one of them seems to want thexargument of.add_color_to_row_plotand.add_color_to_column_plotto be calledx_aesinstead. Was this just changed?
Kevin Rue-Albrecht (16:31:14): > Possible@Aaron Lunmentioned described it as x in a message, I initially coded it as x_aes, but he might have edited it
Kevin Rue-Albrecht (16:31:31): > I just got home, I can look at it too now
Kevin Rue-Albrecht (16:36:22): > ok yeah i see now: Aaron added an explicitx="jitteredXin the.add_color_to_row_plotcalls
Kevin Rue-Albrecht (16:36:40): > while I’ve named the arguments asx_aesin the function definition
Charlotte Soneson (16:37:36): > changing tox_aesandy_aesseems to fix the NOTE
Kevin Rue-Albrecht (16:37:54): > oh. I was just writing “we can simplify the argument names tox, really” (andy, of course)
Charlotte Soneson (16:38:11): > yeah, that probably works too:slightly_smiling_face:
Federico Marini (16:41:31): > runs fine on my machine as well:wink:
Charlotte Soneson (16:42:24): > so -xorx_aes?
Kevin Rue-Albrecht (16:48:14): > argh.. such hard questions on a Friday evening
Kevin Rue-Albrecht (16:49:01): > looking back at the code, I kinda see why I liked having the_aessuffix. It’s not like the function name itself is very explicit about needed aesthetics
Charlotte Soneson (16:49:23): > The aes versions seem to work. I’ll push it now and merge
Kevin Rue-Albrecht (16:50:08): > Thanks a ton for the quick fix. Sorry I forgot to run check one last time before pushing 13
Charlotte Soneson (16:51:12): > ok, merged
Kevin Rue-Albrecht (16:57:49): > I pulled. > Still, no offense, but I learned my lesson: I’m building now, checking in a minute, and only then I’ll push this time:sweat_smile:
Kevin Rue-Albrecht (17:03:31): > uk, this timedevtoolsis happy, while I’m havingclangissues in the terminal
Kevin Rue-Albrecht (17:04:00): > sorry, forgot to say: i was updating a handful of packages to the latest version
Charlotte Soneson (17:05:08): > you think there’s anything else that needs to be fixed? I can remove the requirement for passed tests for merging into master for everyone temporarily.
Kevin Rue-Albrecht (17:08:13): > 1. i don’t thing anything else needs fixing right now > 2. i’d rather keep the requirement. Maybe, just maybe, we could remove the requirement while still asking Travis to check and send the status back
Charlotte Soneson (17:08:58): > ok. I’m off to bed then:slightly_smiling_face:
Kevin Rue-Albrecht (17:09:51): > thanks again
Kevin Rue-Albrecht (17:19:46): > Alright. Alert is over: I’ve double checked 1.1.14, built, check (2 NOTES) and pushed upstream
Kevin Rue-Albrecht (17:20:04): > Bottom line: don’t push a version 13 two days before the deadline
Aaron Lun (17:26:36): > Well, I pushed 7 of my packages today. Livin’ on the edge…
2018-10-27
Kevin Rue-Albrecht (05:01:34) (in thread): > Hi@Vince Carey. > I’ve just updated and finalised the demo that I mentioned a few days ago:https://github.com/kevinrue/iSEE_customsee the example “plot_multiRedDim” > Can you please let me know if that is what you were looking for whenever you get a chance to try it out? - Attachment (GitHub): kevinrue/iSEE_custom > Examples of custom panels for the iSEE package (https://github.com/csoneson/iSEE) - kevinrue/iSEE_custom
Kevin Rue-Albrecht (05:20:11): > Alright guys, a second custom panel tour is up:https://github.com/kevinrue/iSEE_custom
Kevin Rue-Albrecht (10:14:01): > @Federico MariniI gotta say “iSEE_opaque.pdf” sounds… conceptually interesting:laughing:
Kevin Rue-Albrecht (14:43:47): > on the “to do list whenever”: fix “SampeName” to “SampName” below > > .sampAssayXAxisSampName <- "XAxisSampName" > .sampAssayYAxisColTable <- "YAxisColTable" > .sampAssayYAxisSampName <- "YAxisSampeName" >
Aaron Lun (14:50:55): > Done.
Kevin Rue-Albrecht (14:51:18): > on a separate note, can someone runsce <- normalize(sce); iSEE(sce)and tell me whether they have any problem with the Column Statististics Table?
Kevin Rue-Albrecht (14:52:18): > I’m getting a modalDataTables warning: table id=DataTables_Table_1 - Error in filterRange(dj, k): The range of a numeric / date / time column must be of length 2
Kevin Rue-Albrecht (14:53:13): > Runningtmp <- .sanitize_SE_input(sce), I see that the size factors intmpare not properly turned into a “regular” column in the DataFrame
Kevin Rue-Albrecht (14:54:56): > actually, no, the column seems ok once I access it properly
Kevin Rue-Albrecht (14:55:37): > but still, toggling the linesce <- normalize(sce)seems to be sufficient to cause the table to crash
Aaron Lun (14:56:01): > Do not understand.
Aaron Lun (14:56:30): > example(iSEE, ask=FALSE)works fine for me.
Kevin Rue-Albrecht (14:57:27): > oh right. The example includes that command. So it must be something about the data set that I’m using right now
Kevin Rue-Albrecht (14:57:29): > uh
Aaron Lun (14:57:50): > You probably have NAs or some such crap in there.
Kevin Rue-Albrecht (14:58:36): > mmh.. possibly but evenrange(NA)returns[1] NA NA
Kevin Rue-Albrecht (14:58:44): > (which is length 2)
Kevin Rue-Albrecht (14:59:03): > anyway, I’ll keep playing with the data set
Aaron Lun (15:00:19): > It’s javascript-side, nor R-side error.
Kevin Rue-Albrecht (15:00:35): > it’s for my demo at the university (whenever we arrange it), I picked up the raw counts from the Nat. Comm. that we’re about to publish. Everything went fine until rannormalize:cry:
Kevin Rue-Albrecht (15:01:00): > gotta go for a bit, catch up later
Aaron Lun (15:01:06): > Then it sounds like something is wrong with yoursizeFactors.
Aaron Lun (15:01:10): > For whatever reason.
Kevin Rue-Albrecht (15:17:47): > Actually, I haven’t check yet what’s wrong with the size factors, but I think I know that whatever reason: > for the demo I’m naively grabbing the entire dataset, which include single cells (SmartSeq2), bulks, and blanks
Kevin Rue-Albrecht (15:18:06): > I’m guessing the blanks aren’t playing nice
Kevin Rue-Albrecht (15:19:48): > hm.. nope, there’s two of them:[1] 0.2248250 0.1137873
Kevin Rue-Albrecht (15:20:41): > > > range(sizeFactors(sce)) > [1] 0.002736359 2.855802590 >
Federico Marini (15:23:38) (in thread): > naming things haas to be an art:smile:
Kevin Rue-Albrecht (15:25:14): > Ok.. I’ve taken the problem from the other side: > I also preconfigured the app, including the colStatTable receiving point selection from the colDataPlot (i.e. the table is filtered). > So this time, I left thenormalizeactive, but deactivated all the preconfiguration. No crash. > Must be a bad interaction somewhere then.
Kevin Rue-Albrecht (15:34:38): > Ok. I think it has nothing to do with size factors. > I don’t know how to check that, but I’ve narrowed down the bug to the situation where I preconfigure the colStatTable to receive (i.e. filter) the point selection from the colDataPlot, with a predefined point selection on the colDataPlot. > I suppose the app tries to initialize the table (and in particular its filters) before it’s populated. I imagine that JavaScript tries to compute a range on an empty vector. > But, actually, why that happens only whensizeFactorsis set, I don’t understand.
Kevin Rue-Albrecht (15:38:29): > building a MWE
Kevin Rue-Albrecht (15:42:44): > Here goes > > library(scRNAseq) > data(allen) > class(allen) > > # Example data ---- > > library(scater) > sce <- as(allen, "SingleCellExperiment") > counts(sce) <- assay(sce, "tophat_counts") > sce <- normalize(sce) > > colDataArgs <- colDataPlotDefaults(sce, 1) > > colDataArgs$YAxis <- "driver_1_s" > > colDataArgs$XAxis <- "Column data" > colDataArgs$XAxisColData <- "Core.Type" > > colDataArgs$BrushData[[1]] <- list( > xmin = 1.5, xmax = 2.5, ymin = 1.5, ymax = 2.5, mapping = list(x = "X", y = "Y"), > log = list(x = NULL, y = NULL), direction = "xy", > brushId = "colDataPlot1_Brush", outputId = "colDataPlot1") > > colStatArgs <- colStatTableDefaults(sce, 1) > > colStatArgs$SelectByPlot <- "Column data plot 1" > > initialPanels <- DataFrame( > Name=c("Column data plot 1", "Column statistics table 1"), > Width = c(4, 8) > ) > > # launch the app itself ---- > > app <- iSEE(se = sce, colDataArgs = colDataArgs, colStatArgs = colStatArgs, initialPanels = initialPanels, appTitle = "Bug report #999") > > if (interactive()) { > shiny::runApp(app, port = 1234) > } >
Kevin Rue-Albrecht (15:45:40): > SetcolStatArgs$SelectByPlot <- "---"and it works. > Don’t set the colDataPlot brush and it works.
Aaron Lun (17:46:05): > Raise an issue, I’m at SCG2018 starting tomorrow and I won’t be able to check this out.
2018-10-28
Aaron Lun (06:18:12): > “Fixed”, of a sort.
Kevin Rue-Albrecht (07:18:46): > works for me! so “thanks”, of a sort:wink:
Kevin Rue-Albrecht (07:26:10): > hm.. actually… it fixes the initialisation of the tables, but then they break with a Shiny error (red message in the panelinvalid 'times' argument) whenever I change their linked panel or the selection in the linked panel
Kevin Rue-Albrecht (07:26:48): > (I saw your note about internal fields, btw, maybe it’s about them, I don’t know)
Aaron Lun (07:27:14): > Probably something to do with updating of thesearch_cols. Play around with it, I’ve got to go.
Kevin Rue-Albrecht (07:27:35): > Yup, I’ll find some time
Kevin Rue-Albrecht (07:27:48): > You may spot Steve at the conference btw
Kevin Rue-Albrecht (11:47:08): > Ok. Problem temporarily fixed, but I’m definitely not rushing a push upstream for that, let’s properly plan and go through the refactoring that Aaron suggested in the issue comments.
Kevin Rue-Albrecht (12:13:15): > If you guys can see the Powerpoint presentation in Dropbox, feel free to point out stuff to add to the acknowledgements slide. I’ve only started, and I’m going off the top of my head, reading the DESCRIPTION file. I’ll may also add names like Davis who gave us feedback from early on
Charlotte Soneson (12:15:38): > Bioconductor?:slightly_smiling_face:
Kevin Rue-Albrecht (14:04:49): > Fair point. I put the full logo initially, and then dropped it when I added their sticker
Kevin Rue-Albrecht (18:20:29): > Having way too much fun with custom panels:smile: - File (PNG): Screenshot 2018-10-28 22.20.02.png
Kevin Rue-Albrecht (18:21:46): > I need to fix the colors though ..
2018-10-29
Aaron Lun (11:25:59): > Ran into the S-bomb
Federico Marini (11:50:50): > huh?
Kevin Rue-Albrecht (11:51:37): > I guess that was directed to me. As in “Steve” or “Supervisor”:stuck_out_tongue:
Federico Marini (11:51:48): > Aah
Aaron Lun (11:54:56): > Yeas.
Kevin Rue-Albrecht (12:09:36): > Mind you, he’s possibly on the hunt for a postdoc. We have one leaving end of November.:zipper_mouth_face:
Aaron Lun (12:10:07): > Hm.
Aaron Lun (12:10:31): > listening to more of these atlasing talks:sleeping:
Kevin Rue-Albrecht (12:13:05): > I’m falling asleep too, between sore throat that started yesterday, dentist who drilled my tooth this morning, and a bunch of Rmd notebooks to update because I need to filter an outlier from all downstream analyses:sleepy:
Kevin Rue-Albrecht (12:13:57): > and not a chance of “lobstah” roll for dinner:stuck_out_tongue:
Kevin Rue-Albrecht (15:25:50): > Motivation forggforce::geom_sina()? > Tweet here:https://twitter.com/strnr/status/1056908633454362626 - File (PNG): Pasted image at 2018-10-29, 7:25 PM
Charlotte Soneson (15:51:03): > 1.1.14 looks fine:slightly_smiling_face: - File (PNG): Screen Shot 2018-10-29 at 20.49.50.png
Aaron Lun (16:03:36): > I don’t understand ggforce.
Aaron Lun (16:03:42): > WHat’s the point.
Federico Marini (16:10:14): > Aaron, when is poster time there?
Aaron Lun (16:10:43): > we had one session already.
Aaron Lun (16:10:48): > Waiting for the pre-dinner session.
Federico Marini (16:12:40): > was yours already in that one?
Federico Marini (16:12:47): > I am curious of the reactions
Aaron Lun (16:12:59): > no, two mini-sessions per poster session
Aaron Lun (16:13:05): > And yes, I am in this one.
Federico Marini (16:13:35): > Cool, best luck then!
Federico Marini (16:13:49): > Take a pic of you in front of that:slightly_smiling_face:
Federico Marini (16:20:08): > About the sticker, I’ll submit soon the image for the proof
Kevin Rue-Albrecht (16:20:53): > I’ve never usedggforce. I’m just interested because it’s ageom_*, so it sounds simple to use, and seems to yield a result similar tovipor::offsetXwhich requires a bit more code
Aaron Lun (16:24:41): > The point is not that we have to useoffsetX, the point is that we have to save the coordinates. We could usegeom_quasirandomif we wanted to.
Kevin Rue-Albrecht (16:25:05): > oh right i forgot that for a moment, indeed
2018-10-30
Federico Marini (04:24:41): > yo channel
Federico Marini (04:24:55): > I thought evo-devo on despacito was top
Federico Marini (04:24:57): > but this one wins hands down
Federico Marini (04:24:59): > https://youtu.be/whreNeJGCWk - Attachment (YouTube): Postdoc Me Now - A “Don’t Stop Me Now” Science Parody
Federico Marini (10:00:52): > Order’s placed on stickermule!
Federico Marini (10:01:14): > If you order something there, I got a 9euro for you-9 euro for me deal
Federico Marini (10:02:55): > Plus, another happy news: the presentation for iSEE at EuroBioc is accepted, slot of 15 mins
Kevin Rue-Albrecht (10:03:18): > neat, that’s a fair amount of time
Federico Marini (10:22:25): > Grazie per avere approvato le prove di stampa. Non vediamo l'ora di iniziare con la produzione del tuo ordine R234670256. Lo riceverai entro il November 08.
Federico Marini (10:22:40): > Soon I should hold them in my hands
Kevin Rue-Albrecht (10:28:07): > Our precious
Kevin Rue-Albrecht (10:28:31): > The one package to plot them all
Kevin Rue-Albrecht (10:46:09): > iSEE sticker looks good on Bioconductor/BiocStickers:slightly_smiling_face:
Aaron Lun (14:36:32): > Can someone check and mergeminorfixes?
Federico Marini (16:49:57): > I got these
Federico Marini (16:50:14): > > ══ testthat results ═════════════════════════════════════════════════════════════════════════════════════════════════ > OK: 1070 SKIPPED: 3 FAILED: 2 > 1. Failure: memory setup works correctly (@test_misc.R#176) > > 2. Failure: memory setup works correctly (@test_misc.R#226) >
Kevin Rue-Albrecht (16:53:19): > Right.. I only fixed the targeted runtime issue, didn’t check the unit tests.
Kevin Rue-Albrecht (16:59:29): > Maybe at the moment, a simple fix of the unit test can do the trick, but Aaron commented on generalizing the fix:https://github.com/csoneson/iSEE/issues/254 - Attachment (GitHub): Table panels fail to initialise when they receive a point selection at startup · Issue #254 · csoneson/iSEE > The following MWE reveals an issue related to the initialisation of table panels when they are initialised to receive a selection from a plot panel itself initialised with a point selection. The ob…
Kevin Rue-Albrecht (17:22:53): > If it’s still there, I’ll have a proper look this weekend. Right now I need to catch up some sleep:sneezing_face:
2018-10-31
Federico Marini (05:02:42): > Looks like soon we might also have soon a conda recipe for iSEE ready
Federico Marini (05:03:11): > https://github.com/bioconda/bioconda-recipes/pull/8803 - Attachment (GitHub): adding iSEE (and one of its dependencies, scRNAseq) from R/Bioconductor by federicomarini · Pull Request #8803 · bioconda/bioconda-recipes > I have read the guidelines for bioconda recipes. This PR adds a new recipe. AFAIK, this recipe is directly relevant to the biological sciences (otherwise, please submit to the more general purpos…
Kevin Rue-Albrecht (06:15:04): > Neat
Kevin Rue-Albrecht (06:15:51): > btw, I just realized that I won’t be able to do anything this weekend. I’m going to France for the weekend to renew my ID card on Monday.
Federico Marini (08:47:54): > Other info relevant to us all maybe, dear<!channel>: after catching up with Stephanie Hicks re#hca_rfaupcoming activities, she offered to > > [I would] just add your name to the document with a description of what you are interested in working on
Federico Marini (08:48:12): > I’d say we should all be listed?
Kevin Rue-Albrecht (08:50:28): > Yup. Happy with that. I had a look at her section on interactive graphics, and I definitely want to keep an eye on developments in that area… which usually means that I won’t be able to help sending some PR’s in anyway:stuck_out_tongue_winking_eye:
Charlotte Soneson (08:50:58): > :+1:
Federico Marini (08:51:15): > None of us being officially a PI; I wonder where we could fit at best
Federico Marini (08:51:41): > from the existing doc, it is more like everyone planned also for their groups
Federico Marini (08:53:23): > Basically, we are in a situation where it is hard to make requests:open_mouth:
Federico Marini (08:54:10): > Lord Commander Aaron is unfortunately in other time zones, he might have a clever tip for this
Kevin Rue-Albrecht (08:54:33): > True. That’s a major limitation for me indeed given that the HCA wants a “list of co-PIs”. But I guess we could be mentioned as being community contributors to the field perhaps (I’m looking right now at the draft linked in the#hca_rfachannel to figure out exactly where and how)
Kevin Rue-Albrecht (08:56:04): > … I wonder whether I could convinceclusterExperimentto use theExperimentColorMap… the heat maps could definitely use some colormap polishinghttp://www.bioconductor.org/packages/release/bioc/vignettes/clusterExperiment/inst/doc/clusterExperimentTutorial.html#step1
Kevin Rue-Albrecht (08:56:17): > hurts my eyes
Federico Marini (08:58:09): > Careful to not hurt Davide’s feelings:smile:
Kevin Rue-Albrecht (08:58:52): > to be fair, tough to define a “good” set of colors, when there are so many to combine in a single figure
Federico Marini (08:59:57): > true that
Federico Marini (09:00:38): > anyway. I messaged Stephanie and invited her to join us here, at least for a while to discuss with the four of us instead of having me reporting back n forth
Kevin Rue-Albrecht (09:00:44): > maybe that colorbrewer web page (too lazy to google right now) that uses geographical maps to play with different colormaps could be useful in this case
Kevin Rue-Albrecht (09:01:17): > well.. that thing:http://colorbrewer2.org/#type=qualitative&scheme=Set3&n=12
Stephanie Hicks (12:50:13): > @Stephanie Hicks has joined the channel
Stephanie Hicks (12:54:31): > Hi everyone! I’m joining the channel by invitation from@Federico Marini. He mentioned on#hca_rfahis interest in seeing effort being put towards#iseein the next round of CZI applications. idea here:https://community-bioc.slack.com/archives/C6EMXJ0RY/p1540849039006300 - Attachment: Attachment > in brief: take any SummarizedExperiment object, view different panels of its data, with plots and tables linked by brushing, and a spray of reproducibility with self-generated code
Stephanie Hicks (12:55:58): > Is this something you guys would like to see happen? or could you tell me more about what you have in mind?
Kevin Rue-Albrecht (12:59:37): > Just to clarify Fede’s quote, what he describes isn’t an idea, it’s whatiSEEalready does.
Stephanie Hicks (13:00:29): > oops, sorry! copied the wrong line
Stephanie Hicks (13:00:31): > https://community-bioc.slack.com/archives/C6EMXJ0RY/p1540848985005200 - Attachment: Attachment > Would the package be worth considering in this scope?
Stephanie Hicks (13:01:02): > meaning I believe his suggestion was to include effort and funding to support iSEE (unless I misunderstood)
Stephanie Hicks (13:01:35): > (which is totally possible ha)
Kevin Rue-Albrecht (13:01:55): > I haven’t taken the time yet to play withclusterExperiment, I looked briefly at the vignette today, and I’m genuinely curious about the idea of “scalable, interactive graphics for millions of cells” that you’ve described in the HCA RFA. Even if first as an early user, and as far as I can contributing to it.
Kevin Rue-Albrecht (13:04:44): > But indeed, I think we were discussing how we could get involved/supported as part of the HCA RFA, as Federico pointed out earlier - Attachment: Attachment > None of us being officially a PI; I wonder where we could fit at best
Stephanie Hicks (13:06:34): > Ah, this stemmed out of a discussion with@Davide Rissoand@hcorradaat#bioc2018in Toronto. We were discussing how it would be cool e.g. to be able to visualize a heatmap with e.g. a million cells. I tossed the idea of making it interactive so at a high-level, the heatmap would be summaries or averages of cells but as you zoom in (meaning subset of cells) you could see the intensity for those cells.@hcorradahas done a lot of work on this already in the metavizr package and we started#palmtreechannel to think more about this.
Stephanie Hicks (13:09:40): > So, I was a postdoc at the time Bioc applied for the first round of CZI funding and I joined@Davide Risso’s proposal (even though we were at different institutions) and asked for a % of salary support in the proposal as a postdoc. By the time the application was actually funded, I was a faculty member, but I don’t think that mattered. If your advisor is supportive of your interest in the CZI bioc proposal, I’m happy to include funding to support your effort.
Kevin Rue-Albrecht (13:25:43): > Thanks a ton for the offer. > Personally I’ll have to get back to you on that after a chat with my PI (he’s at the SCG2018 this week), he’ll rightly argue that we’re already at capacity with several ongoing projects, but he might be convinced to get in touch with you and discuss details. > If I found the right information, the deadline for applications is November 13th, 2018 ?
2018-11-01
Charlotte Soneson (13:06:56): > Ok, here’s a weird behaviour: > On the allen data, do the following (in this order): > - Set Row statistics table 1 to receive selection from Row data plot 1 > - Set Row data plot 1 to plot ave_count vs n_cells > - Select some genes in Row data plot 1 (the table updates) > - Set Reduced dimension plot 1 to color by Feature name, received from Row statistics table 1 > -> selecting genes in the table for coloring doesn’t work (either does nothing, or selects the wrong gene) > > If you do things in another order, it seems to work, e.g. the following: > - Set Reduced dimension plot 1 to color by Feature name, received from Row statistics table 1 > - Set Row statistics table 1 to receive selection from Row data plot 1 > - Set Row data plot 1 to plot ave_count vs n_cells > - Select some genes in Row data plot 1 (the table updates) > -> now, selecting genes in the table for coloring works fine… > > Can anyone reproduce this (or even better, understand why it’s happening)?
Kevin Rue-Albrecht (13:09:00): > I can’t remember whether table selections are passed as row name or index?
Kevin Rue-Albrecht (13:09:45): > could be a wrong index, if there’s any reordering
Charlotte Soneson (13:09:46): > index, by looking at the parameter table. But shouldn’t the two things above give the same results?
Kevin Rue-Albrecht (13:11:01): > from reading the list or actions, i imagine they should, but obviously I trust you that one of them doesn’t work
Kevin Rue-Albrecht (13:15:57): > i’ve just tested scenario 1: i can see the bug too
Kevin Rue-Albrecht (13:17:17): > Scenario 2 does the same for me too. Curiouser and curiouser
Charlotte Soneson (13:18:35): > Same as for me, or same as scenario 1?
Kevin Rue-Albrecht (13:18:44): > same as you i meant
Charlotte Soneson (13:19:22): > Ok
Charlotte Soneson (13:20:00): > So at least it’s reproducible
Kevin Rue-Albrecht (13:20:14): > yep (with 1.2.0)
Charlotte Soneson (13:20:22): > Yup
Aaron Lun (13:22:32): > Keep in mind that the DT table never actually gets subsetted.
Kevin Rue-Albrecht (13:22:41): > uh. hang on: is it normals that when the redDimPlot1 colorByFeatureName is linked to rowStatTable1, the selectize still lists all the features in the data set?
Kevin Rue-Albrecht (13:23:09): > Shouldn’t the choices be limited to the features present in the table?
Aaron Lun (13:24:18): > No.
Aaron Lun (13:24:21): > That’s what I’m saying.
Aaron Lun (13:24:42): > The table never actually gets subsetted.
Aaron Lun (13:24:50): > Only theSelectedthing is set toTRUE.
Kevin Rue-Albrecht (13:24:57): > yup, I just saw that again
Aaron Lun (13:24:59): > Otherwise you get some crazy effects
Kevin Rue-Albrecht (13:25:22): > plus that’s actually a different issue: while I can manually select another gene that’s not currently in the table, selecting a gene again in the table reset the selectize selection again
Aaron Lun (13:26:27): > yes, that’s desired behaviour.
Aaron Lun (13:27:10): > … though I guess user’s shouldn’t have the opportunity to manually seelct a gene when they’ve linked it to a table.
Kevin Rue-Albrecht (13:27:42): > oohhhh I think my helper could.. help.. with that
Kevin Rue-Albrecht (13:27:45): > hang on that I find it again
Kevin Rue-Albrecht (13:28:25): > iSEE:::.disableButtonIf
Kevin Rue-Albrecht (13:28:49): > although I might need to write a selectize flavour of it
Aaron Lun (13:55:30): > Unfortunately, it would need to disable a selectize.
Aaron Lun (13:55:38): > Which is a bit more ambitious.
Aaron Lun (13:55:44): > Because the selectize will still need to silently update.
Kevin Rue-Albrecht (14:57:26): > I’ve never disabled a selectize, but it’s worth looking into. Also, you’ll see iniSEE:::.disableButtonIfthat disabling and updating are handled separately
2018-11-04
Kevin Rue-Albrecht (12:53:43): > Btw@Charlotte Soneson, I’ve been looking a bit at the table issue with some print statements. When the table link is set “too late” (after the row is selected), the selected index refers to the row indexin the visible subsetinstead of the full table. Not sure why yet, but@Aaron LunI’d like that one solved before mergingminor_fixes
Kevin Rue-Albrecht (12:56:04): > What puzzles me is this “visible index” being returned, if as Aaron wrote > > The table never actually gets subsetted. > That said, I don’t completely understand how the subsetting is achieved, as when the table updates to the selection, the “Selected” column filter is blank yet the table is filtered appropriately
Kevin Rue-Albrecht (12:56:19): > looks like a javascript hickup
Kevin Rue-Albrecht (13:11:45): > I may have just solved it. Stand by while I run again the steps that Charlotte listed
Kevin Rue-Albrecht (13:13:17): > Yep. Looks to me like it works now
Kevin Rue-Albrecht (13:13:23): > Cleaning up and pushing
Kevin Rue-Albrecht (13:16:36): > tadaa
Kevin Rue-Albrecht (13:18:21): > can you give it a go@Charlotte Sonesonand let me know if you can still spot any odd behaviour around that point?
Kevin Rue-Albrecht (13:18:54): > (whenever, I’ll be on the move tomorrow, so not much I will be able to do)
Charlotte Soneson (14:59:48): > Thanks@Kevin Rue-Albrecht! This seems to work indeed:slightly_smiling_face:
Aaron Lun (15:32:08): > huh
Aaron Lun (15:32:09): > what’s going on
Aaron Lun (15:32:14): > I just got back into the office
Charlotte Soneson (15:33:15): > This:https://community-bioc.slack.com/archives/C8BJLSP8T/p1541092016132100 - Attachment: Attachment > Ok, here’s a weird behaviour: > On the allen data, do the following (in this order): > - Set Row statistics table 1 to receive selection from Row data plot 1 > - Set Row data plot 1 to plot ave_count vs n_cells > - Select some genes in Row data plot 1 (the table updates) > - Set Reduced dimension plot 1 to color by Feature name, received from Row statistics table 1 > -> selecting genes in the table for coloring doesn’t work (either does nothing, or selects the wrong gene) > > If you do things in another order, it seems to work, e.g. the following: > - Set Reduced dimension plot 1 to color by Feature name, received from Row statistics table 1 > - Set Row statistics table 1 to receive selection from Row data plot 1 > - Set Row data plot 1 to plot ave_count vs n_cells > - Select some genes in Row data plot 1 (the table updates) > -> now, selecting genes in the table for coloring works fine… > > Can anyone reproduce this (or even better, understand why it’s happening)?
Charlotte Soneson (15:34:19): > From Kevin’s commit: - File (PNG): Screen Shot 2018-11-04 at 21.33.46.png
Aaron Lun (15:36:56): > I don’t get the odd behaviour in the first place.
Charlotte Soneson (15:38:03): > Ok, interesting. Both I and Kevin had the same problem.
Charlotte Soneson (15:38:34): > I mean, on the currentminor_fixesbranch it works.
Aaron Lun (15:38:46): > Yes, and if I roll back the last commit, it still works.
Aaron Lun (15:39:20): > Can either of you explain exactly what the problem is? Selection of genes in the table for coloring works fine for me on9e90019f639471121cc5ba478e06b29308ab197e.
Charlotte Soneson (15:41:48): > Well, the problem we had was that depending on the order in which the transmissions were set up, the wrong gene (not the one selected in the table) could be used for the coloring. So it did indeed color by a gene, just not the one that was selected. If the transmissions were set up in a different order (scenario 2 in the message above), things worked as expected.
Aaron Lun (15:42:26): > Yeah, I’m doing it by 1, there’s no problem.
Aaron Lun (15:43:15): > and yes, this is pre-“fix”, I’m looking at theiSEEsource code in the console.
Aaron Lun (15:47:40): > Nup, not getting it.
Charlotte Soneson (15:48:30): > Just got this, for example. It works for some genes, but not for all. - File (PNG): Screen Shot 2018-11-04 at 21.47.20.png
Aaron Lun (15:52:49): > Still works fine for me.
Aaron Lun (15:52:55): > (i) Is it a safari thing,
Aaron Lun (15:53:01): > (ii) are you using R-devel?
Charlotte Soneson (15:54:15): > (i) I’m running in the RStudio browser. (ii) No, R 3.5.1. Does it work for you even if you make multiple successive gene selections?
Aaron Lun (15:56:21): > looks pretty reasonable to me, re multiple gene selections.
Charlotte Soneson (15:57:06): > Hmm, weird.
Aaron Lun (15:57:29): > Try running it on safari.
Charlotte Soneson (16:00:47): > Yes, there it seems to work better
Kevin Rue-Albrecht (16:05:45): > Well anyway, basically, the issue that I fixed, whether related or not, is that the value of the “Selected” column filter was incorrect, it should have been["true"], not"true"
Kevin Rue-Albrecht (16:06:46): > (because the column filter is technically a multiple selectize, not a single selection dropdown)
Kevin Rue-Albrecht (16:08:19): > I drilled down to that point because I observed that the selected row index sent from the table to the plot was the one of the “visible” table, instead of the row index of the full table
Kevin Rue-Albrecht (16:08:52): > Stilll, weird that you couldn’t replicate the bug in the order that Charlotte reported
Aaron Lun (16:08:52): > what
Aaron Lun (16:09:09): > the row index should always be of the full table.
Aaron Lun (16:09:20): > I think rstudio’s browser is fucked up
Kevin Rue-Albrecht (16:10:55): > well, when I added debugprintstatement of theparam_choices, and selected ~3 genes following Charlotte’s steps before my fix, I could clearly see that the*FeatNamein the range 1-3 exactly in the order where genes were listed in the subsetted table
Aaron Lun (16:16:06): > Perhaps. But it works fine on safari and firefox, so this just seems like a problem with Rstudio.
Aaron Lun (16:17:14): > Certainly it’s not obvious that it should have been fixed by changing"true"to["true"]
Aaron Lun (16:17:24): > I don’t see how that would affect anything.
Kevin Rue-Albrecht (16:19:47): > to my eyes, that definitely affects the column filter of the Selected column, I can finally see the field prepopulated with thetruevalue
Aaron Lun (16:19:59): > Yes, but why?
Aaron Lun (16:20:23): > Why does it change the behaviour of the indices?
Aaron Lun (16:20:50): > Or specifically, why does it change the indexing behaviour of the DT?
Aaron Lun (16:21:06): > And only in Rstudio as well.
Kevin Rue-Albrecht (16:22:08): > That I can’t say, I was chasing Charlotte’s bug, spotted that other one, and somehow they seem connected
Aaron Lun (16:22:59): > sigh. Okay, well, add a comment about Rstudio there.
Kevin Rue-Albrecht (16:23:10): > I didn’t think to test in another browser
Kevin Rue-Albrecht (16:23:27): > (in the code you mean?)
Aaron Lun (16:23:33): > Yes.
Aaron Lun (16:23:54): > Putting aside the indexing behaviour, why should it be["true"]and not"true"?
Kevin Rue-Albrecht (16:24:56): > well, maybe that was also just an RStudio browser thing, but when it was “true”, the table was filtered but the column filter field appeared empty
Kevin Rue-Albrecht (16:25:40): > that was bugging me, so I selected thetruevalue, and then looked into the “Panel settings” tracker, and saw how it was reported
Kevin Rue-Albrecht (16:25:50): > aka["true"]
Kevin Rue-Albrecht (16:26:21): > … which makes sense when you consider that multiple values can be selected
Kevin Rue-Albrecht (16:26:31): > i’ll be back in 10min
Aaron Lun (16:28:09): > Hm. Shows up fine for me.
Aaron Lun (16:29:16): > Though I would like the_Selectedcolumn to not be visible at all.https://datatables.net/examples/basic_init/hidden_columns.html
Kevin Rue-Albrecht (16:47:09): > Ok.. but while your suggestion to try safari instead works, have you tried yourself in rstudio to see what we mean?
Aaron Lun (16:47:55): > I don’t have rstudio
Aaron Lun (16:48:02): > anywhere within reach
Aaron Lun (17:08:22): > Well, anyway, I’m going home now. Add the comment (also see if you can hide the column) and merge it.
Aaron Lun (17:08:46): > Hiding is important if possible because otherwise the user could manually change the search field in_Selectedand this would lead to chaos.
Aaron Lun (17:21:49): > Just tested: this is achievable viacolumns=c(list(NULL), list(NULL), list(NULL), list(list(visible=FALSE)))as an extra field in the list ofoptions=indatatable().
Aaron Lun (17:22:23): > Note that it assumes that the last column is the_Selected, you’ll need to do some conditional adding otherwise it will fail when thecolumnslist is longer than the actual number of columns.
Kevin Rue-Albrecht (17:47:57): > eh.. I think that I came up with something simpler in the meantime
Kevin Rue-Albrecht (17:48:54): > i’ve just pushed it, you can have a look
Kevin Rue-Albrecht (17:49:27): > g’night!
Aaron Lun (18:09:44): > yep, that’ll work as well.
2018-11-05
Kevin Rue-Albrecht (03:22:15): > I just don’t have an internet connection right now, so I can’t push a couple more commits that fix unit tests and an annotation example that requires an internet connection
Kevin Rue-Albrecht (03:22:28): > Soon as I can, I’ll push and merge
Kevin Rue-Albrecht (03:22:47): > For some reason the phone hotspot doesn’t work
Kevin Rue-Albrecht (05:33:34): > What the hell seriously: “dependencies … ‘corpus.useR.2008.abstracts’ … are not available”
Federico Marini (05:33:42): > no clue
Kevin Rue-Albrecht (05:33:44): > (Travis )
Federico Marini (05:33:48): > travis is goin crazy
Federico Marini (05:33:56): > for my repos I don’t have that issue
Federico Marini (05:34:06): > so, no clue where it comes from
Federico Marini (05:34:13): > anyone else gettin the same?
Kevin Rue-Albrecht (05:34:29): > For the record what I last pushed on “minor fixes” passed with 2 notes on my laptop
Kevin Rue-Albrecht (05:34:40): > As far as I’m concerned it’s ready for merge
Kevin Rue-Albrecht (05:35:08): > I’m just on RER circling back to Paris then London and oxford:sleepy:
Kevin Rue-Albrecht (05:35:56): > All that to renew my French ID with a 2 week wait instead of 3 month at the embassy in London
Federico Marini (05:41:17): > which reminds me I also have to renew mine
Kevin Rue-Albrecht (05:46:20): > Don’t feel the need to rush… I’ve only realised recently that mine was outdated for the past 3 years…
Kevin Rue-Albrecht (05:46:46): > You can read about French ID cards being given extra 5 years validity beyond their printed expiry date “if” you were an adult when you received it
Kevin Rue-Albrecht (05:47:08): > I didn’t pay attention enough to the last bit
Kevin Rue-Albrecht (10:58:07): > Sorry for the pressure on you@Charlotte Soneson, but it seems that those Travis issues are here to stay awhile. Can you force mergehttps://github.com/csoneson/iSEE/pull/255(and perhaps run one lastcheckbefore). > Your recent idea to remove some of the continuous integration failsafes is probably worth it (i..e still run the checks but don’t require them to allow merging) - Attachment (GitHub): Minorfixes by kevinrue · Pull Request #255 · csoneson/iSEE
Federico Marini (10:59:20): > Kev, do you want to check what is different in our travis.yaml vs the one of my repos? Can’T figure out why it sucks only for iSEE
Federico Marini (10:59:30): > The sudo mode?
Kevin Rue-Albrecht (11:00:33): > From the looks of it, it looks like it’s only the chain of dependencies that’s causing it to fail
Kevin Rue-Albrecht (11:00:47): > did you have a look at the Travis build log?
Federico Marini (11:05:08): > yes, this list of packages is a real RKO
Charlotte Soneson (11:12:41): > Ok - I can’t check right now, but I can do it this evening. But maybe I’ll just open up for everyone to merge for now.
Kevin Rue-Albrecht (11:14:37): > No rush about merging - whenever you get the chance. Widening rights to force merge for a while sounds good. We’re all responsible adults… kind of… ^^
Charlotte Soneson (11:17:16): > Ok, I removed the CI requirement for merging.
Kevin Rue-Albrecht (11:29:44): > @Aaron Lun@Federico Mariniany wish to check anything before I force-merge?
Aaron Lun (11:29:49): > Nope.
Federico Marini (11:30:39): > I trust you and C on this, plus I have too many things open
Kevin Rue-Albrecht (11:31:21): > Cool. Let’s do it then.
Aaron Lun (11:31:49): > FINISH HIM:-1:
Aaron Lun (11:31:53): > :crossed_swords:
Aaron Lun (11:31:58): > :woman-swimming:
Aaron Lun (11:31:59): > oops.
Kevin Rue-Albrecht (11:32:02): > critical strike
Federico Marini (11:42:42): > WARNINGless victory
2018-11-06
Federico Marini (03:58:44): > Huhu:slightly_smiling_face:
Federico Marini (03:58:49): > Couple of nice things
Kevin Rue-Albrecht (03:58:59): > ?
Federico Marini (03:59:13): > UPS Update: Paket ist morgen für die Zustellung terminiert
Federico Marini (03:59:20): > -> maybe the stickers can be already there
Kevin Rue-Albrecht (03:59:46): > :slightly_smiling_face:
Federico Marini (03:59:49): > and 2.
Federico Marini (03:59:51): > https://bioconda.github.io/recipes/bioconductor-isee/README.html
Federico Marini (04:00:07): > the recipe for the 1.0.1 is also in (bio)conda
Kevin Rue-Albrecht (04:00:23): > neat
Charlotte Soneson (04:00:50): > Cool:slightly_smiling_face:so how does it work then for updating - will the recipe be automatically run somehow for Bioc 3.8 - I haven’t really understood how this all works.
Kevin Rue-Albrecht (04:01:05): > I know some people in O-place who use conda a lot for pipeline environments. Curious to see if they can make use of it for iSEE
Federico Marini (04:01:11): > Someone needs to update it
Federico Marini (04:01:18): > and I am already in the process to do so
Charlotte Soneson (04:01:28): > :+1:
Federico Marini (04:01:38): > it probably needs to wait for some other bioc pkgs to be brought to newest release version
Federico Marini (04:01:54): > but it is also there in this format
Federico Marini (04:02:52): > It can also be a potential additional point for reply to reviewer (Lorena?) abt easy install procedures
Kevin Rue-Albrecht (04:04:34): > Can you dig up the GoogleDocs that I started for replies to reviewers?
Federico Marini (04:05:20): > I’m in the middle of a couple of meetings, but yes, I can (should) be able to find it
Federico Marini (04:05:35): > One more relevant thing for usiSEEers
Charlotte Soneson (04:06:19) (in thread): > https://docs.google.com/document/d/1nFIW0oVSlU5fqB8Om6H8nZlh_UpnpCMZgwj-7A2QMQE/edit
Federico Marini (04:06:31) (in thread): > :pray:
Federico Marini (04:06:36): > Kev, since you are around, and noone wants to cut anyone out: me and Charlotte are joining a conf call
Federico Marini (04:07:07): > today, from the#palmtreechannel, where among others, Stephanie and Hector (Corrada-Bravo) should be in
Federico Marini (04:07:34): > and one item would be to discuss the whethers and the hows for us/iSEE joining the Bioc effort
Federico Marini (04:08:17): > In brief, I have been doing some homework here ->https://github.com/federicomarini/HCAData - Attachment (GitHub): federicomarini/HCAData > Access data from the Human Cell Atlas project, directly via R/Bioconductor - federicomarini/HCAData
Federico Marini (04:08:48): > With the aim of having a small example instance of iSEE on the data they released already
Federico Marini (04:09:49): > I am encountering some (probably) DelayedArray-related troubles, but I am documenting everything in the vignette-like make-data.Rmd scripts
Federico Marini (04:10:19): > (I am happy to have you in the repo if you want to contribute, that goes without saying)
Kevin Rue-Albrecht (04:19:02): > when is the call?
Kevin Rue-Albrecht (04:21:21): > I think I found the info: 11:00 AM EST
Kevin Rue-Albrecht (04:21:42): > so.. 4pm UK time, Google tells me (?)
Charlotte Soneson (04:21:43): > yep - should be 5pm Swiss/German time
Kevin Rue-Albrecht (04:21:49): > ok perfect
Kevin Rue-Albrecht (04:21:52): > I’ll try to be there
Charlotte Soneson (04:22:06): > the agenda is also in the#palmtreechannel
Kevin Rue-Albrecht (04:22:27): > thanks. i opened the link but haven’t read through yet
Kevin Rue-Albrecht (04:22:41): > (nevermind, it’s 10 lines:sweat_smile:)
Charlotte Soneson (04:22:49): > yes, should be doable:wink:
Charlotte Soneson (07:48:32): > I just presented iSEE in our compbio meeting, and one of the participants was interested in swapping their in-house single-cell shiny app for iSEE, but he was missing the ability to size and alpha the points in a scatter plot by a colData column (we can just color and shape). He can make a custom plot, but the main idea was to be able to select and transmit points from the plot to other panels. Any objections to including such an option?
Aaron Lun (07:49:59): > Both of these are fine but will require some thought as to how they will fight with the selection alpha.
Kevin Rue-Albrecht (07:50:55): > beat me to it
Aaron Lun (07:51:16): > Size, for example, will need some editing of the UI to group together options (i.e., size by “none”, “column data”, “feature name”).
Aaron Lun (07:51:25): > Mimicking the colour by choices.
Kevin Rue-Albrecht (07:51:29): > also, I think size and alpha only work with continuous variables
Aaron Lun (07:51:43): > Certainly it would only make sense there.
Aaron Lun (07:51:53): > Which is not a problem per se, as we restrict to discrete things for shape anyway.
Kevin Rue-Albrecht (07:52:02): > so we’ll have to apply the complementary approach to faceting, which restricts selectize inputs to categorical variables
Charlotte Soneson (07:52:23): > Yes, I agree with all of that.
Kevin Rue-Albrecht (07:56:21): > as a word of warning to users, I’m just also thinking that “size by” may be a misleading feature in combination with downsampling, as the visible points may not be representative of the size of the hidden points
Kevin Rue-Albrecht (07:56:45): > (possibly the same thing applies to “alpha by”)
Charlotte Soneson (07:57:20): > true, but the same would hold for “color by”?
Kevin Rue-Albrecht (07:58:00): > Duh. Sorry! Didn’t really think of the whole thing til now, in fact
Kevin Rue-Albrecht (07:58:07): > oh well
Aaron Lun (07:58:41): > Arguably if points are all large enough, it shouldn’t matter.
Aaron Lun (07:59:07): > Though really, anyone using size for anything meaningful is probably asking for trouble.
Kevin Rue-Albrecht (07:59:37): > in any case, sounds like those features just re-use the mechanics of our existingparam_choices. So completely possible, just requires adequate attention and time.
Stephanie Hicks (19:39:18): > Hi everyone!@Federico Mariniand I were wanting to see if we could maybe get a smaller group of people on a call to discuss next steps for the CZI application that Bioc is putting together (led by@Martin Morgan). At the moment, I’m budgeting \[ for a postdoc at Hopkins to work on interactive data viz for scRNA-seq data in Bioc, but I can easily imagine that \] could go toward % effort for some of the amazing#iseeteam members. I’m not sure who all is interested and how it would work, but I wanted to at least open up the conversation.
2018-11-07
Kevin Rue-Albrecht (02:28:44): > @Stephanie HicksThanks again for the offer. I’m happy to be part of a call, even if only to be part of the conversation, but maybe not an explicit part of the CZI application (this time?). I haven’t seen my PI yet, but as a heads up: > - my postdoc salary isalreadycoming from two sources of collaboration for the next slightly-less-than 2 years, which means that it’s relatively difficult for me to argue adding more ‘explicit’ effort on my plate, before I clear out some ongoing projects > - that said, of course I’m using and looking after iSEE whenever possible, so I’m happy to be mentioned as a community contributor (if that’s a thing) to the CZI dataviz effort. (I have no clue either about the “how it would work” re: the CZI application) > - Steve is a relatively young PI, so it’s still important for both him and me that I put effort primarily in papers where I’m first and he’s last author. I always need to keep that in mind while navigating discussions like these, as there are only 24h in a day.
Charlotte Soneson (04:21:44): > Hi@Stephanie Hicks, thanks for reaching out! Like Kevin, I’m also obviously happy to be in the loop and discuss how to use and/or adapt and develop iSEE for the purposes of application to HCA and beyond, but with my current position my job description has changed a bit (and I also just started) and I’m just not sure that I can commit to delivering enough that it would warrant actual funding at this stage.
Stephanie Hicks (07:56:43): > Thanks@Kevin Rue-Albrecht@Charlotte SonesonI completely understand. At a minimum, if this project is funded, the hypothetical postdoc would spend their time supporting and contributing to projects like iSEE (and interactive data viz in bioC more generally) in a formal way. Thinking of this application, what do you think of me listing your names as developers of iSEE as possible mentors to such a postdoc? If we make it through the pre-proposal round, then maybe a formal letter of support from the#iseeteam?
Kevin Rue-Albrecht (08:01:31): > happy to be mentioned as developer/possible mentor too:cry:
Stephanie Hicks (08:02:11): > shoot! sorry about the typo above. now fixed! Yes, that was my intention:slightly_smiling_face:
Charlotte Soneson (08:02:19): > That sounds good to me!
Kevin Rue-Albrecht (08:04:56): > no harm done:joy:we’re all around to advise users and contributors
Stephanie Hicks (08:05:42): > Thanks! I’ll update my section in the CZI application then
Kevin Rue-Albrecht (08:06:13): > Cool. Give us a shout if we need to proof-read anything!
Federico Marini (08:07:32): > Joining the rest of the band: probably this is the best solution, given our last minute involvement on top of our situations
Federico Marini (08:08:01): > But in any case, happy to keep on the stting up of instances on the HCA datasets
Stephanie Hicks (08:12:45): > :thumbsup:
Federico Marini (12:49:11): > Stickersssssssssss!
Federico Marini (12:49:43): - File (JPEG): IMG-20181107-WA0000.jpeg
Federico Marini (12:50:06): - File (JPEG): 20181107_181218.jpg
Federico Marini (12:52:50): > Now we are officially ready for EuroBioc2018:stuck_out_tongue:
Federico Marini (12:52:57): - File (JPEG): 20181107_184606.jpg
Aaron Lun (12:54:56): > Sweet
Kevin Rue-Albrecht (13:45:59): > :party_parrot:
Kevin Rue-Albrecht (13:48:25): > how’s the pixel quality of the data points? looks great, i think the picture resolution is the limit here right?
Federico Marini (14:08:12): > I have the impression it can’t be better
Federico Marini (14:08:37): > Downside, at least on some, is that the cut is not mega exact on each side of the hexagon
Kevin Rue-Albrecht (14:25:04): > Oh. That explains why I’ve seen it before for other stickers. I would have been surprised if it were a design flaw
2018-11-08
Federico Marini (04:35:35): > @Kevin Rue-Albrecht@Aaron Lun@Charlotte Soneson: who of you might want her/his own portion of stickers sent via mail before Munich takes place?
Federico Marini (04:35:56): > I’m happy to pack it up and send it over
Federico Marini (04:36:42): > I’d need a physical address then (up to you whether private address or institute)
Federico Marini (04:39:36): > (heads up: the “bundles” are about 56 stickers each, so I will need to open them up - hope it is not a big deal)
Aaron Lun (04:42:02): > I can wait until I see you.
Aaron Lun (04:42:07): > I’m not going to EuroBioC anyway.
Aaron Lun (04:42:14): > And I have to exchange my trousers with you.
Aaron Lun (04:42:29): > Okay, that last sentence needs#sc-batch-correctionfor context.
Federico Marini (04:46:46): > ahahah
Federico Marini (04:47:05): > btw, kudos forbatchelor
Federico Marini (04:47:46): > wouldn’t that be already perfect, you can have something alongscratchMyBatch
Kevin Rue-Albrecht (04:47:46): > First package with Aaron’s profile pic’ on the sticker
Charlotte Soneson (07:51:26): > I can probably wait for EuroBioc for my batch (although now I’m curious:stuck_out_tongue_winking_eye:). How are we doing with the payment?
Federico Marini (07:56:41): > I covered this round, we can just split it - once we meet we settle that
Federico Marini (07:56:53): > or bank transfer if you want
Federico Marini (07:56:55): > btw
Federico Marini (07:56:58): > they came from italy
Federico Marini (07:57:07): > so, no extra custom tax
Kevin Rue-Albrecht (12:18:47): > Speaking of payment… how abouthttps://cran.r-project.org/web/packages/RStripe/index.html?
Aaron Lun (12:19:57): > Make a custom panel, I think.
Federico Marini (13:06:02): > ahaha
Federico Marini (17:09:07): > can one HATE java?
Federico Marini (17:09:59): > damn JRE JDK this and that, and no way to understand why having JDK R still sees only JRE, even after javareconf
Kevin Rue-Albrecht (17:26:24): > I’m not going to make fun… I suffered too much at the hands of Java enthusiasts in R
Kevin Rue-Albrecht (17:26:38): > usually because the xlsx-something package
2018-11-09
Federico Marini (03:57:35): > writexl is the solution
Stephanie Hicks (13:42:20): > Hi everyone: this is my current summary in the CZI proposal: “Develop scalable, interactive graphics for millions of cells, including heatmaps and dendrograms, that is compatible with existing R/Bioconductor packages using the Interactive SummarizedExperiment Explore (iSEE) framework. The interactive graphics allow simultaneous examination of gene and cell specific meta data and analysis results, dynamic linking between plots and code tracking for reproducibility. We will leverage the use of memory-efficient, file-backed, containers to store the output, by loading into memory only the data summaries needed for each plot, rather than the full data.”
Stephanie Hicks (13:42:35): > Feel free to make edits here:https://docs.google.com/document/d/1aK8ZjGHVRoaojWb_SnfNCVltIbsVwQb7Cb5RoY50ffQ/edit#
Kevin Rue-Albrecht (13:50:25): > Thanks! Sounds good already. I’ll have a second look ASAP
Federico Marini (16:45:09): > How much space you have?
Federico Marini (16:45:21): > In short it sounds convincing already:slightly_smiling_face:
Federico Marini (16:45:42): > Feel free to throw in the sticker as well, if you want to!
Federico Marini (17:02:18): > I was wondering:https://f1000research.com/articles/7-741/v1#ref-17 - Attachment (f1000research.com): F1000Research Article: iSEE: Interactive SummarizedExperiment Explorer. > Read the latest article version by Kevin Rue-Albrecht, Federico Marini, Charlotte Soneson, Aaron T.L. Lun, at F1000Research.
Federico Marini (17:02:25): > does this have to be dated 2008?
Federico Marini (17:02:39): > or should this be 2018?
Kevin Rue-Albrecht (17:03:27): > what does or should be dated what?
Kevin Rue-Albrecht (17:03:49): > oh wait 2008/2018, didn’t see
Kevin Rue-Albrecht (17:04:50): > Uh. I don’t know. What 2008 a good R year?
Federico Marini (17:10:35): > I turned 1/4 of century that year
Federico Marini (17:10:39): > if that counts:stuck_out_tongue:
Stephanie Hicks (23:19:07): > @Federico Marininot too much more space unfortunately. The entire description of PIs has to be under 750 and I believe we are around 700ish.
Stephanie Hicks (23:19:21): > but happy to add something that is super relevant
2018-11-10
Charlotte Soneson (02:24:30): > Hi@Stephanie Hicks, I think your description looks great. I made a couple of minor edits (is -> are, Interactive -> interactive, explore -> explorer).
Stephanie Hicks (05:50:50): > Thank you!
2018-11-12
Stephanie Hicks (11:46:22): > quick question, would it be OK if we include this link (https://marionilab.cruk.cam.ac.uk/iSEE_tcga/) as a demo for the czi application?
Kevin Rue-Albrecht (11:52:57): > I think the demos on the MarioniLab server are probably the most readily available examples to toy with. Perhaps you might prefer the PBMC4k, as an example of scRNA-seq ?https://marionilab.cruk.cam.ac.uk/iSEE_pbmc4k/
Stephanie Hicks (11:53:10): > perfect, thanks!
Kevin Rue-Albrecht (11:54:11): > Here is the home page:https://marionilab.cruk.cam.ac.ukThere are other two examples, including a CyTOF example to show off with measurements for 172,791 cells - Attachment (marionilab.cruk.cam.ac.uk): Marioni Shiny apps > Shiny applications from the Marioni group
Kevin Rue-Albrecht (11:54:23): > (speaking of scaling up)
Vince Carey (14:30:24): > Around the time I started to learn about iSEE I had the notion that crosstalk might be useful for some comparative visualizations. The current crosstalk-based package is atgithub.com/vjcitn/pcmp, and an illustrative video is athttps://www.youtube.com/watch?v=vMy2lFQTO-U… it isn’t anywhere near as flexible as iSEE and the approach to differential expression is utterly naive … but as a sanity check it has been useful. Information on selections is collected in .GlobalEnv and then pushed into an object. - Attachment (YouTube): pcmpMur3k
Federico Marini (15:34:51): > Nice one. Back then before we implemented thebrush anything into anything, crosstalk was also one of the options we had in mind
2018-11-13
Kevin Rue-Albrecht (16:49:16): > I got the green light to apply for a travel scholarship for bioc2019 today. To earn the scholarship, I’m tempted to submit iSEE as a workshop. They are short (50-minute) and long (1h 45m) formats. What do you think?
Aaron Lun (17:27:53): > Go for it. You should deliver an epic 2 hour talk on iSEE customization.
Aaron Lun (17:28:02): > On hindsight, probably go with the 50 minutes.
Federico Marini (17:29:12): > Totally favoring any such spotlight
Federico Marini (17:29:27): > 50 minutes is probably the sweet spot?
Federico Marini (17:29:38): > unless we come up with enough custom panels
Kevin Rue-Albrecht (17:29:42): > I’ll think about it overnight. 50min could be enough.. though wedohave 10 panel types at the last count:stuck_out_tongue_winking_eye:
Federico Marini (17:30:20): > and/or you do something along the “let’S take these published data, make the plots of that, and make even a small tour of this”
Aaron Lun (17:31:45): > custom panels, etc. etc.
Federico Marini (17:35:05): > maybe the 1.45h is indeed the better one after all
Federico Marini (17:35:22): > say that by then we even have something based on epivizr in that
Federico Marini (17:40:47): > Did anyone of you already get the warningWarning: multiple methods tables found for 'lengths'
Kevin Rue-Albrecht (17:40:51): > if I ever figure out how to efficiently use epivizr:sweat:
Federico Marini (17:40:58): > (or whatever function)
Kevin Rue-Albrecht (17:41:29): > Uh. Sounds like a method dispatch issue, though I can’t remember seeing this kind of error recently
Federico Marini (17:42:13): > Given the fact I do not use at all that func, it seems something troubling probably the dependencies I am touchin
Jayaram Kancherla (17:42:59): > I’d be more than happy to help.@Federico Marinigave me the repo link to custom panels. I’ll take a look and take a first stab at creating a custom panel. (will prob have a lot of questions)
Federico Marini (17:45:52): > -> bringing it also here: at least in the README of iSEE, we should mention your iSEE_custom repo
Federico Marini (17:46:10): > for this “passive” way of calling out potential further developers:stuck_out_tongue:
Kevin Rue-Albrecht (17:47:01): > Indeed I was also wondering about a better way to “link” our GH repos to one another (iSEE, iSEE2018, iSEE_custom, iSEE_instances)
Federico Marini (17:47:45): > A subsection for something like## Want more?
Kevin Rue-Albrecht (17:55:50): > @Jayaram KancherlaHappy to see examples whenever you get a chance. Paradoxically, I don’t find as much time as I’d like to play with interactive visualisation tools, and I haven’t found a place for epiviz in my projects yet. The ChIP-seq data that I have tends to finish in metagenes and heatmaps at the moment.. not much genomic track visualisation.
Kevin Rue-Albrecht (17:59:49): > Off to sleep. Catch up tomorrow!
Stephanie Hicks (19:54:02): > great idea@Kevin Rue-Albrecht!
2018-11-14
Charlotte Soneson (01:56:18): > @Kevin Rue-AlbrechtI agree, a workshop sounds like a great idea!
Kevin Rue-Albrecht (03:00:22): > Thanks for the feedback everyone. I’ll start drafting a workshop template (https://github.com/Bioconductor/BioC2019/blob/master/docs/workshop-syllabus.md). Keep an eye on#bioc2019BTW. > Also, whenever anyone knows they’ll go too, I’m happy to collaborate on the workshop. In particular, it would be awesome if the workshop could provide instances for a range of data format, sources, and usages (e.g. respectively, RNA-seq, HCA portal, comparison of batch correction methods). That said, the workshop should start by covering the basics (SummarizedExperiment, SingleCellExperiment, Shiny). - Attachment (GitHub): Bioconductor/BioC2019 > BioC2019: Where Software and Biology Connect. Contribute to Bioconductor/BioC2019 development by creating an account on GitHub.
Federico Marini (10:21:03): > I’d have a feature request proposal
Federico Marini (10:21:30): > We all like reproducibility of all the plots in a snapshot, but sometimes we do lots of plots
Federico Marini (10:21:37): > can we do one of the following:
Federico Marini (10:21:55): > - small download button to each plot, on-the-fly
Federico Marini (10:21:57): > or
Federico Marini (10:22:33): > - storing a list of ggplot objects, thenggsaved upon user request
Federico Marini (10:24:15): > Feel free to call it nonsense as well
Federico Marini (10:25:01): > it came to me as a feedback in form “i’d like to save this and this and this”, and I formulate the issue to you in two potentially viable ways
Federico Marini (10:26:30): > Other thing: did you see the new release of shiny? It has plot caching in it:slightly_smiling_face:
Aaron Lun (10:35:26): > can we use it?
Aaron Lun (10:36:37): > The problem is that these things require transfer of data.
Aaron Lun (10:36:56): > If it’s running on a server, you need to make sure that the objects are transferable.
Aaron Lun (10:37:15): > The only generalizable way would be to have a pop-up box with downloads of files or objects.
Aaron Lun (10:37:24): > Probably objects, then they could fiddle around with it.
Federico Marini (10:39:48): > I did not read into detail how it works as I checked it on twitter a few minutes ago while I was moving to my office:stuck_out_tongue:
Federico Marini (10:40:43): > but back to the feature request: does it make sense to you as well?
Kevin Rue-Albrecht (10:42:43): > As a feature, it does. We just need to make sure it works for both web-servers and local apps. that said, adding a “Save plot” button for each panel could be done as an extra icon in the sidebar, as part of theforloop.
Federico Marini (10:44:12): > I was thinking the same: keepin it compact, and kind of automated
Federico Marini (11:25:04): > (are you all getting the warning for the multiple tables?)
Aaron Lun (11:26:47): > just reinstall everything.
Aaron Lun (11:27:07): > It’s because some of your packages are usingBiocGenerics::lengthbut that’s reverted back to the same generic asbase::length
Federico Marini (11:27:36): > uh thanks for spotting it
Federico Marini (11:27:46): > everything everything?:stuck_out_tongue:
Aaron Lun (11:28:29): > Probably everything dependent onBiocGenerics.
Aaron Lun (11:28:33): > As well asBiocGenericsitself.
Aaron Lun (11:28:44): > Maybe justBiocGenericswould also work.
Federico Marini (11:28:55): > Ok I’mmatr
Federico Marini (11:28:56): > y
Federico Marini (11:35:34): > I was asking because now it also pops up for checking iSEE
Aaron Lun (11:36:09): > Are youBiocManager::valid()?
Federico Marini (11:37:36): > couple out of date now, updating
Federico Marini (11:55:51): > valid, but still poppin up
Aaron Lun (12:01:51): > Well, reinstall BiocGenerics.
Federico Marini (12:02:10): > did that as well already
Aaron Lun (12:02:20): > Then your thing is fucked I think.
Federico Marini (12:02:26): > probably nothing left but reinstall the whole thingo
Federico Marini (12:02:34): > :disappointed:
Aaron Lun (12:02:39): > I don’t get any warnings.
Aaron Lun (12:02:44): > Uponlibrary(iSEE), I assume.
Federico Marini (12:02:52): > Yeps
Federico Marini (12:03:15): > Travis is still choking
Federico Marini (12:04:16): > I was working on some product placement updates - logo in the readme, title (upper left corner), and in the modal
Federico Marini (12:05:12): > (plus, pointing towards the custom shop)
Federico Marini (12:07:48): > (now there for you to take a look in the PR)
Aaron Lun (12:09:18): > huh
Kevin Rue-Albrecht (17:33:04): > I’m just catching up with some package updates for 3.9 and I’ll take Fede’s PR for a spin
Kevin Rue-Albrecht (18:47:15): > Placement of choice:sunglasses: - File (JPEG): Image from iOS
2018-11-15
Federico Marini (03:05:11): > Well placed:smile:
Federico Marini (03:05:32): > as if did not already stand out with the white color:smile:
Kevin Rue-Albrecht (03:21:09): > “illuminating data” suddenly made complete sense when the logo turned on:grin:
Federico Marini (05:18:10): > Damn you Apple you took away my bright apple from the lid
Stephanie Hicks (05:29:21): > @Kevin Rue-Albrechtyou should put that image in your next talk about iSEE !
Kevin Rue-Albrecht (05:34:22): > Great idea@Stephanie Hicks. So far I had this cover slide. Not sure whether to lead or to finish with the laptop one now! - File (PNG): Presentation1_KRA.png
Stephanie Hicks (05:58:58): > Lead, definitely lead with the laptop. Adds humor and a sense of honesty that will relax your audience. Plus this is better to finish with so people will take pictures of the slide from the audience as you are answering Qs
Kevin Rue-Albrecht (06:02:08): > Riiiiight. Experience talking, thanks!
Stephanie Hicks (06:12:59): > :joy_cat:
Aaron Lun (09:31:12): > Mike was trying to set up the intiial panels, and he was getting confused between the encoded and decoded names ininitialPanels. Perhaps worth putting a note about it and the translations in the vignette.
Kevin Rue-Albrecht (09:33:40): > That was a bit my own motivation for having thepanelTypesandpanelCodesand associated help page
Kevin Rue-Albrecht (09:33:48): > but vignette is a good idea too
Aaron Lun (09:37:07): > Yes, maybe a brief mention in the vignette at that point would be sufficient.
Aaron Lun (09:46:32): > Man, BatchtoolsParam is amazing. Simply amazing.
Kevin Rue-Albrecht (11:00:50): > Forgot to say: I got it to work on our cluster. Didn’t need much more than launching themungeSLURM daemons. Though the last hurdle being that it only runs from a Shell R session, not within RStudio server for some reason. I think an executable not on the PATH in the RStudio environment or something like that.
2018-11-16
Federico Marini (02:41:07): > the new dream: using iSEE hands free:slightly_smiling_face:
Federico Marini (02:41:09): > https://www.jumpingrivers.com/blog/voice-control-your-shiny-apps/ - Attachment (Jumping Rivers): Voice Control your Shiny Apps - Jumping Rivers > An introduction to combining Shiny with voice control.
Charlotte Soneson (02:43:19): > There was also this one already some time ago:https://yihui.shinyapps.io/voice/
Federico Marini (02:43:34): > I do remember that as well!
Federico Marini (02:43:43): > for making points bigger and so on right?:stuck_out_tongue:
Charlotte Soneson (02:43:55): > yeah - change title, colors etc
Federico Marini (02:46:24): > “iSEE - select cluster 6”
Federico Marini (02:46:38): > “iSEE - perform DE, 6 vs 13”
Federico Marini (02:46:50): > “iSEE - write up material and methods”
Charlotte Soneson (02:47:21): > :+1:
Kevin Rue-Albrecht (04:28:52) (in thread): > “… in reduced dimension panel 3”
Kevin Rue-Albrecht (04:29:16) (in thread): > I’m speechless at the idea that it sounds kinda feasible
Kevin Rue-Albrecht (04:31:35) (in thread): > although I suspect that speech recognition won’t like column names that include dots, hyphens or underscores
Federico Marini (04:32:15): > Shall we put up a mock implementation for doing just one thing and show that @eurobioc2018?:smile:
Federico Marini (04:33:05): > Risk is to turn down people after you show that this one thing worked:stuck_out_tongue:
Kevin Rue-Albrecht (04:34:04): > To be honest, the mere idea just overrode my memories of any weekend plans:sweat_smile:Calming down, I remember that we’re going out tomorrow evening, but I should have time during the day to try a minimal example:smile:
Kevin Rue-Albrecht (04:34:57) (in thread): > On second thought, for the selection of cluster, it’s probably best to store it as a colData and select it there
Kevin Rue-Albrecht (04:35:28) (in thread): > I don’t really want to be the one implementing the voice-generated lasso
Kevin Rue-Albrecht (04:37:18) (in thread): > “iSEE - select the cluster of blue points roughly at (+10, -5), yeah, draw the lasso through here between the green and pink points, no actually a little bit more to the left..”
Federico Marini (04:37:40) (in thread): > to the left to the left:musical_note:
Federico Marini (04:37:47) (in thread): > and beyonce plays in the background
Federico Marini (04:38:46): > If you are not joking: cluster up the allen, store the colData, and have the voice activate the selection for one cluster - no more than that
Federico Marini (04:39:22): > (or we fake it with an animation:grinning_face_with_star_eyes:)
Kevin Rue-Albrecht (04:40:04): > I wasn’t. I’ve just been a bioinformatician for too long already and worry that the advertising is “it should just work out of the box” ends up in days of debugging:wink:
Kevin Rue-Albrecht (05:06:53): > From a very brief look at Yihui’s example code, it sounds like we’ll “just” have to create another command store as a reactive value that gets populated upon speech, and clear that command store upon evaluation.
Kevin Rue-Albrecht (07:21:58): > …. something line that? …:kissing::musical_note: > > > shiny::runApp(app, port = 1234, launch.browser=FALSE) > Loading required package: shiny > > Listening on[http://127.0.0.1:1234](http://127.0.0.1:1234)Voice add panel: > Voice add panel: reduce dimension panel one > Voice add panel: reduced dimension panel one >
Kevin Rue-Albrecht (07:27:50): > (6 lines of code and 2 extra *js files in the www directory)
Federico Marini (07:29:16): > Is it already running?:stuck_out_tongue:
Federico Marini (07:29:50): > Over lunch break we have been fantasying about navigable 3d tSNE plots in VR space
Kevin Rue-Albrecht (07:29:52): > Basically yes, although I’m just printing a message to the console at the moment.
Federico Marini (07:30:16): > We can just add a trigger that does something if the msg is what we expect
Federico Marini (07:30:21): > a la easter egg:smile:
Kevin Rue-Albrecht (07:31:22): > Well, from the example above, you see that we’ll probably need to allow some “nearest match” between the speech and the expected keywords (e.g. “reduce” –> “reduced”)
Kevin Rue-Albrecht (07:31:33): > and numbers converted from text to digits
Federico Marini (07:31:58): > Well I am very fine with “select cluster five”, and cluster five gets highlighted
Federico Marini (07:32:15): > no need for a real real implementation. YET.
Kevin Rue-Albrecht (07:33:10): > Although, at the moment, the trigger is “add” followed by the name of the panel to show. Which made me realise that we’re not really “adding” panels to the UI, but rather showing/hiding them. Because even hidden, they’re always present in thememory
Kevin Rue-Albrecht (07:33:45): > Short point is: we’ll need to set up a proper lexicon of keywords
Kevin Rue-Albrecht (07:37:40): > So, before diving deep into the fairly advancedselect <value> in <column> in <table panel name>,color <panel name> by [featureName|sampleName](colData), I’d rather start withshow <panel name>andhide <panel name>
Kevin Rue-Albrecht (07:38:39): > That’ll already provide a good MWE to sort out a minimal “voice autocorrect”
Kevin Rue-Albrecht (07:39:29): > Alternative being having aliases to make the panel names a bit shorter to pronounce:stuck_out_tongue:
Kevin Rue-Albrecht (07:41:03): > … Imma have a fun weekend:smile:…
Federico Marini (07:41:57): > Don’t get yourself killed from the lady, please:slightly_smiling_face:
Federico Marini (07:42:10): > do pay attention to her as well!
Kevin Rue-Albrecht (07:42:12): > Btw, let’s try to keep the news of this upcoming feature to this channel, to make a better surprise when it’s officially released
Kevin Rue-Albrecht (07:43:18): > Ah and separate note, it really only seems to work with Chrome. I’ve tried Firefox as well as Safari
Kevin Rue-Albrecht (07:47:53): > I’m on the lady’s good side for a little while: Co-authored Nat. Comm. paper accepted and scheduled for publication on the 19th:innocent:
Federico Marini (07:51:29): > yay very good
Federico Marini (07:51:39): > did not know she was also in the same business
Kevin Rue-Albrecht (07:52:22): > Experimental side. Anna Aulicino in Alison Simmons group in the WIMM, if you wanna look it up.
Federico Marini (07:52:45): > thx
Aaron Lun (07:54:10): > What? you publish with your wife?
Federico Marini (07:54:57): > Damn, Aaron. We cannot lure you out with voice-controlled iSEE, but this wakes you up:stuck_out_tongue:
Kevin Rue-Albrecht (07:55:29): > Sure we’re not the first ones:stuck_out_tongue:
Aaron Lun (07:56:30): > Damn, if I had a experimentalist wife, I could just pump out papers all the time.
Kevin Rue-Albrecht (07:56:52): > it’s started with a … well …. “I can have a look at the QC this weekend”
Federico Marini (07:56:57): > Go get her:tiger2:
Federico Marini (07:57:00): > eheheh
Aaron Lun (07:57:02): > Rowwr
Federico Marini (07:57:08): > ahahahah
Federico Marini (07:57:26): > “maybe I can show you my collection of QC failures”
Kevin Rue-Albrecht (07:57:47): > To be fair, she’s holding the higher power: homemade italian food
Kevin Rue-Albrecht (07:57:55): > I better generate those plots, I tell you
Kevin Rue-Albrecht (07:59:01): > apparently the punishment is pasta aglio e olio, is that right Fede?
Federico Marini (07:59:29): > and no peperoncino as punishment
Federico Marini (07:59:34): > eheh
Aaron Lun (07:59:37): > chicken + pineapple pizza FTW
Kevin Rue-Albrecht (08:00:02): > :man-running::hammer::woman-running::flag-it:
Kevin Rue-Albrecht (08:05:05): > PS: iSEE indirectly owes her theExperimentColorMap. You have no idea how many times I’ve cried about synchronising the color-coding of conditions across all 8 figures and 16 supp figures
Aaron Lun (08:05:41): > And she got fed up with your whining and just said “Godammit Kevin just do it”?
Kevin Rue-Albrecht (08:06:25): > That’s about the picture of it:stuck_out_tongue:
Kevin Rue-Albrecht (09:19:49): > @Federico Marini: I just check again: with the EuroBioc208 on the 6-7 Dec, and my Oxford demo on the 11th, there’s some good motivation to get a couple of demo voice functions on the devel branch
Aaron Lun (09:29:25): > Before you all get too excited - do we really need this in the main branch?
Kevin Rue-Albrecht (09:30:03): > well it’ll obviously start on a side branch, and from then we’ll see if we want it on themasterone
Kevin Rue-Albrecht (09:30:24): > But I’ve already though thatiSEE(..., voice=FALSE)should probably be default
Aaron Lun (09:32:14): > This sounds pretty gimmicky.
Aaron Lun (09:32:32): > Especially if it messes up the internals.
Aaron Lun (09:34:16): > I mean, I don’t mind seeing it in play for your presentation, but I can’t see how it would be useful in general.
Kevin Rue-Albrecht (09:41:01): > what do you call the internals here?
Kevin Rue-Albrecht (09:42:32): > I’ll push the code to GH after testing, but I’m pretty sure that I can only parse the JS scripts whenvoice=TRUE, and then the whole app would behave as it does right now
Aaron Lun (09:43:02): > The shiny infrastructures, observers, etc.
Kevin Rue-Albrecht (09:45:57): > nah.. all this thing does is add extra elements ininput$..., one for each voice command.
Kevin Rue-Albrecht (09:49:12): > given that none of the existing observers look at those new ones, it shouldn’t affect anything
Kevin Rue-Albrecht (09:50:42): > But of course, the new observers will start updating thememorythemselves (e.g.,...ColorByColData <- "field", which in turn should simply trigger the usual mechanisms of replotting
Kevin Rue-Albrecht (09:51:12): > Anyway, I’ll look at it properly tonight/tomorrow and push it on a side branch
Aaron Lun (09:51:25): > Give me an example page.
Kevin Rue-Albrecht (09:51:46): > ?
Kevin Rue-Albrecht (09:52:00): > btw@Federico Marini: i’m currently on theextrainfobranch. Looks ready to merge, isn’t it?
Aaron Lun (09:52:01): > An example instance, the one on the page doesn’t work for me.
Kevin Rue-Albrecht (09:52:39): > ah yeah yeah, i’ll push something. I’m just fixing some other stuff right now
Aaron Lun (09:53:13): > No, I mean, an example instance of ANY shiny app using this voice control.
Aaron Lun (09:53:17): > not iSEE.
Kevin Rue-Albrecht (09:53:22): > ah
Aaron Lun (09:53:28): > Are you looking at the one where you have to say JUMPING RIVERS
Aaron Lun (09:53:30): > because it doesn’t work.
Aaron Lun (09:53:35): > I don’t know what it’s meant to do.
Kevin Rue-Albrecht (09:54:08): > that one works on Chromehttps://yihui.shinyapps.io/voice/
Kevin Rue-Albrecht (09:55:13): > just say “title something”
Kevin Rue-Albrecht (09:56:41): > the title of the plot should just update to “something”
Aaron Lun (09:56:57): > Doesn’t work for me.
Kevin Rue-Albrecht (09:57:05): > is it the accent?:stuck_out_tongue:
Kevin Rue-Albrecht (09:57:42): > btw, the first time you load the page, Chrome should throw a pop up asking for access to the microphone
Kevin Rue-Albrecht (09:58:07): > otherwise it probably means you’ve got a security setting somewhere
Aaron Lun (09:58:35): > I’ve already allowed it.
Kevin Rue-Albrecht (09:59:31): > Ok, I can’t help you more right now. I need to work with someone for a bit
Aaron Lun (10:01:19): > Chrome doesn’t even see my mic.
Aaron Lun (10:02:00): > Had enough of this gimmicky crap.
Federico Marini (10:02:34): > :stuck_out_tongue:
Federico Marini (10:02:57): > As I said, I am happy with the PoC for “select cluster 5”
Kevin Rue-Albrecht (15:51:55): > Alright, it’s a boringmessagein the terminal, but checkout branchvoice, build, and in a terminal R session run the typical examples amended withlaunch.browser = FALSEbefore openingChrome: > > library(iSEE) > library(scRNAseq) > data(allen) > class(allen) > > # Example data ---- > > library(scater) > sce <- as(allen, "SingleCellExperiment") > counts(sce) <- assay(sce, "tophat_counts") > sce <- normalize(sce) > > sce <- runPCA(sce, ncomponents=4) > #sce <- runTSNE(sce) > rowData(sce)$ave_count <- rowMeans(counts(sce)) > rowData(sce)$n_cells <- rowSums(counts(sce)>0) > sce > > # launch the app itself ---- > > app <- iSEE(sce) > if (interactive()) { > shiny::runApp(app, port = 1234, launch.browser = FALSE) > } > > Then say “show panel “, and keep an eye on the terminal
Kevin Rue-Albrecht (15:53:35): > (I can’t wait for the hate messages coming from people working in open-space offices)
Kevin Rue-Albrecht (15:54:38): > Don’t expect the UI to do anything, I haven’t plugged anything together
Kevin Rue-Albrecht (17:38:37): > Someone needs to teach me how to pronounce “assay” properly..:cry:
2018-11-17
Kevin Rue-Albrecht (03:51:07): > btw, the app reacts to “show panel ” and “hide panel “. > e.g. “show panel reduced dimension plot two” > I suggest the following startup script for best effect: > > library(iSEE) > library(scRNAseq) > data(allen) > class(allen) > > # Example data ---- > > library(scater) > sce <- as(allen, "SingleCellExperiment") > counts(sce) <- assay(sce, "tophat_counts") > sce <- normalize(sce) > > sce <- runPCA(sce, ncomponents=4) > sce <- runTSNE(sce) > rowData(sce)$ave_count <- rowMeans(counts(sce)) > rowData(sce)$n_cells <- rowSums(counts(sce)>0) > > # Minimal initial panels, for best effect ---- > > initialPanel = DataFrame(Name="Reduced dimension plot 1", Width=4, Height=200) > > # launch the app itself ---- > > app <- iSEE(sce, initialPanels = initialPanel) > if (interactive()) { > shiny::runApp(app, port = 1234, launch.browser = FALSE) > } >
Kevin Rue-Albrecht (03:52:01): > However, heading into the next level of command (e.g.color panel <reduced dimension plot two> by <column data>), I’m running into an issue whereby only a single “splat” (aka wildcard) by command is supported, because it is greedy. > > Use splats to capture multi-word text at the end of your command (greedy). > https://www.talater.com/annyang/Basically, I’ve tried'color panel *panel by *title'as a pattern, and it just won’t trigger - Attachment (annyang): annyang! Easily add speech recognition to your site > annyang is a JavaScript SpeechRecognition library that makes adding voice commands to your site super-easy. Let your users control your site with their voice.
Kevin Rue-Albrecht (07:30:27): > Alright, I’ve refactored a bit of code, and I now dare call thevoicebranch “stable” for the addition/removal of panels
Kevin Rue-Albrecht (07:31:10): > did you sort out the gimmick@Aaron Lun? Sorry that I could only reply sporadically yesterday
Kevin Rue-Albrecht (09:11:08): > To save some time, I’ve just thrown in slider inputs to control the width and height of new panelsbeforethey get added to the UI. I don’t really like having to insert it first, go into the settings modal, and then resize it by row and column. It’s 4 UI refreshes compressed into 1
Kevin Rue-Albrecht (09:22:46) (in thread): > Now that I have the mechanics for a simple example (show/hide panels), I’ll point out that this example is actually misleading. > In our current setup, what you describe as"highlight cluster 5 in panel reduced dimension plot 1"actually means"select value <5> in column <cluster> in panel <column data plot 1>"followed by"highlight selection from panel <column data plot 1> in panel <reduced dimension plot 1>"
Kevin Rue-Albrecht (09:35:45): > While I’m in the web inspector, I just spotted that upon initialization, there’s an error about > > Error evaluating expression: (input["redDimPlot1_VisualChoices"].includes("Color")) >
Kevin Rue-Albrecht (09:36:18): > looks like something that should already exist on themasterbranch
Kevin Rue-Albrecht (10:23:33): > ah actually, i think it’s because it’s trying to access it before it’s initialized > > VM38:7 Error evaluating expression: (input["redDimPlot1_VisualChoices"].includes("Color")) > (anonymous) @ VM38:7 > VM38:8 Uncaught TypeError: Cannot read property 'includes' of undefined > at Object.eval (eval at scopeExprToFunc (utils.js:176), <anonymous>:5:55) > at utils.js:193 > at ShinyApp.$updateConditionals (shinyapp.js:420) > at ShinyApp.dispatchMessage (shinyapp.js:517) > at WebSocket.c.onmessage (shinyapp.js:112) >
Aaron Lun (10:47:31): > I can’t test any of this voice stuff.
Kevin Rue-Albrecht (10:47:37): > owww
Aaron Lun (10:47:50): > Chrome doesn’t like it.
Aaron Lun (10:47:55): > Firefox doesn’t even know about it.
Kevin Rue-Albrecht (10:47:58): > i’m having so much fun setting up the helper functions
Kevin Rue-Albrecht (10:48:10): > For me, Chrome is the only one that works
Kevin Rue-Albrecht (10:48:40): > (following some documentation/forum that I don’t feel like tracking down right now)
Aaron Lun (10:49:32): > As I said before, I don’t mind it being on a separate branch, but I don’t know whether its inclusion is worth the maintenance burden onmaster.
Aaron Lun (10:49:43): > It seems like something with not much practical application to me.
Kevin Rue-Albrecht (10:49:48): > That said, Fede’s dream of “select cluster 5” is not around the corner, as per thread - Attachment: Attachment > Now that I have the mechanics for a simple example (show/hide panels), I’ll point out that this example is actually misleading. > In our current setup, what you describe as "highlight cluster 5 in panel reduced dimension plot 1" actually means "select value <5> in column <cluster> in panel <column data plot 1>" followed by "highlight selection from panel <column data plot 1> in panel <reduced dimension plot 1>"
Kevin Rue-Albrecht (10:51:27): > well, I agree with you that I do not intend rushing amergetomaster, but i’m happy to play with the idea for a bit on the side branch, and then decided if we’d like to let users toy with it as an optional feature toggled on/off in theiSEEcall
Kevin Rue-Albrecht (10:52:14): > however, I have a feeling that it won’t be practical for everyday use
Kevin Rue-Albrecht (10:52:44): > and probably require even more patience and expert knowledge of our panel terminology to get it working (as a user)
Kevin Rue-Albrecht (10:53:40): > That said, it did motivate some refactoring today. Check outhttps://github.com/csoneson/iSEE/compare/voice?expand=1#diff-cf84cd4eda1cf12ba2ff41198d758a14R61 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Aaron Lun (10:54:05): > I don’t mind you developing it for the BioC talk. It would probably blow people away as a flashy presentation tool. But it’s basically useless IRL. No one does hands-free analysis like that.
Kevin Rue-Albrecht (10:54:27): > Definitely not in open-space offices … (thereby excluding >80% of academics)
Kevin Rue-Albrecht (10:57:00): > btw, as part of my reading, I fell onto the GUI version of the voice recognition (https://github.com/TalAter/SpeechKITT), which gives a button to turn on/off at runtime, and also displays more nicely the words recorded, but I can’t test it on localhost because it runs only with HTTPS:confused: - Attachment (GitHub): TalAter/SpeechKITT > :speaking_head_in_silhouette: A flexible GUI for Speech Recognition. Contribute to TalAter/SpeechKITT development by creating an account on GitHub.
Aaron Lun (11:20:26): > SO BORED
Aaron Lun (11:20:31): > Grinding through updates toFurtherNorm2018.
Kevin Rue-Albrecht (11:31:49): > Well, I’m about to head out with the miss’s, I’m just about 20min away from finishing"<Update panel> [Reduced dimension plot 1] field [ColorBy] to value TODO". I’ll finish tomorrow morning I think, with a refactoring of the other two vocal commands so far, which all copy the same chunk of code to parse the vocal command and determine which panel to work on
Kevin Rue-Albrecht (11:36:26): > perhaps more than 20min now i that i think of it: valid choices for the last fieldvaluedepend on thefield. arf
Kevin Rue-Albrecht (11:36:38): > good moment to take a break
Kevin Rue-Albrecht (18:45:14): > - “Show panel column data plot 3” > - “Update panel column data plot 3 option color by to value column data” > at least that’s what I said.. - File (PNG): Screenshot 2018-11-17 23.43.44.png
2018-11-18
Kevin Rue-Albrecht (05:19:54): > (top message is the raw parsed one, bottom is after ‘magical’ matching to possible choices)
Kevin Rue-Albrecht (05:22:37): > that said, today I’ll already refactor this. It’swaytoo painful to say the whole sentence correctly.
Kevin Rue-Albrecht (05:24:43): > I’ve already added “memorisation” of the latest panel handled, which means my next move is to have the following succession of voice commands: > - “show panel CDP1” or “control panel CDP1” > - “open visual parameter box” (with the app doing it on RDP1) > - “color by ….” (with the app using the current “ColorBy” field to know which choices are possible)
Kevin Rue-Albrecht (07:52:59): > Voice: “Show panel […]” > Result: > - transiently show recorded voice > - transiently show interpreted command > - show if not visible yet > - set as “active panel”, which means it will be available for follow-up commands > - persistently show the current “active panel” - File (PNG): Screenshot 2018-11-18 12.51.38.png
Kevin Rue-Albrecht (07:59:38): > Voice: “control panel […]” > Result > - transiently show recorded and interpreted commands > - if the panel is visible, update “active panel’ to that one - File (PNG): Screenshot 2018-11-18 12.53.50.png
Kevin Rue-Albrecht (08:00:06): > Tell the user if he’s trying to take control of a panel that is not visible - File (PNG): Screenshot 2018-11-18 12.54.04.png
Federico Marini (08:05:24): > Referring to Aaron’s msg: > > It would probably blow people away as a flashy presentation tool
Federico Marini (08:06:03): > That would be my primary aim actually - we should not forget we want the highlight the tool as it is now, and how it concretely has to be used
Federico Marini (08:06:40): > Don’T want to kill anyone’s enthusiasm:stuck_out_tongue:
Federico Marini (08:10:59): > I’ve got to say still: it is already in the mindblowing stage I could use for munich
Federico Marini (08:11:04): > :slightly_smiling_face:
Federico Marini (08:11:53): > We have guests now at home and they just asked me if I am crazy
Kevin Rue-Albrecht (08:12:02): > hahaha
Kevin Rue-Albrecht (08:12:08): > you should try what i just pushed
Kevin Rue-Albrecht (08:12:34): > “show panel …“, “hide panel …“, “control panel …”
Kevin Rue-Albrecht (08:13:04): > try “… heat map 1” , it’s the easiest voice voice recognition
Federico Marini (08:13:38): > I managed to get reduced dimension plot:stuck_out_tongue:
Federico Marini (08:18:02): > iSEEwhat’s on your menu today!
Charlotte Soneson (08:18:56): > Haha, this is so cool. I managed to get it to understand the reduced dimension plot too
Federico Marini (08:19:04): > YAY:slightly_smiling_face:
Federico Marini (08:19:25): > tested with italian and scandinavian accent::white_check_mark:
Charlotte Soneson (08:20:26): > “Feature assay plot 1” was heard as “feature acid +1”, but it understood it anyway
Federico Marini (08:22:09): > :stuck_out_tongue:
Federico Marini (08:22:25): > we should probably rename the repo toiHEAR
Federico Marini (08:22:38): > and make it a standalone project
Charlotte Soneson (08:22:39): > Now we need to expand it to all the senses…
Federico Marini (08:23:28): > lemme suggest the next step
Charlotte Soneson (08:23:39): > Or very generally,iFEEL
Federico Marini (08:23:41): > I think I saw it on biorXiv recently
Federico Marini (08:23:43): > ooouh
Charlotte Soneson (08:23:46): > Which automatically senses what you want to do
Federico Marini (08:23:58): > please do not think of a sticker for that already:slightly_smiling_face:
Charlotte Soneson (08:24:15): > :stuck_out_tongue:
Federico Marini (08:25:24): > https://www.biorxiv.org/content/early/2018/04/25/307769 - Attachment (bioRxiv): BioVR: a platform for virtual reality assisted biological data integration and visualization > Background: The effective visualization and presentation of biological data is of critical importance to research scientists. The increasing rate at which experiments generate data has exacerbated the visualization problem. While biological datasets continue to increase in size and complexity, the shift to adopt new user interface (UI) paradigms has historically lagged. Consequently, a major bottleneck for analysis of next-generation sequencing data is the continued use of the UIs primarily inspired from single-type and small-scale data in 1990’s. Results: We have developed BioVR, an easy-to-use interactive, virtual reality (VR)-assisted platform for visualizing DNA/RNA sequences and protein structures using Unity3D and the C# programming language. It utilizes the cutting-edge Oculus Rift, and Leap Motion hand detection, resulting in intuitive navigation and exploration of various types of biological data. Using Gria2 and its associated gene product as an example, we present this proof-of-concept software to integrate protein sequence and structure information. For any residue of interest in the Gria2 sequence, it can be quickly visualized in its protein structure within VR. Conclusions: Using innovative 3D techniques, we provide a VR-based platform for visualization of DNA/RNA sequences and protein structures in aggregate, which can be extended to view omics data.
Charlotte Soneson (08:25:51): > Right
Federico Marini (08:26:25): > fellows swedes did this
Federico Marini (08:26:27): > https://www.biorxiv.org/content/early/2018/05/24/329102 - Attachment (bioRxiv): CellexalVR: A virtual reality platform for the exploration and analysis of single-cell gene expression data > Single-cell RNAseq is a powerful tool for the dissection of cell populations at the transcriptome level, and a myriad of techniques are available to project cells on to 3-dimensional space to construct cellular maps that aid the visualisation of heterogeneity and any sub-populations formed. Current visualisation methods for 3-dimensional data on conventional computer displays are poor, and coupled with a lack of intuitive point/cell selection methods often hinders a rapid exploration of finer details contained in the data. Here we present CellexalVR (www.cellexalvr.med.lu.se), a feature-rich, fully interactive, and immersive virtual reality environment for the analysis of single-cell RNAseq experiments that allows researchers to quickly and intuitively gain an understanding of their data.
Federico Marini (08:26:54): > … or this
Federico Marini (08:26:55): > https://www.biorxiv.org/content/early/2018/05/17/324855 - Attachment (bioRxiv): starmap: Immersive visualisation of single cell data using smartphone-enabled virtual reality > We report a new smartphone-enabled virtual reality (VR) program, starmap (https://vccri.github.io/starmap/), that enables immersive visualisation of single-cell data for hundreds of thousands of cells using a mobile-enabled web browser and low-cost VR head mount device.
Federico Marini (08:27:05): > pokemon go applied to tSNE hunting
Charlotte Soneson (08:27:17): > :joy:
Federico Marini (08:32:35): > CATCH THAT CLUSTER!
Federico Marini (08:32:45): > cluster two, I CHOOSE YOU!
Kevin Rue-Albrecht (08:32:46): > a little fun incoming
Kevin Rue-Albrecht (08:32:51): > 20% transferred
Federico Marini (08:33:11): > kev, gimme an easter egg implementation with this
Kevin Rue-Albrecht (08:33:13): > (damn it’s hard to get files out of iPhone into the laptop)
Kevin Rue-Albrecht (08:33:27): > I’ll give you better, wait for it:wink:
Kevin Rue-Albrecht (08:33:32): > 64% …
Federico Marini (08:33:38): > if the message is exactly this, then it selects cluster two
Kevin Rue-Albrecht (08:34:02): > Works? - File (QuickTime Movie): iSEE_480.mov
Federico Marini (08:34:19): > I’ll fly over the comments I a m hearing
Kevin Rue-Albrecht (08:35:00): > I hope the video above works for you guys too. Works for me
Federico Marini (08:35:28): > You know this will end up in the presentation, right?:smile:
Kevin Rue-Albrecht (08:36:49) (in thread): > I’ll take it that the video works
Federico Marini (08:39:33) (in thread): > video works
Kevin Rue-Albrecht (08:39:55) (in thread): > cool, it’s crepe-time then !
Kevin Rue-Albrecht (08:43:29) (in thread): > At the moment, I’ve allowed 5 edits between the recorded voice and the panel name, with a partial match allowed (i.e. “reduced dimension one” should work too)
Kevin Rue-Albrecht (08:44:54) (in thread): > it’s a bit dangerous, and it makes “row/column statistics table” tricky, as it doesn’t capture “row” very well, and therefore the matching ends up using the common “statistics table” part
Kevin Rue-Albrecht (08:45:08) (in thread): > anyway, at that point it’ll just be about fiddling with the matching parameters
Charlotte Soneson (08:46:28) (in thread): > Yeah - I guess most of the time, even if it is a good enough fit to multiple panels, one will be closer.
Kevin Rue-Albrecht (08:47:23) (in thread): > more clearly: I am first checking if there is any match within 5 edits (otherwise i just ignore the command altogether), only then i take the closest match
Charlotte Soneson (08:47:37) (in thread): > Yes, sounds sensible
Kevin Rue-Albrecht (08:47:49) (in thread): > that was the first compromise to make sure i have a match
Kevin Rue-Albrecht (09:45:51): > FYI: voice control is now optional and default toFALSEas part of theiSEE(...)arguments. > I swear personal communications won’t be recorded:sleuth_or_spy:
Kevin Rue-Albrecht (09:47:15): > Feel free to speak freely… with your computers
Aaron Lun (10:22:54): > Putting that aside, we should merge inextrainfo.
Kevin Rue-Albrecht (10:23:16): > yup, i like the logos in there
Kevin Rue-Albrecht (10:25:03): > I had a brief chat with Fede on Friday, he and I are happy with it, I think he was just waiting to hear your feedback about the placement (i.e. readme, modal popup, and also title up there on the left)
Aaron Lun (10:31:50): > I have to replant a cactus, but will be back in an hour.
Kevin Rue-Albrecht (10:32:17): > ok. i just dived into the much anticipated “color by …” command
Kevin Rue-Albrecht (10:32:31): > it’ll take me that hour
Aaron Lun (11:24:29): > the cactus is repotted.
Kevin Rue-Albrecht (11:25:26): > i’m likethatclose to having “color by [column data|feature name|…]” to work, but laptop is acting like a teenager
Kevin Rue-Albrecht (11:26:09): > listens every now and then, and just plain ignores me without giving me any indication what’s not clear
Aaron Lun (11:33:57): > @Federico Marini@Kevin Rue-AlbrechtI don’t think the logo should be in the total of the app.
Aaron Lun (11:34:04): > Everything else is fine.
Aaron Lun (11:39:08): > Possibly related, the title of the app on the browser is fucked up because of this img.
Kevin Rue-Albrecht (11:43:35): > oh is it? I’ll check ASAP.
Kevin Rue-Albrecht (11:44:24): > I just realized that the reason why my laptop was ignoring me is because this smartass is writing in UK english, while I wrote my regex in US english
Kevin Rue-Albrecht (11:44:41): > “coloUr”:sob:
Aaron Lun (11:56:19): > For some reason, my sidebar icons fall onto two lines.
Aaron Lun (11:57:17): - File (PNG): Pasted image at 2018-11-18, 4:57 PM
Aaron Lun (11:57:27): > See the left. Is no one else seeing this?
Aaron Lun (11:59:19): > Don’t know when this happened, I would guess it’s a firefox update to how it handles the sidebar width.
Kevin Rue-Albrecht (12:00:20): > Ow, you know what, i didn’t even remember that the icons were on a single line before. didn’t shock me. Plus, with Fede (I think on the channel) we were discussing the idea of adding a “export PDF” icon anyway
Aaron Lun (12:00:43): > Don’t put that there.
Aaron Lun (12:00:55): > Give an “export PDF” for all plots somewhere on the top bar.
Aaron Lun (12:01:13): > In the meantime, what width do you have for your sidebar?
Kevin Rue-Albrecht (12:01:53): > where’d you check that?web inspector?
Aaron Lun (12:02:15): > yeah
Federico Marini (12:02:28): > About the icon two lines: this came somewhat before the title logo got in
Federico Marini (12:02:38): > I thought it was a browser thing?
Kevin Rue-Albrecht (12:02:42): > @Charlotte Soneson@Federico Marinienjoy switching “color by {none|None|Column data|Feature name|Sample name}” now
Kevin Rue-Albrecht (12:03:21) (in thread): > don’t forget to “control panel {name}” first
Aaron Lun (12:03:27): > @Federico MariniI would get rid of the icon in the title. It doesn’t add anything and will probably annoy people who want to make their own title.
Aaron Lun (12:03:59): > It’s in the help and repo, so it’s not like people can miss it anyway.
Aaron Lun (12:04:09): > If you really wanted to add it, have it as a loading image, but I don’t know how to do that.
Federico Marini (12:04:24): > that’s a good idea maybe
Federico Marini (12:04:34): > I had seen something some time ago
Federico Marini (12:04:38): > with custom loaders
Aaron Lun (12:04:49): > Anyway, back to the sidebar problem.
Aaron Lun (12:04:59): > What is the width of the sidebar in your browers?
Federico Marini (12:05:01): > we can make the logo in small with a loader whirling around:slightly_smiling_face:
Aaron Lun (12:05:01): > I have 230 px.
Federico Marini (12:05:28): > same for me
Kevin Rue-Albrecht (12:05:59): > ah i see it now, 230px here as well
Charlotte Soneson (12:06:55) (in thread): > Cool! This works first, but for some reason it seems to stop working if I open the “Selection parameters” box. Anyone else sees this?
Aaron Lun (12:07:35): > So the immediate fix is to set it to 245 px.
Aaron Lun (12:07:54): > I would prefer it to automatically choose a width that fit everything on one line.
Aaron Lun (12:08:05): > I thought it was doing this already, TBH, but apparently not.
Charlotte Soneson (12:08:25): > Mine seems to be 230 too, but I have everything in one row (Chrome)
Kevin Rue-Albrecht (12:09:07): > using Chrome here too
Aaron Lun (12:09:08): > Man, WTF.
Aaron Lun (12:09:19): > So, who has everything in one row, and what is your width?
Charlotte Soneson (12:09:40): > :woman-raising-hand:230, Chrome
Aaron Lun (12:09:50): > on one row?
Aaron Lun (12:09:54): > yes.
Aaron Lun (12:10:01): > okay, 230, Firefox, Not on one row.
Kevin Rue-Albrecht (12:10:05): > I have 230, Chrome, 2 rows
Aaron Lun (12:10:09): > Jesus.
Aaron Lun (12:11:06): > I think it’s determined by the selectize at the top.
Charlotte Soneson (12:11:22) (in thread): > It also doesn’t want to change panel after I’ve colored one panel
Aaron Lun (12:12:04): > No, that’s not it.
Kevin Rue-Albrecht (12:12:19): > Gotta love the “If you don’t see this button, you’re on the latest version.” > > To update Google Chrome: > > On your computer, open Chrome. > At the top right, click More More. > Click Update Google Chrome. If you don't see this button, you're on the latest version. > Click Relaunch. >
Kevin Rue-Albrecht (12:12:45): > I like manually checking for updates
Kevin Rue-Albrecht (12:13:53) (in thread): > Oh really? I haven’t tried all combinations yet ^^
Charlotte Soneson (12:15:03) (in thread): > Or maybe I’m experiencing the teenage problem too:slightly_smiling_face:it’s not always listening to me
Kevin Rue-Albrecht (12:15:06) (in thread): > nope, still works with the “Selection parameters” box open for me.
Kevin Rue-Albrecht (12:15:26) (in thread): > I would have been surprised of any interaction between an open box and a vocal command, tbh
Kevin Rue-Albrecht (12:15:36) (in thread): > they’re completely unrelated
Charlotte Soneson (12:15:52) (in thread): > yeah, me too. For me, it typically works once, but then I can’t tell it to control another panel.
Charlotte Soneson (12:15:58) (in thread): > Do you speak very loud?:slightly_smiling_face:
Kevin Rue-Albrecht (12:15:58) (in thread): > but - as you noticed earlier - the voice recognition is a bit all over the place at times
Aaron Lun (12:16:20): > Okay, this is annoying me enough to make a major change to the UI. I propose eliminating the sidebar completely, and moving all the current bits and pieces to the bottom of each panel. There should be enough space by default, and you can add your export button as well.
Kevin Rue-Albrecht (12:16:47) (in thread): > not very loud, but for your 2nd issue, i can switch between panels after changing colors
Kevin Rue-Albrecht (12:16:56) (in thread): > I’ll make another video sometime:wink:
Charlotte Soneson (12:17:07) (in thread): > Ok:slightly_smiling_face:I’ll have to practice
Kevin Rue-Albrecht (12:17:59) (in thread): > yeah, it took me a bit of practice this weekend
Aaron Lun (12:18:10): > We can accommodate the addition of new panels by creating a blank panel at the end that just says “add a panel” when you click on it.
Kevin Rue-Albrecht (12:18:27) (in thread): > … plus i’ve got the advantage of setting up the keywords and deciding the matching rules, i guess
Charlotte Soneson (12:18:55) (in thread): > I’ll give it some more try
Kevin Rue-Albrecht (12:18:57) (in thread): > but now that i’ve sorted the “last panel memory”, speech starts to flow more naturally
Aaron Lun (12:19:00): > Alternatively, we can have another modal that allows people to freely rearrange the panel order and change the size.
Charlotte Soneson (12:19:09) (in thread): > yeah, that’s pretty cool
Kevin Rue-Albrecht (12:19:34): > mhh
Aaron Lun (12:19:43): > Basically, I’d like to free up the screen real estate that is currently occupied by the sidebar.
Kevin Rue-Albrecht (12:20:11): > well, for that you can just click on the 3 lines at the right of the title
Aaron Lun (12:20:27): > But the default is to leave it in.
Kevin Rue-Albrecht (12:20:49): > yeah, I vaguely remember trying to find how to initialize without it, but couldn’t
Aaron Lun (12:21:08): > We can disable it easily, but that’s not the point.
Aaron Lun (12:21:19): > Either we solve the row problem, or we need to do something else.
Aaron Lun (12:22:02): > Expanding and hiding the sidebar also forces all plots to re-render, so that’s not good.
Kevin Rue-Albrecht (12:22:27): > I see your point, I just don’t know whether stuffing the panel boxes with more options is the solution, is all
Aaron Lun (12:22:59): > Top bar.
Aaron Lun (12:23:13): > A single modal in the top bar with a multiselectize drag and drop to change the order of panels.
Aaron Lun (12:27:57): > The multiselectize will naturally allow deleting and addition of new panels as well; meaning that we only need further options for changing the panel sizes.
Aaron Lun (12:31:14): > Note that the panel sizing needs to go in a separate modal, so that its values are filled dynamically with the current ordering of the panels.
Aaron Lun (12:31:44): > However, this is no different to what we have now, and it would probably be even easier (i.e., re-order panels all at once, and then resize panels all at once, rather than clicking through 2 buttons - possibly multiple times - for each panel).
Aaron Lun (12:32:13): > In the meantime@Federico Marini, regardless of whether you get a loading screen to work, we should probably get rid of the logo in the title.
Aaron Lun (12:32:50): > I’ll wait for the latest changes to settle down before I do anything, but what I described above is my plan. Comments?
Kevin Rue-Albrecht (12:42:43) (in thread): > I definitely upvote the multi selectize. Moving plots with the button is a massive pain at the moment
Kevin Rue-Albrecht (12:43:38) (in thread): > on a separate note, voice control is currently the only way to “skip” ids when adding a new panel (i.e. inserting redDim3 when redDim2 is not visible yet)
Kevin Rue-Albrecht (12:44:42) (in thread): > (needless boasting: I’ve also already made sure that users can’t add a panel id above the max limit)
Kevin Rue-Albrecht (12:46:49) (in thread): > Does ithaveto go in another modal? It sounds helpful to have both the order and the size side by side, to know which dimensions correspond to which panel, no?
Kevin Rue-Albrecht (12:48:12) (in thread): > > i.e., re-order panels all at once, and then resize panels all at once, rather than clicking through 2 buttons - possibly multiple times - for each panel > exactly the behaviour that annoys me right now
Kevin Rue-Albrecht (12:50:01): > I’ve added the comments as threads on the individual points. Feels more natural, but not sure if it’s easiser to track individual discussion points
Aaron Lun (12:52:15) (in thread): > If you rearrange the order/identity in the selectize, you should also rearrange the order/identity of plots in the part of the UI that allows you to change size.
Aaron Lun (12:52:48) (in thread): > This would be easier to achieve in another modal. Though I guess we could use auiOutputwith an observer that will just trigger a re-rendering if the selectize changes. Don’t know how happy the modal would be with UI updates.
Kevin Rue-Albrecht (12:55:34) (in thread): > yep, don’t know how painful it will be, but probably worth the effort for the final product. Plus I imagine that we’ll need to implement the same observers no matter whether they’re in the same of a different modal
Federico Marini (14:16:34): > Re: logo in the title, I have no problem in removing it. I just sticked to the one solution I saw some other shiny apps had
Federico Marini (14:16:52): > If we get the custom loader running it would be even cooler
Federico Marini (14:17:21): > Does setting a title manually make problems with that?
Kevin Rue-Albrecht (14:18:34): > make problem with what?
Federico Marini (14:18:57): > with the logo in the title
Federico Marini (14:19:10): > (or with the space it should take)
Kevin Rue-Albrecht (14:19:58): > I haven’t tried yet.
Federico Marini (14:21:29): > Tried now, from thevoicebranch - if the title is short, the logo comes a little into the middle
Kevin Rue-Albrecht (14:21:33): > I’ll have a look again after dinner - I also want to finish the color by feature to handle values (in addition to titles/modes that I implemented this afternoon)
Kevin Rue-Albrecht (14:21:54): > ah right, bit weird then I guess
Kevin Rue-Albrecht (14:22:19): > anyway, let’s just remove it already, and we can figure out the loading screen approach
Federico Marini (14:23:39): > FWIW, we can have a tiny param for displaying it, and set that to FALSE as default?
Kevin Rue-Albrecht (14:26:21): > i’m cool with that.
Federico Marini (14:29:38): > re: loaders -> if we want to put all plot calls wrapped in awithLoader
Federico Marini (14:29:43): > like inhttps://github.com/emitanaka/shinycustomloader - Attachment (GitHub): emitanaka/shinycustomloader > Add a custom loader for R shiny. Contribute to emitanaka/shinycustomloader development by creating an account on GitHub.
Federico Marini (14:30:09): > which, if I recall correctly, was the pkg we already linked here once - back then not on CRAN
Kevin Rue-Albrecht (14:31:39): > > if we want to put all plot calls > I think that’s “just” about wrapping.panel_generation(...)in that.
Kevin Rue-Albrecht (14:31:59): > Anyway, dinner time. I’ll chime back in to update you on “color by …”
Federico Marini (14:32:06): > I’ll experiment with that later
Federico Marini (14:32:22): > I’m past dinner, but need to do some home cleanings
Kevin Rue-Albrecht (14:32:48): > I just really want to see it done tonight, so that I can focus on the week ahead
Federico Marini (14:50:36): > Quick try with that - wrapping does not really work out of the box . In our case the image sticks to the plot, e.g. after moving the panel
Kevin Rue-Albrecht (14:51:07): > i just sneaked in the PoC for “color using column data” followed by “color by driver one s”. Have fun
Federico Marini (14:51:12): > but it does work when we change e.g. the plot settings
Federico Marini (14:52:05): > ’twas a quick dinner:stuck_out_tongue:
Kevin Rue-Albrecht (14:52:21): > we’re sitting down now
Kevin Rue-Albrecht (14:52:24): > catch up later
Federico Marini (14:53:21): > :slightly_smiling_face:buon appetito
Federico Marini (14:58:05): > @Aaron Lun: I was also thinking of the export button in the panel itself. So yes, no extra icon on the left
Stephanie Hicks (14:59:29): > oh man…. I have been out of the loop on this channel for a few days. Did I read correctly that you guys have implemented voice commands for#isee???:heart_eyes_cat:
Federico Marini (14:59:47): > plus, I don’t know whether it could work with the app as we do it now, but I found a couple of links back then
Federico Marini (15:00:13): > @Stephanie Hicks: shhhhh:squirrel:
Federico Marini (15:00:29): > Started out as a game mostly
Stephanie Hicks (15:00:44): > ….:tada::fireworks::heart:
Federico Marini (15:00:56): > But you can check out a real version in thevoicebranch
Stephanie Hicks (15:01:49): > That isamazing!! what package/code are you using to enable voice commands??
Federico Marini (15:02:12): > or well, see Kev doing it live
Federico Marini (15:02:33): > ok, the last sentencedoesneed more context
Stephanie Hicks (15:04:59): > :grimacing:
Stephanie Hicks (15:05:16): > well i’m excited to try it out!
Federico Marini (15:05:41): > Quick one still for Aaron: > 1 -https://yang-tang.github.io/shinyjqui/2 -https://dragular.zstat.pl/ - Attachment (yang-tang.github.io): ‘jQuery UI’ Interactions and Effects for Shiny > An extension to shiny that brings interactions and animation effects from ‘jQuery UI’ library. - Attachment (dragular.zstat.pl): Drag and Drop Elements in ‘Shiny’ using ‘Dragula Javascript Library’ > Move elements between containers in ‘Shiny’ without explicitly using ‘JavaScript’. It can be used to build custom inputs or to change the positions of user interface elements like plots or tables.
Stephanie Hicks (15:05:44): > but also, why the:squirrel:? Is it top-secret?
Federico Marini (15:07:29): > Back on Friday, I thought to have it as a flashy jaw-dropper but mostly for fun to show in the upcoming presentation at EuroBioc2018
Federico Marini (15:07:57): > then I think we had too much free time:stuck_out_tongue:
Stephanie Hicks (15:07:59): > Ah makes sense
Federico Marini (15:08:53): > but really, I wanted a toy thing, no more than that.@Kevin Rue-Albrechtfound out it is not so far from being at least useful
Federico Marini (15:10:22): > So here we are - relies on the annyang library, if I recall correctly, works mostly on chrome alone (mic needs to get permissions)
Kevin Rue-Albrecht (15:17:29): > @Stephanie HicksI’ve just pinned the video that I posted earlier on the channel. Can you see it?
Stephanie Hicks (15:18:25): > yes! downloading no
Stephanie Hicks (15:18:27): > *now
Kevin Rue-Albrecht (15:19:01): > right, you can also watch it in the channel itself if you click on the timestamp
Stephanie Hicks (15:19:26): > :heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat::heart_eyes_cat:The real question is, what were you baking?
Federico Marini (15:19:45): > You can see that in the previous screenshots:smile:
Kevin Rue-Albrecht (15:19:48): > crepes
Stephanie Hicks (15:21:13): > Will you show EuroBioc2018 the video too??
Kevin Rue-Albrecht (15:21:32): > Fede will
Kevin Rue-Albrecht (15:21:46): > I’ll go to Bioc2019
Stephanie Hicks (15:22:24): > oh fun! we will get to meet in person!
Federico Marini (15:23:27): > (curbing enthusiasms down: the app crashes when I try thecolor by, followed by the name)
Federico Marini (15:23:37): > but, alone, the switch to color by coldata works
Kevin Rue-Albrecht (15:24:45): > oh. sorry .. i moved some code between my last test and rushing the push before dinner. Got cocky there for a second i guess
Kevin Rue-Albrecht (15:25:07): > what’s the error message?
Federico Marini (15:26:46): > > Warning: Error in .colorByChoices: object 'input' not found > [No stack trace available] >
Kevin Rue-Albrecht (15:26:51): > riiight
Kevin Rue-Albrecht (15:27:36): > for devp’t I wrote the function inside the server definition, so it had access to the Shiny inputs
Kevin Rue-Albrecht (15:27:40): > now that it’s out..
Federico Marini (15:28:04): > aha, ok
Kevin Rue-Albrecht (15:29:20): > blind fix incoming
Kevin Rue-Albrecht (15:29:37): > give it a try (i can’t voice test right now)
Federico Marini (15:30:39): > a sec
Federico Marini (15:33:10): > driver 1 s->Warning: Error in if: argument is of length zero
Kevin Rue-Albrecht (15:33:25): > sorry. i’ll fix that a bit later properly then
Federico Marini (15:33:43): > no prob, I was just trying it out
Kevin Rue-Albrecht (15:34:41): > Thanks for that, sorry i wasted your time, watching a movie. I’ll just relax for a bit and do things properly after
Federico Marini (15:35:45): > (primry type raises the same error)
Federico Marini (15:35:50): > no waste:wink:
Kevin Rue-Albrecht (16:51:30): > Alright@Federico Marini: it’s properly fixed now. I forgot that reactive values had to be isolated when passed to subfunctions
Kevin Rue-Albrecht (16:52:35): > it’s very muchad hocat the moment (ifelseblock that only supports coloring by a column data field), but it’s proof of concept enough for me
Kevin Rue-Albrecht (16:54:02): > Example use (Allen dataset): > - “control panel reduced dimension plot 1” > - “color using sample name” > - “color using feature name” > - “color using column data” <- this is required for the next ones > - “color by driver one s” > - “color by core type”
2018-11-19
Kevin Rue-Albrecht (02:32:52): > That Monday morning feeling:rolling_on_the_floor_laughing: - File (JPEG): Image from iOS
Federico Marini (03:30:33): > it workzzz
Kevin Rue-Albrecht (04:48:52): > Anyone for the “Interpreted Speech for interactiveSummarizedExperiment” ? - File (SVG): Isis.svg
Kevin Rue-Albrecht (04:49:23): > oh svg don’t work. hold on a minute
Kevin Rue-Albrecht (04:51:03): > As I was saying “Interpreted Speech for InteractiveSummarizedExperiment” - File (PNG): Isis.png
Kevin Rue-Albrecht (04:51:45): > I stayed on the “see” thematic with the emphasis on the eye
Aaron Lun (04:51:53): > Thought you were going to go all islamic state.
Kevin Rue-Albrecht (04:52:10): > yeah.. they really messed up the name I have to say
Kevin Rue-Albrecht (04:52:21): > I like Egyptian mythology
Aaron Lun (05:41:22): > Just gave up on one of my manuscripts.
Kevin Rue-Albrecht (05:41:29): > oww
Aaron Lun (05:41:29): > https://doi.org/10.1101/404962 - Attachment (bioRxiv): Overcoming systematic errors caused by log-transformation of normalized single-cell RNA sequencing data > Applying a log-transformation to normalized expression values is one of the most common procedures in exploratory analyses of single-cell RNA sequencing (scRNA-seq) data. Normalization removes systematic biases in sequencing coverage between cells, while the log-transformation ensures that downstream computational procedures operate on relative rather than absolute differences in expression. We show that the log-transformation can introduce systematic errors when cells vary in sequencing coverage, leading to spurious non-zero differences in expression and artificial population structure in simulations. We observe similar effects in real scRNA-seq data where the difference in transformed values between groups of cells is not an accurate proxy for the log-fold change. We provide some practical recommendations to overcome this effect and analytically derive an expression for a larger pseudo-count that controls the transformation-induced error to a specified threshold.
Aaron Lun (05:41:40): > Well, the reviews were fairly friendly.
Aaron Lun (05:41:49): > But I have no more fucks to give.
Aaron Lun (05:42:36): > I wrote it to keep the publications ticking along, but Genentech doesn’t care for anything less than CNS, so it’s kind of pointless to push this through.
Charlotte Soneson (05:47:31) (in thread): > So it’s decided?
Charlotte Soneson (05:48:48) (in thread): > You’re going to SF?
Aaron Lun (06:14:04) (in thread): > Pretty much.
Kevin Rue-Albrecht (06:19:14): > https://www.nature.com/articles/s41467-018-07329-0 - Attachment (Nature Communications): Invasive Salmonella exploits divergent immune evasion strategies in in > Salmonella employ a range of strategies to counter host defences during infection. Here, Aulicino et al. use single-cell RNA-sequencing to examine the effects of invasive and non-invasive strains of Salmonella, revealing discrete and divergent immune evasion strategies in infected and bystander dendritic cells.
Aaron Lun (06:22:58): > Yeah, yeah, rub it in.
Aaron Lun (06:23:25): > I see you only cited me once?
Aaron Lun (06:23:28): > :sad-parrot:
Kevin Rue-Albrecht (06:24:18): > I remember citing the f1000 workflow paper like 4-5 times throughout the manuscript at some point
Aaron Lun (06:27:34): > :disappointed:Not even double digits.
Kevin Rue-Albrecht (06:29:17): > Are you sure you’re only once in there? I’ll have to check
Aaron Lun (06:32:18): > Looked for “Lun”.
Kevin Rue-Albrecht (06:32:23): > Hang on. The pdf only shows 45 references. They limited us to 70 but I don’t see the others
Aaron Lun (06:32:43): > Even online there’s only one Lun.
Kevin Rue-Albrecht (06:32:52): > I’m on the phone right now, but indeed ive found your name only once
Kevin Rue-Albrecht (06:33:42): > Anyway I’ll have to log off, I’m on a celebratory pizza run to have lunch with coauthors
Aaron Lun (06:35:26): > cold hearted bastard
Aaron Lun (06:35:33): > I never got any pizza
Charlotte Soneson (06:36:49) (in thread): > Cool, congrats then!
Kevin Rue-Albrecht (06:38:22): > Well… in this case Anna and me are the ones buying… so there better be:pizza:
Aaron Lun (06:52:52) (in thread): > :money_mouth_face:
Aaron Lun (06:53:00) (in thread): > That’s what I’ve decided I’m all about now.
Kevin Rue-Albrecht (07:00:18): > Is that citing you enough Aaron ?:stuck_out_tongue_winking_eye:I don’t have scran yet - File (JPEG): Image from iOS
Aaron Lun (07:00:56): > I think you should just stick them everywhere. Like, on passing cars and stuff.
Charlotte Soneson (09:13:25) (in thread): > Fair enough
Federico Marini (09:52:46) (in thread): > … and here is me finding out the existence of this emoji:slightly_smiling_face:
Federico Marini (09:54:18): > I did it for iSEE on the bobby car of Alva:smile:
Federico Marini (09:54:32): > no fellow kid asked me what this is for
Federico Marini (09:54:51): > but wait till I show them 2yo that they can voice control it, HUH
Aaron Lun (10:31:24): > nosidebardone.
Aaron Lun (10:32:07): > Enjoy the new, cleaner interface - topright for panel organization. I’ll leave it to others to beautify the aesthetics.
Aaron Lun (10:32:58): > And, as usual, all tests involvingpanel_organizationare probably broken.
Federico Marini (11:08:06): > Uh cool
Federico Marini (11:08:23): > for tomorrow, still, I’ll stick to what’s on thevoicebranch
Federico Marini (11:08:45): > I will have an institute internal seminar
Kevin Rue-Albrecht (11:09:05): > oh tomorrow, I don’t know why i remembered Friday
Kevin Rue-Albrecht (11:09:17): > can’t wait to hear feedback
Federico Marini (11:09:30): > I don’T expect that much actually
Federico Marini (11:09:41): > the institute is pretty colorful in term of composition
Federico Marini (11:09:46): > but no wet lab persons
Federico Marini (11:10:01): > we have epidemiology and medical informatics as well
Federico Marini (11:10:54): > at it was damn hard to not say this to any of my colleagues, even the closest ones:smile:
Kevin Rue-Albrecht (11:10:54): > Well, you don’t need wet lab persons to impress. I thinkiSEEcan impress a few of our peers by now:stuck_out_tongue:
Kevin Rue-Albrecht (11:13:00): > On a separate note, I’ll soon have to start a lexicon of keywords to document the voice control
Federico Marini (11:15:40): > I found also the perfect quote for munich
Federico Marini (11:16:09): > Funny enough it is from the hitchhiker’s guide to galaxy
Federico Marini (11:16:23): > and I see Charlotte put it already in the recent peerj one
Federico Marini (11:17:11): > > See first, think later, then test. But always see first.
Federico Marini (11:17:20): > I’ll take the liberty to add ani
Stephanie Hicks (11:17:45) (in thread): > Congratulations@Aaron Lun!!
Stephanie Hicks (11:19:00) (in thread): > I hadn’t heard the news! When will you start and what will you be doing??
Charlotte Soneson (11:20:35) (in thread): > To be fair, that one wasn’t my contribution. I honestly don’t remember much from the book, so I just suggested that the paper should be 42 pages…
Federico Marini (11:24:04) (in thread): > good catch, it did end up like this
Federico Marini (11:24:10) (in thread): > still
Federico Marini (11:24:34) (in thread): > I think many excerpts from the series can be so actual now
Charlotte Soneson (11:25:21) (in thread): > If you say so…there was something about a towel I have come to understand
Aaron Lun (11:25:32) (in thread): > Feb/March next year. Bioinf software dev.
Stephanie Hicks (11:27:38) (in thread): > :tada:
Federico Marini (11:29:52) (in thread): > Yes, you should always carry with you one
Federico Marini (11:30:11) (in thread): > Just read the first chapters (or chapter) of the guide
Charlotte Soneson (11:30:15) (in thread): > Good to know, I’ve failed on that one
Federico Marini (11:30:22) (in thread): > I do have one in my office
Charlotte Soneson (11:30:26) (in thread): > Towel or book?
Federico Marini (11:30:28) (in thread): > not for that reason
Federico Marini (11:30:30) (in thread): > towel
Charlotte Soneson (11:30:34) (in thread): > Ok
Federico Marini (11:30:44) (in thread): > book, it is at home
Charlotte Soneson (11:30:54) (in thread): > I think I just gave my copy away this summer
Kevin Rue-Albrecht (11:31:01) (in thread): > Sweet move
Stephanie Hicks (11:33:13) (in thread): > so happy for you@Aaron Lun
Aaron Lun (11:33:56) (in thread): > I too am happy for myself.
Aaron Lun (11:34:15) (in thread): > No more having to write papers to justify my existence.
Aaron Lun (11:34:27): > I take it travis is still offline?
Federico Marini (11:34:55): > it does work but
Federico Marini (11:34:58): > not for iSEE
Kevin Rue-Albrecht (11:35:36): > Yeah.. it’s a real pain. I honestly don’t wish to runchecklocally every time anymore. I liked being lazy
Kevin Rue-Albrecht (11:36:40): > I wonder whether it can be fixed by refreshing the cache of packages though. That did the trick a few times for me in the past
Kevin Rue-Albrecht (11:37:27): > i.e. temporarily removing that linehttps://github.com/csoneson/iSEE/blob/master/.travis.yml#L7 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Charlotte Soneson (11:39:25): - File (PNG): Screen Shot 2018-11-19 at 17.39.04.png
Charlotte Soneson (11:39:36): > There doesn’t seem to be any cache:thinking_face:
Aaron Lun (11:40:09): > Is it already using 3.6?
Federico Marini (11:40:23): > shouldbe
Kevin Rue-Albrecht (11:42:03): > > Setting up build cache > $ export CASHER_DIR=${TRAVIS_HOME}/.casher > 0.05s$ Installing caching utilities > 0.00s2.39sattempting to download cache archive > fetching nosidebar/cache-linux-trusty-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--R-devel.tgz > fetching nosidebar/cache--R-devel.tgz > fetching master/cache-linux-trusty-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--R-devel.tgz > fetching master/cache--R-devel.tgz > could not download cache > > https://travis-ci.org/csoneson/iSEE
Aaron Lun (11:42:27): > Okay, I think we should bump the version number dependency to 3.6 then.
Kevin Rue-Albrecht (11:43:22): > I mean I see a lot of “3.6” all over the place in the build log: > > R session information > $ Rscript -e 'sessionInfo()' > R Under development (unstable) (2018-11-18 r75626) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Ubuntu 14.04.5 LTS > Matrix products: default > BLAS: /home/travis/R-bin/lib/R/lib/libRblas.so > LAPACK: /home/travis/R-bin/lib/R/lib/libRlapack.so > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > attached base packages: > [1] stats graphics grDevices utils datasets methods base > loaded via a namespace (and not attached): > [1] compiler_3.6.0 BiocManager_1.30.4 tools_3.6.0 >
Kevin Rue-Albrecht (11:44:33): > There is just not a lot of information to explain the final: > > Error: (converted from warning) dependencies 'Rcompression', 'rnaturalearthhires', 'starsdata', 'cacheSweave', 'loomR', 'MAST', 'spDataLarge', 'corpus.useR.2008.abstracts', 'lqa', 'codetoolsBioC', 'adehabitat', 'kmndirs', 'Rcampdf', 'tm.lexicon.GeneralInquirer', 'R2wd', '[sylly.de](http://sylly.de)', '[sylly.es](http://sylly.es)', 'glmmADMB', 'FRB', 'RDCOMClient', 'bigFastlm', 'ActuDistns', 'BiocInstaller', 'corpus.JSS.papers' are not available >
Aaron Lun (11:45:54): > I didn’t even think we depended on some of these.
Federico Marini (11:46:24): > that’s the thing, we nevah evah did
Aaron Lun (11:47:04): > Probably thesuggestsis pulling in a whole bunch of crap.
Aaron Lun (11:47:30): > bigFastlm,, for example, no longer exists.
Kevin Rue-Albrecht (11:47:37): > I’ll try a few things later this evening if you like. > Actually, look everything beingNAs for the package updates: > > 13.40s$ Rscript -e 'devtools::install(dependencies = c("Depends", "Imports", "Suggests", "Enhances"))' > Annotatio... (NA -> 1.45.0 ) [CRAN] > beachmat (NA -> 1.5.0 ) [CRAN] > beeswarm (NA -> 0.2.3 ) [CRAN] > BH (NA -> 1.66.0-1 ) [CRAN] > bindr (NA -> 0.1.1 ) [CRAN] > > I honestly think that dropping thecache: packagesline could do the trick. > It doesn’t look like an issue with 3.6, as it does call in “R-devel” and thesessionInfolooks right
Federico Marini (11:57:28): > give it a shot Kev
Kevin Rue-Albrecht (12:09:58): > yup, just helping the RA over here
Aaron Lun (12:13:35): > You can usenosidebaras an test example, it passesCHECKlocally.
Aaron Lun (13:12:56): > I’m done withnosidebar, you can play around with it if you like. Looks pretty damn good.
Kevin Rue-Albrecht (13:18:12): > I look forward
Aaron Lun (14:18:56): > looks like cache clearing didn’t work.
Kevin Rue-Albrecht (14:19:02): > yup, just saw that
Kevin Rue-Albrecht (14:19:41): > i wonder if we’re installing a heavier burden of dependencies than necessary because ofEnhancesin- Rscript -e 'devtools::install(dependencies = c("Depends", "Imports", "Suggests", "Enhances"))'
Kevin Rue-Albrecht (14:19:55): > let me switch the cache back and remove the Enhances
Kevin Rue-Albrecht (14:20:49): > iSEE doesn’t have any “Enhances”, but I wonder whether this is applied recursively
Aaron Lun (14:20:50): > do we need the suggests of our dependencies?
Kevin Rue-Albrecht (14:21:25): > I think Travis needs the suggests (e.g.testthat) to runcheck
Aaron Lun (14:21:53): > Usually I just dodependncies=TRUE.
Federico Marini (14:23:20): > I have this forideal
Federico Marini (14:23:22): > https://github.com/federicomarini/ideal/blob/master/.travis.yml - Attachment (GitHub): federicomarini/ideal > Interactive Differential Expression AnaLysis - DE made accessible and reproducible - federicomarini/ideal
Federico Marini (14:23:51): > so the dependencies are not really needed to be specified, apparently
Federico Marini (14:25:08): > BiocManager is not required specifying as well, I think it is implicit withr: bioc-devel
Kevin Rue-Albrecht (14:26:08): > It’s possible that BiocManager is included, it just doesn’t hurt to add it explicitly just in case.
Aaron Lun (14:26:42): > If you see the docs fordependencies=TRUEininstall.packages, you will see that it’s the way to go.
Kevin Rue-Albrecht (14:26:47): > Foridealfor instance, you installstringrexplicitly in the travis YAML while it’s declared in yourImports
Aaron Lun (14:27:08): > In particular, it won’t pull down theSuggestsfor the packages that weDependorImportson.
Kevin Rue-Albrecht (14:27:18): > ah
Federico Marini (14:27:37): > I do installstringrand others that otherwise take too long to compile from source
Kevin Rue-Albrecht (14:27:55): > Shall I push that one in parallel to the current one, of wait to see the result of the current one?
Kevin Rue-Albrecht (14:28:05): > I’m happy with option 1
Aaron Lun (14:29:27): > Your call. Might as well wait IMO. ¯*(ツ)*/¯
Aaron Lun (14:29:43): > More importantly, what do you think about the no side bar look?
Kevin Rue-Albrecht (14:29:45): > yeah.. doesn’t take that long to crash anyway.. positive thinking FTW
Federico Marini (14:30:01): > I am installing now
Kevin Rue-Albrecht (14:30:09): > installing what?
Federico Marini (14:30:44): > the pkg from nosidebar
Aaron Lun (14:30:46): > Does anyone know what happens if we delete all panels? Maybe need some protection there.
Kevin Rue-Albrecht (14:30:57): > ah, i love the new layout
Kevin Rue-Albrecht (14:31:09): > didn’t have an issue when i did it by voice
Kevin Rue-Albrecht (14:31:17): > just a blank background
Aaron Lun (14:31:32): > How does the voice know to delete panels?
Kevin Rue-Albrecht (14:31:36): > hehehe
Kevin Rue-Albrecht (14:31:55): > “hide panel ”
Aaron Lun (14:32:05): > This is going to be a bit awkward in the modal.
Kevin Rue-Albrecht (14:32:17): > why?
Federico Marini (14:32:30): > sleek design, nice idea
Aaron Lun (14:32:31): > Or does the voice directly interact with thepObjects?
Aaron Lun (14:32:37): > YEAH (for fed’s comment).
Kevin Rue-Albrecht (14:33:47): > well, initially I’ve copied exactly the same code as you wrote for the click action, then I’ve refactor it out to two functionsshowPanelandhidePanel
Federico Marini (14:33:50): > only contra thing: we lose some anchors for the tour
Aaron Lun (14:34:07): > That was anticipated and we’ll just have to deal with it.
Federico Marini (14:34:18): > but on the plus side: the new version is even more intuitive
Kevin Rue-Albrecht (14:34:19): > long way of saying that thanks to you’re well written code, I didn’t have to think much xD
Aaron Lun (14:34:36): > You should make sure we don’t end up copying lots of code.
Aaron Lun (14:34:47): > Because a lot of the voice commands will just execute the same code in the observers.
Federico Marini (14:34:50): > I don’T expect a minimal IQ to use the program, but hey:smile:
Kevin Rue-Albrecht (14:35:01): > that’s what I meant: I’m refactoring everything out to functions called by either click or voice input
Aaron Lun (14:35:28): > Or a smoother way to do it would be to use the voice input to “nudge” the observers.
Aaron Lun (14:35:42): > Possibly by editinginput.
Kevin Rue-Albrecht (14:35:48): > nooooo!
Kevin Rue-Albrecht (14:35:55): > inputis read-only
Kevin Rue-Albrecht (14:36:04): > smart move is toupdateSelectize
Aaron Lun (14:36:13): > No, i’m talking in general.
Kevin Rue-Albrecht (14:36:16): > which then triggers the typical observers
Kevin Rue-Albrecht (14:36:22): > ahh
Kevin Rue-Albrecht (14:36:29): > well, then we think the same thing
Aaron Lun (14:36:29): > Otherwise you’ll just end up rewriting all of our current stuff.
Kevin Rue-Albrecht (14:37:01): > whenever you like just peek into thevoicebranch, you’ll see that there’s really minimum extra code
Kevin Rue-Albrecht (14:37:29): > i’m throwing us flowers, but iSEE is so well written that as you said, I just need to nudge things that already do 90% of the job
Federico Marini (14:37:37): > if we touch the UI that much, we should think of the second manuscript soon:stuck_out_tongue:
Federico Marini (14:38:06): > or on a more serious thought, should we then update the figure 1 for the f1000res manuscript?
Aaron Lun (14:38:32): > Well, I just hope there’s enoughupdate*Inputfunctions to cover all possibilities.
Kevin Rue-Albrecht (14:38:35): > i will point out thought that “color by” is the only exception at the moment, as it’s still at prototype stage
Kevin Rue-Albrecht (14:38:45): > so far so good
Kevin Rue-Albrecht (14:39:42): > ask Fede and Charlotte about the “colour using …” command whichupdateSelectizeInputon the “color by
” widget:heart_eyes:
Kevin Rue-Albrecht (14:40:25): > i got the same chills as when we first close a lasso that switch togeom_polygon
Kevin Rue-Albrecht (14:40:29): > :stuck_out_tongue:
Federico Marini (14:40:50): > You never forget your first rodeo after all:dealwithit-parrot:
Aaron Lun (14:40:53): > I will test it out tomorrow on my mac. I had better not be disappointed.
Kevin Rue-Albrecht (14:42:43): > needless to tell you that the only “documentation” of available voice commands is… thewww/voice.jsfile itself
Kevin Rue-Albrecht (14:46:25): > On a separate note, I’m happy to report that the Travis build is now at 12min, while all the recent ones crashed after 5-8 min
Kevin Rue-Albrecht (14:48:47): > It’s probably safe to call for a “Aaron was right” minute of silence
Kevin Rue-Albrecht (15:03:28): > Travis at 30min and still installing R packages like Rcpp. I think we’re happily heading into a timeout
Kevin Rue-Albrecht (15:03:43): > good thing is the package cache is enabled again
Kevin Rue-Albrecht (15:12:21): > suspense, I think we’ll just make it in time, we’re at* checking whether package 'iSEE' can be installed ...with 10 min left
Kevin Rue-Albrecht (15:19:42): > goooal:goal_net:build success:party_parrot:in 46/50min
Kevin Rue-Albrecht (15:27:27): > btw, check out this thing that just popped up ob the bioc devel mailing list:https://github.com/lrutter/bigPintv(Diane Cook) - Attachment (GitHub): lrutter/bigPint > Analysis and scripts for Google Summer of Code project. - lrutter/bigPint
Aaron Lun (15:35:42): > merge. And see if you can updatevoicefor the newest thing, should be sufficient to do aupdateSelectizeInput.
Federico Marini (15:37:52): > shall we restore the automatic checks from Travis,@Charlotte Soneson?
Charlotte Soneson (15:39:50): > Done - File (PNG): Screen Shot 2018-11-19 at 21.39.21.png
Aaron Lun (16:03:13): > Merged.
Kevin Rue-Albrecht (16:42:40): > Cool I’ll merge into voice
Kevin Rue-Albrecht (16:51:07): > btw, I’ve tested packagebigPint, there are some design ideas in there that could interest us (e.g. use of theshinycusttomloaderpackage, but not much to worry about in terms of overlap with iSEE, as far as I can see
Aaron Lun (17:20:38): > I’m all about taking other people’s shit.
Aaron Lun (17:20:47): > Excuse me, “liberating”
Aaron Lun (17:20:54): > Got to brush up on my American English.
Aaron Lun (18:39:29): > Willvoicework with the show/hide? Or is that broken with thenosidebarchanges?
Kevin Rue-Albrecht (18:41:53): > still works for me
Kevin Rue-Albrecht (18:44:31): > argh, i’ve just run into an issue withselectizeInputstoring choices server-side
Kevin Rue-Albrecht (18:45:32): > if I callupdateSelectizeInputwhen the choice is not in the top N choices (the ones preloaded in the client), thenupdatedSelectizeInput(select=...)doesn’t work
Kevin Rue-Albrecht (18:46:51): > andupdatedSelectizeInput(server=TRUE)does not takeselectedas argument, onlychoices, as all this one does is update the choices server side, while the filtering and selection is done dynamically as users start typing to filter through the choices
Kevin Rue-Albrecht (18:46:54): > damn
Kevin Rue-Albrecht (18:51:19): > nope. problem solved
Kevin Rue-Albrecht (19:05:25): > there you go, “color by” works for sample name and gene names. Try “color by TNF” and other friends like IL1B, ACTB
Kevin Rue-Albrecht (19:06:21): > for sample names, i suggest renaming colnames to something more palatable, likepaste("cell", seq_len(ncol(sce)))
Kevin Rue-Albrecht (19:09:43): > One last thing, try multitasking voice on one panel and clicks on another, it’s fun ^^
2018-11-20
Kevin Rue-Albrecht (04:40:36): > reminiscing about the race to commit “1000”, we’re slowly approaching both commit “2000” and PR “200”
Aaron Lun (06:07:10): > Give me the minimum working example
Aaron Lun (06:07:13): > I’m on the mac right now.
Kevin Rue-Albrecht (06:10:21): > hold on one sec
Aaron Lun (06:10:34): > example(sce, ask=FALSE), and then…?
Kevin Rue-Albrecht (06:10:47): > switching to laptop where I have the MWE
Kevin Rue-Albrecht (06:11:01): > > library(iSEE) > library(scRNAseq) > data(allen) > class(allen) > > # Example data ---- > > library(scater) > sce <- as(allen, "SingleCellExperiment") > counts(sce) <- assay(sce, "tophat_counts") > sce <- normalize(sce) > > sce <- runPCA(sce, ncomponents=4) > sce <- runTSNE(sce) > rowData(sce)$ave_count <- rowMeans(counts(sce)) > rowData(sce)$n_cells <- rowSums(counts(sce)>0) > sce > > # launch the app itself ---- > > initialPanels <- DataFrame( > Name=c("Reduced dimension plot 1", "Column data plot 1"), > Width=c(4, 4) > ) > > app <- iSEE(sce, initialPanels = initialPanels, voice=TRUE) > if (interactive()) { > shiny::runApp(app, port=1234, launch.browser = FALSE) > } > > # devtools::reload(); app <- iSEE(sce, initialPanels = initialPanels, voice=TRUE); shiny::runApp(app, port=1234, launch.browser = FALSE) >
Kevin Rue-Albrecht (06:11:34): > you just caught me as I was about to go meet with Steve (11-12:00)
Kevin Rue-Albrecht (06:14:00): > actually, he’s still meeting with the PhD student, so if you have first impressions, lay it on me
Aaron Lun (06:14:26): > Give me some things to say
Kevin Rue-Albrecht (06:14:36): > “show active panel”
Aaron Lun (06:14:40): > what does that do
Kevin Rue-Albrecht (06:14:44): > “control panel reduced dimension plot 1”
Kevin Rue-Albrecht (06:14:54): > “color using column data”
Kevin Rue-Albrecht (06:15:08): > “color by driver one s”
Federico Marini (06:15:42): > “good boy”
Kevin Rue-Albrecht (06:15:55): > I’ve set up a persistent notification, that shows which is the panel currently controlled by voice
Kevin Rue-Albrecht (06:16:25): > “show active panel” is there if you have manually closed the notification and want to bring that up again
Kevin Rue-Albrecht (06:16:49): > the others should be pretty self-explanatory
Kevin Rue-Albrecht (06:17:07): > oh wait
Kevin Rue-Albrecht (06:17:17): > “show panel heat map 1”
Kevin Rue-Albrecht (06:17:28): > “hide panel reduced dimension panel 2”
Kevin Rue-Albrecht (06:18:35): > for the record, “it’s not working” or “i’m tired of this $#1t” or “I give up” don’t work yet
Kevin Rue-Albrecht (06:19:32): > but as@Federico Marinipointed out, “good boy” does work
Kevin Rue-Albrecht (06:19:57): > so if you’re happy, feel free to throw him a bone
Aaron Lun (06:22:07): > Let’s see if it works on safari…
Federico Marini (06:22:55): > chrome is the best one till now
Aaron Lun (06:24:25): > How have you been diverting it to use chrome?
Aaron Lun (06:24:34): > you just look at the address, I suppose.
Federico Marini (06:25:59): > the latter one
Aaron Lun (06:29:58): > It worked once but it doesn’t do so consistently.
Aaron Lun (06:37:19): > This shit is too unreliable
Aaron Lun (06:40:26): > Not nearly responsive enough.
Aaron Lun (06:44:26): > There’s about a ~5sec delay on my machine (chrome), which isn’t good enough.
Federico Marini (06:44:51): > I agree it is not production ready
Federico Marini (06:45:04): > not to forget most life scientists work in open lab spaces
Aaron Lun (06:52:50): > Yeah, Nils is already pissed with me.
Aaron Lun (06:53:11): > I think you should putshow (fucking) panel *panel, because that’s what’s going to happen.
Kevin Rue-Albrecht (07:19:27): > alright i’m back from the meeting
Kevin Rue-Albrecht (07:20:06): > in my experience, there is generally at least 2s delay (best case) between voice and the UI completing the refresh
Kevin Rue-Albrecht (07:21:37): > > How have you been diverting it to use chrome? > That’s why I putiSEE..., launch.browser = FALSE)in a terminal. This way, it just listens on a port, but doesn’t launch your default browser, leaving you to open whichever browser you like
Kevin Rue-Albrecht (07:23:39): > I mean, i generally open Chrome, and type the page with immediate autocompletion by now. > When I want to test a new version of the source code, I Ctrl-C in the R console, and run the last line of the chunk that I pasted above > > devtools::reload(); app <- iSEE(sce, initialPanels = initialPanels, voice=TRUE); shiny::runApp(app, port=1234, launch.browser = FALSE) > > and refresh Chrome
Kevin Rue-Albrecht (07:24:07): > but that’s just for development, users of a web-hosted app would never have to worry about that
Kevin Rue-Albrecht (07:26:31): > As a side note, I don’t mind the few seconds of delay when I manage to multitask: e.g control one panel by voice while clicking around another one
Kevin Rue-Albrecht (07:27:43): > It could be fun when “control panel <1>“, “receive selection from panel <2>“, while you’re setting up your selection in panel <2>
2018-11-21
Rob Amezquita (15:54:11): > @Rob Amezquita has joined the channel
2018-11-22
Federico Marini (06:17:55): > program’s out for EuroBioc
Federico Marini (06:18:15): > we have Svetlana in the same session with the rlc package from Simon
Aaron Lun (06:19:26): > Are you doing the speaking?
Federico Marini (06:20:10): > Yep
Federico Marini (06:20:23): > Charlotte’s in with the jcc
Aaron Lun (06:21:40): > Time to WIN.
Federico Marini (06:22:03): > be sure to send me a slack PM during the talk with textfinish her:smile:
Kevin Rue-Albrecht (06:30:51): > i didn’t mention, but those vocal commands are implemented. Perhaps the most useful so far, despite the 5s delay: > - “send selection to (panel) <column data plot 1>” > - “receive selection from (panel) <reduced dimension plot 1>” - Attachment: Attachment > It could be fun when “control panel <1>“, “receive selection from panel <2>“, while you’re setting up your selection in panel <2>
Aaron Lun (06:31:07): > Jesus
Kevin Rue-Albrecht (06:31:57): > You can click around the options of a panel, all the while sending the selection to another without having to open the “selection parameter box” of that other one
Kevin Rue-Albrecht (06:32:40): > Anyway,@Aaron Lun, don’t fear, with the defaultvoice=FALSEargument, it’s all the same good ol’ behaviour that we all like
Kevin Rue-Albrecht (06:33:13): > voice=TRUEis purely for fanciness factorInf
Kevin Rue-Albrecht (07:16:58): > On a more serious note, after my lab meeting next Wednesday, I’ll spend the time to look at the unit tests if no one gets to them first
2018-11-23
Aaron Lun (15:57:32): > Can people remove their dead/merged branches? Also, can someone addRELEASE_3_8?
Kevin Rue-Albrecht (16:59:03): > I took care of the latter. > I suggest that individual branches are handled by the last person who pushed on it. For example, I’ve removed mymultisplat. > Remaining ones are: > -projsite@Federico Marini-fixedcol@Charlotte Soneson-titlelogo@Federico MariniMerge or delete, make your choicehttps://www.youtube.com/watch?v=6DyWemBLCL8 - Attachment (YouTube): Saw- Live or die. Make your choice.
Aaron Lun (17:43:55): > What the hell - that’s John!
2018-11-25
Federico Marini (10:22:59): > I’ll clean up later tonight:wink:
Kevin Rue-Albrecht (10:32:43): > Thinking about#sc-signature, I thought it could be nice to add amodePBMC(for instance), which opens a series of t-SNE each coloured by a different “typical” marker. It’s kind of what people are doing anyway, to assign identities to their clusters..
Kevin Rue-Albrecht (10:35:29): > Challenge there being to define the list of cell type markers given thatrownamescan be Ensembl ids, “Seurat made-unique” ids, etc …). But just likemodeGating, it could be an argument that could be overridden by users.
Stephanie Hicks (21:46:18) (in thread): > well that’s terrifying!
2018-11-28
Charlotte Soneson (03:33:28): > If anyone wants to try sizing points by covariates, I pushed an attempt to allow this to thesizebybranch yesterday.
Kevin Rue-Albrecht (03:35:09): > Argh. ok. My plan was to branch frommasterand tackle the unit tests once I get passed the group meeting presentation today.
Charlotte Soneson (03:36:11): > The tests are passing
Charlotte Soneson (03:37:56): > or what did you want to tackle?
Kevin Rue-Albrecht (03:39:01): > are they? i only checked when Aaron updated the UI and I remember some tests breaking somewhere
Kevin Rue-Albrecht (03:39:17): > perhaps my intel is just outdated
Charlotte Soneson (03:39:31): > https://travis-ci.org/csoneson/iSEE
Charlotte Soneson (03:39:51): > I fixed a couple, but those were all related to the sizing
Kevin Rue-Albrecht (03:41:19): > ahhhh i know: it’s only branchRELEASE_3_8that’s broken, because it hasn’tt gone through the.travis.ymlupdate
Charlotte Soneson (03:41:34): > ah ok
Kevin Rue-Albrecht (18:30:48): > @Charlotte Sonesonif I merge my PR tomaster, can you then mergemastertosizebyand merge it back tomasteras well? My PR already anticipated the next version bump (1.3.2), so either I revert the version bump before merging, or perhaps you should mergesizebyfirst. > I’ll leave my PR as is for now.
Kevin Rue-Albrecht (18:34:05): > Other alternative is that you just follow up with your own version bump. Doesn’t really matter on bioc-devel.
2018-11-29
Kevin Rue-Albrecht (02:25:04): > Actually, I just realise that I should add some documentation of voice commands before merging. I’m thinking of a simple full-text no-code vignette.
Kevin Rue-Albrecht (02:26:55): > In any case, feel free to PR-merge sizeby at any time with or without version bump. They’re independent features anyway
Charlotte Soneson (02:26:57): > Any order is fine with me. I’ll make a couple of more tests ofsizebybefore creating the PR (and I’ll merge in the then-current master before merging)
Kevin Rue-Albrecht (02:27:33): > Perfect. Yes I’ve also tested voice last night before opening the PR
Federico Marini (15:05:01): > throwback time - buttons for iSEE, in 8 bit land:smile:
Federico Marini (15:05:02): > https://bcrikko.github.io/NES.css/ - Attachment (bcrikko.github.io): NES.css > NES-style CSS Framework | ファミコン風CSSフレームワーク
Kevin Rue-Albrecht (15:44:54): > Is that another “dare” likevoice?:stuck_out_tongue:
Federico Marini (15:56:26): > nah:stuck_out_tongue:
Federico Marini (15:56:50): > voice was actually a thought thrown out, which later became something indeed
Federico Marini (15:57:16): > next steps is to do all this oculus rift or so, and kick out outliers in VR environments
Kevin Rue-Albrecht (15:57:34): > speaking of which: i’m adding the unit tests, hence the pause before merging the PR
Federico Marini (15:58:23): > since I am setting up the presetnation: Kev, are you fine in displaying you also in the vid?
Federico Marini (15:58:57): > as a backup plan in case I see the control does not do what is intended to do
Kevin Rue-Albrecht (15:59:08): > sure, i kinda cleaned up the kitchen to make sure it was PG-friendly
Kevin Rue-Albrecht (15:59:34): > same here: I wanted a backup for my presentations
Federico Marini (16:00:01): > eheh
Kevin Rue-Albrecht (16:00:14): > the video will get old fast, wrt the number of vocal commands supported but hey
Kevin Rue-Albrecht (16:00:19): > it’ll do
Federico Marini (16:00:20): > it would be to easy to have rated R videos with the language
Avi Srivastava (23:16:45): > @Avi Srivastava has joined the channel
2018-11-30
Aaron Lun (20:02:24): > Haha, R rated videos.
Aaron Lun (20:02:29): > I get it.
2018-12-02
Kevin Rue-Albrecht (11:21:02): > FYI, I’ve just locally updated the default tour steps, as the recent “user interface” update moved some UI elements to the panel organization modal.. I’ll push a branch and merge shortly
Aaron Lun (11:26:38): > :+1:
Kevin Rue-Albrecht (11:56:12): > incoming, with version bump and all
Kevin Rue-Albrecht (11:58:15): > Darn, I’m at best a half-Aaron - File (PNG): Pasted image at 2018-12-02, 4:58 PM
Kevin Rue-Albrecht (11:58:20): - File (PNG): Pasted image at 2018-12-02, 4:58 PM
Kevin Rue-Albrecht (11:58:50): > I should cut down my sleeping time, I guess
Aaron Lun (12:00:07): > Damn straight
Kevin Rue-Albrecht (12:12:20): > @Aaron Lunthe LTLA/iSEE2018/pbmc4k tour doesn’t seem to preselectc("PF4", "PPBP", "TMSB4X", "B2M", "ACTB", "SDPR")anymore. Is that just on my side?
Aaron Lun (12:14:04): > ¯*(ツ)*/¯
Aaron Lun (12:14:16): > I didn’t change anything.
Kevin Rue-Albrecht (12:14:41): > maybe it’s something to do with the character to numeric index conversion
Kevin Rue-Albrecht (12:14:53): > :sleepy:
Kevin Rue-Albrecht (12:24:02): > Yeah, somehow, that’s what I see forparam_choicesof the heat map when I run the iSEE2018 tour - File (PNG): Pasted image at 2018-12-02, 5:23 PM
Kevin Rue-Albrecht (12:24:19): > (thec(1,1,1,1..))
Kevin Rue-Albrecht (12:27:47): > I’m guessing the character to index conversion may not be happy with lists
Kevin Rue-Albrecht (12:30:51): > uh.. that should not be a problemhttps://github.com/csoneson/iSEE/blob/5fa2548e08c55953b4e4ce5697b251e26e72d3dd/R/iSEE-extras.R#L309 - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (12:33:35): > well, no matter what the code says, giving the numeric indices solves the problem: > > heatMap <- DataFrame(FeatName=I(list(c(4885,7711,7712,11908,13435,24384))), > ColData=I(list(c("Cluster"))), Lower=-5, Upper=5) >
Kevin Rue-Albrecht (12:38:26): > ah! > > all_args$heatMapPlot <- .name2index(all_args$heatMapPlot, .heatMapFeatName, colnames(se)) > > “features = columns” was one of my early Bioconductor mistakes… during my PhD:stuck_out_tongue:
Kevin Rue-Albrecht (12:41:13): > time for agit blame:https://github.com/csoneson/iSEE/commit/94016f74d835738ca6b268581f712bfbfe92d89f:stuck_out_tongue: - Attachment (GitHub): Lay groundwork (defaults, constants) for new colStatTable panel. · csoneson/iSEE@94016f7 > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Aaron Lun (12:48:17): > Don’t know what’s going on, but it sounds like you fixed it.
Kevin Rue-Albrecht (12:49:26): > Yup. Fixed. What happened is that on 13 Aug you swapped arownamesfor acolnames:stuck_out_tongue_winking_eye:
Aaron Lun (12:51:41): > Ah, August 13. I remember that day well.
Kevin Rue-Albrecht (12:51:50): > Monday, 3pm
Kevin Rue-Albrecht (12:52:16): > Must have been hell of a weekend:stuck_out_tongue:
Aaron Lun (12:56:02): > CRAAAZEY
Avi Srivastava (13:25:55) (in thread): > Yea, I was wondering what happened in April (the two whites):stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (13:36:09): > :parrotconga::chart_with_upwards_trend: - File (PNG): Pasted image at 2018-12-02, 6:36 PM
Kevin Rue-Albrecht (18:03:54): > Bug report.@Charlotte Sonesonspotted it and I can confirm it with the very latest version (master branch): > > Btw, I was trying out the latest version (or at least 1.3.2) the other day, and it seemed to me that the new panel organization input was resetting (to the default panel order) every time I opened it - I didn’t have time to investigate further, but I’ll try again - is that something that you have seen too?
Kevin Rue-Albrecht (18:13:52): > I know why. Let’s see if i can fix it
Kevin Rue-Albrecht (18:18:46): > yup. done.
Aaron Lun (18:22:23): > call the temporary variable something else other thanavailable_panels.
Aaron Lun (18:23:43): > I’m surprised the selectize doesn’t respect the order inselected=, and coerces it to the order inchoices=.
Kevin Rue-Albrecht (18:24:21): > the selectize behaviour makes sense to me:choices %in% selected
Aaron Lun (18:26:17): > Even Dan T. was complaining about it:https://github.com/selectize/selectize.js/issues/218 - Attachment (GitHub): Disable automatic sort · Issue #218 · selectize/selectize.js > Hi, I'm receiving a pre-ordered list (JSON) but when I loaded into the selectize control, it re-orders the list by the Id and I don't want this. How I can let selectize use the order of my …
Aaron Lun (18:29:57): > To me, the order should be respected, otherwise there’d be no point of drag and drop.
Aaron Lun (18:30:40): > Because by definition, any permutation would be indistinguishable from each other.
Aaron Lun (18:31:07): > So this seems like a definitive deficiency from the selectize’s side.
Kevin Rue-Albrecht (18:31:18): > right, i didn’t think how the selectize really handles the selection under the hood, as I’m used to iSEE memory controlling the ordering aspects
Aaron Lun (18:31:59): > Well, this is a problem that hasn’t been fixed since 2014 so your hack is probably the best soln.
Kevin Rue-Albrecht (18:32:08): > it’s true that it’s a default in the selectize behaviour that it allowsvisualreordering, if that is not honored in the variable
Aaron Lun (18:32:27): > That seems rather pointless to me.
Aaron Lun (18:32:37): > Anyway, just rename the tmp var to avoid confusion withavailable_panelsscope.
Kevin Rue-Albrecht (18:32:45): > btw, i’ll log off soon, probably before Travis finishes
Kevin Rue-Albrecht (18:33:02): > anyway, i love the smell ofmergein the morning
2018-12-03
Federico Marini (03:04:33): > @Kevin Rue-Albrecht: thank you for updating the default tour, I was planning to do it today
Federico Marini (03:05:00): > I’ll look over again whether something/what needs to be polished after the latest changes
Federico Marini (03:05:12): > is there any item I can/should hop on?
Kevin Rue-Albrecht (03:05:22): > > @Kevin Rue-Albrecht: thank you for updating the default tour, I was planning to do it today > I gotta catch up on my commit gap with Aaron:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (03:07:00): > Well, I think I updated everything to my taste, eg added steps showing the plot itself after each input update.
Kevin Rue-Albrecht (03:08:22): > Not sure what else there is to do. I’d say go through the tour yourself, reading every word (that’s the hardest part when you’ve written half of them), and see if anything looks out of place or missing
Kevin Rue-Albrecht (03:11:03): > Something that’s usually not 100% self consistent is the font choice for package names, functions and other special words throughout the tours. Panel names are the only ones that we explicitly discussed and properly went through.
Kevin Rue-Albrecht (03:12:44): > Ah, perhaps the LTLA2018 tours need a similar update. I haven’t checked each of them yet.
Federico Marini (03:14:03): > I’ll have a look
Federico Marini (03:14:42): > I’ll start from pbmc4k which I likely will use as demo
Kevin Rue-Albrecht (03:16:14): > Here’s my own presentation plan for the 11th > 1. introduce the challenge (data viz), > 2. the list of iSEE features, > 3. SummarizedExperiment and SingleCellExperiment > 4. Shiny > 5. use the allen tour to walk through the panels and input controls > 6. use the CyTOF app to show off with large numbers of cells > 7. use the PBMC4k to grab every 10x user’s attention > 8. questions?
Federico Marini (03:16:35): > how much time did you get allocated?
Kevin Rue-Albrecht (03:16:48): > 1h in total
Federico Marini (03:17:00): > oh
Federico Marini (03:17:11): > that’s luxury
Federico Marini (03:17:16): > here’s my current version for eurobioc
Federico Marini (03:17:33): > https://federicomarini.github.io/EuroBioc2018/
Kevin Rue-Albrecht (03:17:41): > Well it’s the whole institute where Anna works that’s got invited, plus mine and the WTCHG
Kevin Rue-Albrecht (03:18:52): > So I’m giving the grand tour and there should be questions from PIs to student to core biondo I hope
Kevin Rue-Albrecht (03:20:22): > I’ll email you guys the poster of the talk and I’ll put my updated slides on Dropbox (maybe I should start using slide share )
Federico Marini (03:21:29): > now that you say poster
Federico Marini (03:21:40): > Aaron, did you take a picture of your analog poster?
Kevin Rue-Albrecht (03:23:11): > Good slides btw
Kevin Rue-Albrecht (03:40:56): > (!) now that you got me thinking about the presentation, i just found how I might introduce the voice mode:
Kevin Rue-Albrecht (03:42:15): > “[…] and now that speech recognition is implemented, you can talk to iSEE the same way you did with you bioinformatician.” > …. too much?:sweat_smile:
Federico Marini (03:44:47): > I wanted to try that out already
Federico Marini (03:45:04): > if the speech recognition works with sentences “already started”
Federico Marini (03:45:20): > “I wish I couldcontrol panel RD1”
Kevin Rue-Albrecht (03:45:46): > yeah, i partially allowed that withshow (me) (us) panel ...
Kevin Rue-Albrecht (03:46:01): > it works
Kevin Rue-Albrecht (03:46:23): > but then I’m worried about adding unnecessary burden on the voice recognition algorithm
Kevin Rue-Albrecht (03:48:18): > andit doesn’t look like it, or even seem counter-intuitive, but itwouldadd cognitive effort on the user side to remember what the “supported natural sentence starts” are
Kevin Rue-Albrecht (03:53:03): > If you want a (more useful?) challenge, the speech recognition library supports multiple languages:https://github.com/TalAter/annyang/blob/master/docs/FAQ.md#what-languages-are-supported - Attachment (GitHub): TalAter/annyang > :speech_balloon: Speech recognition for your site. Contribute to TalAter/annyang development by creating an account on GitHub.
Kevin Rue-Albrecht (03:53:24): > I haven’t looked into that yet:stuck_out_tongue:
Kevin Rue-Albrecht (03:54:10): > (here is where@Aaron Lun: “And please don’t.”):stuck_out_tongue_winking_eye:
Federico Marini (03:54:18): > eheheheh
Federico Marini (03:54:43): > who does not want to explore RNA-seq data in klingon or dothraki
Kevin Rue-Albrecht (03:55:56): > They do support “latin”.. that could help withannotateEnsemblspecies names …:thinking_face:
Kevin Rue-Albrecht (07:18:57): > hem… how about “Now, I’d like to introduce a feature that’s un-heardof, even in Bioconductor: […]”:innocent:
Federico Marini (10:08:48): > re: updating tours - I rerun the pbmc4k data analysis, can it be some changes in the code end up changing the detection of empty cells?
Federico Marini (10:09:09): > (changing of course VS what we have on the marionilab server)
Aaron Lun (10:17:19): > If you reran itnow, I’ve changed the analysis.
Federico Marini (10:21:27): > ok, then makes sense
Federico Marini (10:21:51): > now the cluster with platelets ends up on the other side:stuck_out_tongue:
Federico Marini (10:22:16): > as long as we do not change the data on the server, there is no need to change the tour
Federico Marini (10:23:39): > otherwise: the rest of the tours did not need any update after the change in the ui
Federico Marini (10:58:51): > alrighty, I had just a small add to the default tour
Federico Marini (10:58:55): > PR’s ready to merge
Federico Marini (10:59:28): > I did not update the version number yet, given it is minor minor
Aaron Lun (11:02:40): > k
Federico Marini (11:03:30): > I’ll likely stick to the allen data for the demo
Aaron Lun (11:14:20): > FINISH HIM:-1:
Federico Marini (11:14:32): > bump/no bump?
Aaron Lun (11:14:42): > Just roll it in with the existing one.
Federico Marini (11:14:55): > plus - I guess all of us got an email from Kayla?
Aaron Lun (11:15:01): > iSEE hasn’t been bumped yet.
Federico Marini (11:15:03): > re: adding ImmunoOncology
Aaron Lun (11:15:03): > on BioC.
Aaron Lun (11:15:09): > I got it for about 5 packages.
Aaron Lun (11:15:22): > I told her she could do it herself.
Federico Marini (11:15:44): > I was not fancying the “please change it if you want, otherwise we do it anyway” tone:smile:
Federico Marini (11:15:45): > ok
Federico Marini (11:15:53): > so I am not alone:stuck_out_tongue:
Aaron Lun (11:16:10): > WTF when did you makeHCAData?
Aaron Lun (11:16:42): > Is there a channel about this?
Federico Marini (11:16:47): > last week or so it got accepted
Federico Marini (11:17:00): > I kept Vince on the loop regarding the effort
Aaron Lun (11:17:06): > Okay, good.
Federico Marini (11:17:42): > from his words he was happy that came along, probably it will take long enough to have the shop-like package to fetch n retrieve
Kevin Rue-Albrecht (11:18:04) (in thread): > Actually I’ve pushed upstream at each of the version bumps. I don’t know why it hasn’t propagated yet
Aaron Lun (11:18:32): > But does BioC re-host the data on Ehub?
Kevin Rue-Albrecht (11:18:38) (in thread): > I didn’t want to waste the core team time, so I just patiently waited
Kevin Rue-Albrecht (11:18:59) (in thread): > But it’s 4 days now since I pushed 1.3.2
Federico Marini (11:19:00): > with this package, it does (only the counts of course)
Federico Marini (11:19:42): > I don’t know exactly what the long term plans will be - hosting somewhere else, and we only have a couple of clever API to access?
Aaron Lun (11:21:55): > Yes, that’s the plan.
Aaron Lun (11:22:06): > Well, someone’s plan, anyway.
Kevin Rue-Albrecht (11:48:22) (in thread): > Scratch that, i pushed 1.3.2 on the 30th November, so just 3 days ago
Kevin Rue-Albrecht (11:48:47) (in thread): > it should go live today/tomorrow
Kevin Rue-Albrecht (11:49:20) (in thread): > I never completely got a proper grasp on the turnaround, with the time difference and build cycle, etc.
Aaron Lun (11:52:58) (in thread): > que sera
Kevin Rue-Albrecht (13:25:03) (in thread): > Pleasedoversion bump it
Kevin Rue-Albrecht (13:25:52) (in thread): > I’ll only handle thepush upstreampart. This is your moment. Go get’em commits!
Federico Marini (14:35:17) (in thread): > hump de bump’d
2018-12-04
Kevin Rue-Albrecht (06:05:17) (in thread): > The propagation system seems a bit sluggish: I could already see the successful build report (1.3.3!) yesterday, but the devel landing page still shows 1.3.1
Kevin Rue-Albrecht (12:21:55): > @Federico Mariniyou wanna merge that PR, or do you have anything else you wanna include? Whenever you merge, just ping me and I’ll push upstream
Aaron Lun (12:22:35): > Suggest adding “#human-cell-atlas” to the repo.
Aaron Lun (12:22:38): > @Charlotte Soneson
Charlotte Soneson (12:24:05): > you mean as a topic? - File (PNG): Screen Shot 2018-12-04 at 18.23.41.png
Aaron Lun (12:24:21): > Yep.
Aaron Lun (12:24:40): > https://github.com/topics/human-cell-atlas - Attachment (GitHub): Build software better, together > GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects.
Aaron Lun (12:24:44): > Added all my shit.
Charlotte Soneson (12:26:07): > ok, done
Kevin Rue-Albrecht (12:28:01): > Now you make me wonder how may tags we could add before they become meaningless. I mean we have two tags for gene expression, but nothing to mention other data modalities (microarray, cytof, etc)
Aaron Lun (12:29:28): > Currently we’re top of the leaderboard.
Aaron Lun (12:32:28): > I wonder what other crap we can add.
Kevin Rue-Albrecht (12:38:17): > Well, stating the obvious, just open up one of your own repos, type “#” and see all the crazy variations of tags there are for ’omics software
Aaron Lun (12:38:40): > No, to “human-cell-atlas”
Aaron Lun (12:38:50): > What other repos are vaguely funded by HCA money?
Kevin Rue-Albrecht (12:38:54): > … or wait until you’re sitting near a warm fireplace in a couple of weeks
Aaron Lun (12:39:09): > In australia? Not a good idea.
Kevin Rue-Albrecht (12:39:19): > fair enough:sweat_smile:
Federico Marini (15:34:27): > Can the propagation delay have to do with this?
Federico Marini (15:34:27): > http://bioconductor.org/packages/stats/bioc/iSEE/
Federico Marini (15:34:40): > -> access forbidden, now
Federico Marini (15:35:01): > Anyway: merging now,@Kevin Rue-Albrecht
Kevin Rue-Albrecht (15:47:46): > Ok. Thanks.
Kevin Rue-Albrecht (15:48:22): > I guess you’re right about the stats.
Kevin Rue-Albrecht (15:48:34): > Btw, how beautiful is that? - File (PNG): Pasted image at 2018-12-04, 8:48 PM
Kevin Rue-Albrecht (16:16:59) (in thread): > I’ve pushed it upstream. just now.
Federico Marini (16:20:14) (in thread): > merci
2018-12-06
Stephanie Hicks (07:10:44): > congratulations@Federico Marinion your talk at #eurobioc2018:slightly_smiling_face:
Stephanie Hicks (07:10:55): > the view from twitter seems to suggest it went well!
Kevin Rue-Albrecht (07:14:23): > We should come up with a “pancake plot” now. (The French in me yells “crepe” but “crepe plot” is a bit of a pain to pronounce)
Charlotte Soneson (07:31:25): > you mean instead of a pie chart?
Federico Marini (07:31:52) (in thread): > Thanks Steph:slightly_smiling_face:
Federico Marini (07:32:10) (in thread): > any neighbour of yours who can grab a sticker for you?
Federico Marini (07:32:20) (in thread): > Is Keegan close by?
Kevin Rue-Albrecht (07:38:51): > mmmh. i don’t have any idea of what the pancake would look like or represent, I just have the name:stuck_out_tongue:
Stephanie Hicks (07:39:23) (in thread): > Somewhat! I’m visiting Boston next week actually
Kevin Rue-Albrecht (07:40:39): > “blueberry pancake plots” could be something like your contour plots on t-SNE, highlighting the centroid of individual clusters
Federico Marini (08:13:50) (in thread): > I can hand over some then to her
Federico Marini (08:14:25) (in thread): > she’s 6 heads away from me in the audience and I’ll make sure to catch up with her in one of the breaks
2018-12-07
Federico Marini (08:57:19): > FYI - if you don’t have it already
Federico Marini (08:57:36): > we could add these couple of lines to travis.yml:
Federico Marini (08:57:42): > > before_install: > - mkdir -p ~/.R > - echo -e 'MAKEFLAGS = -j2' > ~/.R/Makevars > - echo 'options(Ncpus = 2)' > ~/.Rprofile >
Federico Marini (08:58:11): > even if we have caching, it can speed up a little the whole process
Kevin Rue-Albrecht (08:59:21): > Uh. Nice one. I just wonder whether we may want to keep the Travis build as close as possible to the Bioc build configuration?
Federico Marini (08:59:59): > that does nothing but enabling parallel building of dependencies
Kevin Rue-Albrecht (09:00:05): > right
Federico Marini (09:00:20): > and it is not so greedy
Kevin Rue-Albrecht (09:00:56): > Ok, then as long as it doesn’t affectbuild,checkandINSTALL, why not
Federico Marini (09:02:16): > it is basically only for the resolving of the dependencies
Kevin Rue-Albrecht (09:02:44): > I understood that now. What I’m saying is go ahead if you wanna give it a try on a side branch.
Kevin Rue-Albrecht (09:03:10): > Maybe even push it to 4 cores, if you really wanna see a difference?
Federico Marini (09:03:26): > since we build often: probably there is not that much of a need
Federico Marini (09:03:36): > 2 is quite good already
Federico Marini (09:03:52): > did not want to be too nasty with good ol travis
Federico Marini (09:04:24): > especially because anyways sometimes the process will have the bottleneck of 1 parent dependency
Federico Marini (09:04:27): > oh well
Federico Marini (09:04:31): > we can give it a shot
Kevin Rue-Albrecht (09:05:31): > it’s on a side branch initially anyway, we can worry aboutmasteronly if it works
Federico Marini (14:00:39): > dunno if it is a good sign, but the stickers got sold out:smile:
Federico Marini (14:01:06): > I did put something like a good 50-ish in
2018-12-08
Kevin Rue-Albrecht (06:02:25): > @Federico Mariniso.. is11 min 9 sec<12 min 32 secthe speed gain ?
Kevin Rue-Albrecht (06:02:51): > i assume that the difference will be more striking when there are more packages to install/update?
Federico Marini (08:01:10): > yup:stuck_out_tongue:
Federico Marini (08:01:54): > plus we do cache pkgs
Federico Marini (08:01:55): > some people do not
Kevin Rue-Albrecht (09:13:02): > @Federico Marini, it’s backhttp://bioconductor.org/packages/stats/bioc/iSEE/
2018-12-10
Charlotte Soneson (07:19:59): > Ok, codecov is happy again so I’ll merge thesizebybranch unless anyone objects
Kevin Rue-Albrecht (07:49:00): > I don’t have time to test today, but if you’re happy I’m happy
Kevin Rue-Albrecht (16:52:12): > FYI,@Charlotte SonesonI just sent it upstream
Charlotte Soneson (16:53:11): > :+1:
2018-12-12
Kevin Rue-Albrecht (05:10:49): > Hi all, > Just to give some feedback on the presentation that I gave yesterday. > Make sure you include a slide that visually places iSEE in a generic workflow, e.g. > FASTQ files -> HISAT2 -> featureCount -> count files -> count matrix -> iSEE > count matrix + PCA -> iSEE > count matrix + PCA + tSNE -> iSEE > etc … > to help both computational and wet lab users understand what goes in iSEE, what it does and what it doesn’t do
Kevin Rue-Albrecht (05:11:52): > it’s one thing to say “you have to precompute everything you want to visualize and store it in a SE object” > but as always, a picture is worth 1000 words
Aaron Lun (05:18:24): > Where was this?
Kevin Rue-Albrecht (05:18:37): > O-place, WIMM
Kevin Rue-Albrecht (05:19:23): > Not sure if you’ve been but Viper and Davis gave some talks there
Kevin Rue-Albrecht (05:20:16): > audience was a mixture of compbio and wetlab
Aaron Lun (05:20:40): > No, and I don’t want to go there.
Kevin Rue-Albrecht (05:20:41): > i managed to keep them in the seminar room for the whole hour, so I’d call it a success
Aaron Lun (05:21:37): > No encore?
Aaron Lun (05:21:40): > “KEVIN”
Aaron Lun (05:21:42): > “KEVIN”
Aaron Lun (05:21:44): > “KEVIN”
Aaron Lun (05:21:59): > Should have turned up in a singlet and jeans.
Kevin Rue-Albrecht (05:22:35): > well that speech recognition 20s video is definitely a hit
Kevin Rue-Albrecht (05:23:25): > btw, I’ve added subtitles now, in the common case whereby the audio of the conference room doesn’t work
Kevin Rue-Albrecht (05:24:11): > it’s in Dropbox, just open the video using VLC, it picks up the subtitles file automatically, as it’s named identically to the video file
Kevin Rue-Albrecht (05:38:29): > Aside from the issue I’ve opened on the GH repo, there was also a question about 3D plots “because some other dataviz tools support it”. > To that I pointed out that selection would not be possible, therefore if we ever consider it, it would be an “endpoint panel” like the heat map.
Aaron Lun (05:59:29): > The trick with 3D plugins is how to remember the zoom/pan state.
Aaron Lun (06:02:22): > shinyRGL looks pretty promising.
Aaron Lun (06:03:23): > though I would prefer users to modify the viewport via formal shiny UI elements. Probably some opportunity for three sliders that rotate x, y an z axes?
Federico Marini (10:53:27): > hey peeps:slightly_smiling_face:I managed to get an iSEE sticker on the laptop of a Nat comms editor:smile:
Federico Marini (10:53:34): > (FWIW:stuck_out_tongue:)
Kevin Rue-Albrecht (10:55:23): > Hahahah nice
Rob Amezquita (11:00:45): > hi@Aaron Lun@Kevin Rue-Albrechtand whoever else is working on iSEE @Stephanie Hicksand I are working on drafting a paper reviewing some of the current awesome Bioconductor single-cell rna-seq tools. I was wondering if I could recruit somebody with knowledge of such tools to contribute a paragraph talking about interactive visualization in R via Bioconductor tools, one of which could be iSEE
Federico Marini (11:01:36): > We’d be slightly biased, but it makes more than sense:slightly_smiling_face:
Rob Amezquita (11:01:36): > doesn’t have to be long, 3-5 sentences would be fine, here’s an example about “Feature Selection” > > In most experiments, only a subset of the genes drives heterogeneity across the population of cells profiled within an scRNA-seq experiment. Identifying the most informative genes or features, or genes with high variance due to biological effects rather than technical effects, is an important step in the analysis of scRNA-seq data because it reduces the computational burden and removes noise in downstream analyses by removing uninformative genes. A common approach is to identify informative features based on either highly expressed genes (selecting genes with a high overall average across across cells) or the highly variable genes (selecting highly dispersed genes or genes with a high coefficient of variation relative to overall expression). In Bioconductor, packages that implement standalone functions for feature selection include , which uses LOESS regression to model the the mean-variance relationship of log transformed implemented in the and functions to identify biologically informative genes. Other standalone functions include the function in the package, which … . Other methods for feature selection have been implemented into Bioconductor packages such as , which identify both highly and lowly variable genes within and across groups. For a recent review comparing feature selection methods for scRNA-seq data, including the concordance across the different approaches or incorporating prior knowledge, see .
Federico Marini (11:02:39): > We can wrap up a summary of the current status quo (which at least in Bioc is not so much, afaik)
Rob Amezquita (11:02:47): > that would be perfect
Federico Marini (11:02:53): > there’s scClustViz (if I got the capitalization correct)
Federico Marini (11:03:04): > but that is not on Bioc? I should check
Rob Amezquita (11:03:19): > is itSingleCellExperimentcompatible?
Federico Marini (11:03:25): > I’d need to check
Charlotte Soneson (11:03:28): > And the epivizr things
Federico Marini (11:03:30): > it is out on f1000res
Rob Amezquita (11:03:34): > looks like it uses Seurat
Rob Amezquita (11:04:28): > epivizr is perfect!
Kevin Rue-Albrecht (11:04:29): > Booooh (Seurat):stuck_out_tongue_winking_eye:
Rob Amezquita (11:04:32): > thank you@Charlotte Soneson
Rob Amezquita (11:06:14): > any volunteers to write the paragraph? if not, i am happy to do it with y’alls input of course on what should be talked about, these suggestions are great
Stephanie Hicks (11:07:56): > yes, thanks@Rob Amezquitafor the ping!
Kevin Rue-Albrecht (11:08:05): > Shall we Google Drive that, iSEE-team?
Charlotte Soneson (11:08:10): > Sounds good
Rob Amezquita (11:08:28): > thank you all! please be sure to put down the author names and affiliations
Rob Amezquita (11:08:50): > and any (package) references you use, please also generate a Bibtex entry for it
Kevin Rue-Albrecht (11:09:40): > speaking of which,BASiCSis the right capitalization
Charlotte Soneson (11:11:28): > So if I get it right, this is single-cell (SCE), analysis focused, and the paragraph is aimed at visualization tools for data and analysis results (i.e., not e.g. genome/sequence-focused)?
Rob Amezquita (11:13:10): > what do you mean by genome/sequence-focused?
Rob Amezquita (11:13:33): > yes, definitely single-cell (rna-seq) focused though youre right about that
Charlotte Soneson (11:13:37): > Like, coverage plots, genomic ranges etc
Rob Amezquita (11:14:01): > that actually would be helpful
Rob Amezquita (11:14:36): > since scRNA-seq can be visualized with coverage plots/genomic ranges sorts of plots too - we do want to highlight some of the other applications like epigenomics/etc., but this should be briefer than anything pertaining to scRNA-seq
Charlotte Soneson (11:15:13): > ok, got it. I guess all such things are less single-cell specific anyway
Rob Amezquita (11:15:24): > (and ideally, things that worked with SummarizedExperiment/SingleCellExperiment objects are top-line, must mentions)
Kevin Rue-Albrecht (11:15:47): > here goes:https://docs.google.com/document/d/16bYCR1BTEUY9FBRkLigDWfji9xbsIx3-O5orpR2ufSk/edit?usp=sharing - File (Google Docs): single-cell rna-seq interactive dataviz tools
Rob Amezquita (11:15:48): > yeah, honestly if anything comes to mind just put it down, we can always trim later, but hard to extract knowledge from y’all on demand:slightly_smiling_face:
Rob Amezquita (11:16:37): > great! just ping me when there’s a draft, happy to provide feedback but no worries if its rough around the edges, it’ll go through a proper editing phase later this week
Rob Amezquita (11:22:52): > sorry to hijack the channel, but another question: are y’all somewhat familiar with “report generation” software on Bioconductor? do you think there’s enough stuff around that to write another section?
Rob Amezquita (11:23:20): > (besides ad-hoc rmarkdown/bookdown docs:smile:)
Charlotte Soneson (11:23:28): > There’s at least ReportingTools and Glimma
Kevin Rue-Albrecht (11:23:47): > on bioc I don’t know, but there was a talk on that subject at Bioc2017 (ahhh the good memories)
Charlotte Soneson (11:23:50): > If that’s the type of report generation software you meant
Rob Amezquita (11:24:13): > thats exactly it
Kevin Rue-Albrecht (11:24:51): > > Kirsten Van Hoorde, OpenAnalytics, How to automate analysis workflow via reproducible reports (using an R template package) for recurrent standardized data: a case study, video, slides > https://bioconductor.github.io/EuroBioc2017/
Kevin Rue-Albrecht (11:25:28): > To be honest, I just remember not being convinced by it, but for the purpose of a review, a tool is a tool
Rob Amezquita (11:25:41): > yeah we arent giving any opinions in this
Rob Amezquita (11:25:47): > just “here it is, it exists”
Rob Amezquita (11:25:59): > “Bioconductor is awesome why would you ever use Seurat”
Rob Amezquita (11:26:26): > (except for its new integration routine is fairly decent)
Aaron Lun (11:26:58): > Yeah, thanks to us.
Rob Amezquita (11:27:14): > heh, i knew that would elicit a response
Rob Amezquita (11:27:21): > sorry@Aaron Lun
Rob Amezquita (11:28:31): > based on vs. CCA alone, the improvement seems to be mostly MNN driven
Aaron Lun (11:39:40): > Yeah, I’ll bet it is.
Federico Marini (11:51:47): > Re: reporting,@Rob Amezquita: it can look like a shameless plug, but from myself there’s pcaExplorer and ideal
Federico Marini (11:52:41): > both with the principle of a template report, delivered inside the package, and picking the live status of the shiny app, and compiling upon pushing a “generate report” button
Kevin Rue-Albrecht (11:54:45): > Speaking of which, it doesn’t exist yet, but the “R script reporting” of iSEE would only need a handful of tweaks to turn into an Rmd file.
Federico Marini (11:55:45): > I did today the demo live and put that into an R file and knitted that
Kevin Rue-Albrecht (11:56:18): > i cut that one short in my presentation yesterday. I showed them the code, but didn’t run it in the console
Federico Marini (11:56:19): > It would just need the part wheere the sc object is loaded and computed, still. Keep in mind knitting does all the things in a fresh session
Federico Marini (11:56:31): > Oh I was living on the edge today:smile:
Rob Amezquita (11:58:09): > @Federico Marinithat’s exactly the sort of things i’m looking for, thank you! and no, not a shameless plug, we need to get the word out about these awesome softwares
Federico Marini (11:58:47): > Then please let’s do it:slightly_smiling_face:
Federico Marini (11:59:14): > As you can see we are already on the google doc, so there’s no need to ask you “what’s the timeline”:stuck_out_tongue:
Rob Amezquita (11:59:31): > haha im frankly amazed by how fast this all is coming together, y’all are amazing
Federico Marini (11:59:42): > Although, neither pcaExplorer nor ideal are single-cell focused
Federico Marini (11:59:55): > we probably have too much free time?
Federico Marini (12:00:00): > :party_parrot:
Rob Amezquita (12:00:07): > :fiesta_parrot:
Aaron Lun (12:00:22): > It’s 3:30 am over here.
Federico Marini (12:00:27): > uh the fiesta parrot is new
Aaron Lun (12:00:39): > I have a dentist’s appointment in the morning and I’m too nervous to sleep.
Federico Marini (12:00:57): > good choice to keep you busy, A-boy:smile:
Federico Marini (12:01:30): > re: pcaExplorer, if the biorXiv team is quick, we can have a doi for that soon!
Rob Amezquita (12:02:26): > awesome! please ping me when thats up
Rob Amezquita (12:02:51): > did you write an intro paragraph already that talked about existing tools federico?
Charlotte Soneson (12:05:03): > Could also mentionDEGreporton the reporting side
Federico Marini (12:08:10) (in thread): > in the ms, yes. and it mostly encompassed non just R/bioc solutions
Federico Marini (12:08:25) (in thread): > So I might adapt that for a mini-paragraph here
Federico Marini (12:08:52) (in thread): > heading home now, I’ll pick up on it later on:wink:
Rob Amezquita (12:09:40) (in thread): > thank you so much!! actually, yes, i think since you already have that pretty much written, id love to have the expert view on this…if you can adapt it to be more Bioc focused, and then add it to the current google doc under a “Report Generation” section, that would be supreme
Rob Amezquita (12:10:51): - Attachment: Attachment > thank you so much!! actually, yes, i think since you already have that pretty much written, id love to have the expert view on this…if you can adapt it to be more Bioc focused, and then add it to the current google doc under a “Report Generation” section, that would be supreme
Kevin Rue-Albrecht (12:10:59): > A bit of Googling brought this one up:https://github.com/hackseq/vasco/issues/1 - Attachment (GitHub): Welcome to the single cell visualization team! · Issue #1 · hackseq/vasco > Hi everyone, My name is Grace, and I work on single cell applications at 10x Genomics, a biotech startup in the Bay Area, California. I lived in Vancouver for 5 years, and went to UBC for my underg…
Rob Amezquita (12:11:16): > thank you@Charlotte Soneson! the ^ also applies to anyone with expertise in the report generation side
Rob Amezquita (12:11:30): > we’ll just add it to the current doc since its very in line with “interactive visualization” sort of work
Kevin Rue-Albrecht (12:13:40): > Although vasco “Latest commit 43b04aa on 23 Oct 2017”
Rob Amezquita (12:13:59): > eek, yeah, we want to focus mostly on still developed (or highly stable) packages
Rob Amezquita (12:14:15): > good catch…something to keep an eye on
Kevin Rue-Albrecht (12:15:40): > From the extract above, it seemed to be driven by a 10x staff member at least. I’m not sure if they gave up competing against themselves with CLoupe
Rob Amezquita (12:15:50): > ah, that would make sense
Kevin Rue-Albrecht (12:22:56): > Anyone got access to this one?https://www.cell.com/cell-systems/fulltext/S2405-4712(18)30235-7
Charlotte Soneson (12:23:47) (in thread): > Nope:-1:
Charlotte Soneson (12:25:07): > Hmm…many of the packages we have mentioned don’t really have an associated publication.@Rob Amezquita, you have a strategy for this in the rest of the paper - refer to the packages?
Aaron Lun (12:27:22): > I’m pretty sure epivizR has a citation?
Aaron Lun (12:27:35): > Well, singleCellTK has a manuscript somewhere.
Charlotte Soneson (12:29:04): > Ok, so I’m just bad at finding things:slightly_smiling_face:I asked Hector for their preferred way of citingepivizr, there are several papers, but none specifically on that as far as I could see.
Rob Amezquita (12:34:12): > @Charlotte Sonesonprobably refer to the bioconductor site or github…i will try and contact the authors in these (hopefully few) cases to generate a doi
Charlotte Soneson (12:34:46): > Ok - I found some more, probably will be fine
Rob Amezquita (12:35:31) (in thread): > i can get access via my institution library, just have to request, whats this for?
Kevin Rue-Albrecht (12:36:30) (in thread): > well that’s my question, I can’t see what the paper is fully about, but the title sounds relevant
Rob Amezquita (12:36:59) (in thread): > here is this:https://www.nature.com/articles/nbt.4314 - Attachment (Nature Biotechnology): Dimensionality reduction for visualizing single-cell data using UMAP > A benchmarking analysis on single-cell RNA-seq and mass cytometry data reveals the best-performing technique for dimensionality reduction.
Rob Amezquita (12:37:01) (in thread): > similarish
Kevin Rue-Albrecht (12:37:02) (in thread): > (“Generalizable and Scalable Visualization of Single-Cell Data Using Neural Networks” for anyone who doesn’t feel like clicking the link)
Kevin Rue-Albrecht (12:37:50) (in thread): > Ah similarish - but not the same. Up to you whether you want to include one or both
Rob Amezquita (12:51:46) (in thread): > okay requested the article, should hear back later today (e.g. by the time you wake up tomorrow)
Aaron Lun (13:30:57): > gonna try to get a few hours of sleep.
Aaron Lun (13:30:59): > Will probably fail.
Kevin Rue-Albrecht (15:00:33) (in thread): > Thanks to an anonymous source here is the ‘requested’ paper:https://sci-hub.tw/10.1016/j.cels.2018.05.017 - Attachment (sci-hub.tw): Sci-Hub: устраняя преграды на пути распространения знаний > Первый в мире пиратский ресурс, который открыл массовый доступ к десяткам миллионов научных статей
Rob Amezquita (15:01:55) (in thread): > so sketchy
Rob Amezquita (15:02:10) (in thread): > its like the back alley of science
Kevin Rue-Albrecht (15:03:40) (in thread): > so long as it works:stuck_out_tongue:
Kevin Rue-Albrecht (15:04:25) (in thread): > that said, those authors may not deserve citations if we (theoretically) don’t have access to their article
Federico Marini (15:04:43) (in thread): > it is around since a good while
Federico Marini (15:04:44) (in thread): > https://www.sciencemag.org/news/2016/04/whos-downloading-pirated-papers-everyone - Attachment (Science | AAAS): Who’s downloading pirated papers? Everyone > An exclusive look at data from the controversial web site Sci-Hub reveals that the whole world, both poor and rich, is reading pirated research papers.
Rob Amezquita (19:13:25): > thank you so much@Charlotte Soneson@Kevin Rue-Albrecht@Aaron Lun@Federico Marinifor your contributions to the google doc, I really appreciate your expertise and you all are superstars for putting this together from across the globe.
Rob Amezquita (19:15:41): > i have made some preliminary trimming/editing of some of the doc, i think thereport generationsection needs to be fleshed out a bit more, theinteractive visualizationmethod just needs the citations inserted, and theintroneeds to be refined a bit. i will be taking as is probably sometime tomorrow (~24 hrs from now) to begin to incorporate into the remainder of the writing, so if you can spare a little time between now and then to add whatever you see fit that would be great (and yes, please its okay to be selfish and advertise your own packages, thats why we are doing this!).
Rob Amezquita (19:16:26): > dont worry about reaching perfection, the editing phase will smooth out incoherencies, as long as the main ideas are in writing itll likely be okay
Rob Amezquita (19:17:25): > please also make sure to add your author name/affilitation/email so that we have on record who contributed (and so i dont have to dig around for your official affilitiation and how to write that out:stuck_out_tongue:)
Rob Amezquita (19:17:41): > again thank you all!
Aaron Lun (19:29:49): > YES! And my dentist’s appointment is done!
Rob Amezquita (19:31:52): > [insert tooth emoji]
Rob Amezquita (19:32:07): > the one profession they dont have on emoji-paedia
Aaron Lun (19:32:42): > Man, that’s a lot of stress off my shoulders.
Aaron Lun (19:32:53): > The guy was like, “Aaron, your teeth are very clean”.
Aaron Lun (19:32:57): > And I was like fuck yeah.
Aaron Lun (19:33:05): > Well, I thought it at least.
Rob Amezquita (19:33:32): > wow, thats impressive…i usually get scolded for not flossing
Rob Amezquita (19:33:41): > because…i dont floss…or if i do, its very irregularly..
Aaron Lun (19:33:48): > I floss weekly
Aaron Lun (19:33:53): > I guess I don’t eat that much tho
Aaron Lun (19:33:55): > Because I’m an old man.
Aaron Lun (19:34:02): > This dentist looked younger than I was.
Rob Amezquita (19:34:08): > failing metabolism ftw
Rob Amezquita (19:34:22): > if only wisdom scaled with metabolism
Aaron Lun (19:34:27): > Not used to having a young medical professional taking care of me.
Aaron Lun (19:34:36): > I’d be all like “You can’t tell me what to do”
Aaron Lun (19:34:56): > “Back in my day we treated elders with RESPECT”
Rob Amezquita (19:35:41): > my dad the other day was like, “my doctor was YOUR age, since when did people YOUR age get to make critical healthcare decisions for people MY age”
Aaron Lun (19:36:01): > Lol
Rob Amezquita (19:36:39): > so, i think this feeling you have is somewhat universal:stuck_out_tongue:
2018-12-13
Charlotte Soneson (13:55:03): > thanks for cleaning things up a bit@Rob Amezquita- as for the citations, I can put them in. Which format do you want (?)
Rob Amezquita (13:57:13): > Yep exactly everything will end up latex ified :)
Federico Marini (14:10:42): > I can provide proudly the freshly citable doi for pcaExplorer:party_parrot:
Charlotte Soneson (14:10:52): > Already there:wink:what aboutideal?
Federico Marini (14:11:08): > hm, knife in the wound
Federico Marini (14:11:10): > it is in prep
Charlotte Soneson (14:11:17): > :+1:
Federico Marini (14:11:25): > btw, if someone is still “afraid” of preprints
Federico Marini (14:11:38): > this is a piece of email I got today:
Federico Marini (14:12:19): > > Dear Dr. Federico Marini, > > I am writing to you with regard to the pre-print article that you recently submitted to BioRxiv, looking at pcaExplorer. Apologies if you have already submitted your article to a journal but just in case you have not, I hope I am able to catch you at the right time. > > Having read your article, I believe it would be a suitable fit for BMC Bioinformatics, where we focus on exciting research covering all aspects of bioinformatics and computational biology. We pride ourselves in being ranked 3rd for Bioinformatics in SCImago and 1st in open access for the same category. As a BMC series open access journal, we are committed to publishing scientifically sound research and advancement of information in the area is key. The content of your findings are a good fit to our scope, hence I am writing to you to see whether you would be interested in submitting to BMC Bioinformatics. Although we believe your article is well suited for the journal, upon submission, your work would be assessed by an independent Associate Editor and we cannot guarantee peer-review or publication in the journal. I have included a link to our expert international Editorial Board who endeavour to offer a fair, friendly and efficient peer review service. > > Please let me know if you would like to submit to the journal by replying to this email. As an incentive, we can offer a 15% discount on the article-processing charge. Simply enter the discount code 'pre-print_15' as part of a waiver request on the payment page of your submission. In case you are also entitled to a discounted article-processing charge, as part of a membership arrangement or low-income country waiver, we will grant you whichever single discount is greater. If you have already submitted your article elsewhere, we hope you will consider us for your future publications. Please do spread the word to your colleagues and feel free to contact me if you have any questions. >
Federico Marini (14:12:38): > (fun fact: it is already submitted there since 2 weeks, but hey)
Charlotte Soneson (14:14:36): > :tada:
Kevin Rue-Albrecht (14:15:36): > :+1::gift:
Federico Marini (14:16:21): > it is still no guarantee they do send it out, but come on
Federico Marini (14:16:46): > I’m as close as ever to the matt stephens-jeff leek pre-post-preview-review
Federico Marini (14:17:10): > If you are not familiar, here’s a piece:https://medium.com/@biostatistics/guest-post-matthew-stephens-on-biostatistics-pre-review-and-reproducibility-a14a26d83d6f - Attachment (Medium): Guest post: Matthew Stephens on Biostatistics pre-review and reproducibility > Editor’s note: This is a post by Matthew Stephens who is the first person to have a paper pre-reviewed by Biostatistics for publication.
Rob Amezquita (14:19:50): > @Federico Marinican you send me th Bibtex citation for ideal
Rob Amezquita (14:19:57): > btw the doc looks fantastic everyone!
Rob Amezquita (14:20:05): > i think its ready to copy over upon the last citation
Rob Amezquita (14:20:24): > also, i need your affiliations/emails please:smile:
Federico Marini (14:23:13): > bibtex entry and email are in:wink:
Rob Amezquita (14:24:30): > perfect, we are just missing aaron,@Aaron Lunif you can send me your official name/email/affiliation just ping me
Federico Marini (14:25:38): > General thought - picking up on a quick chat with Charlotte in Munich: I think the bioc community at large is ripe in term of pushing out a status update on the project, “overall”. The:musical_note:orchestrating paper is already aged, and there is a stone to put in the single cell era. Should we ping the core members about this?
Federico Marini (14:25:47): > Easy guess, they will well be aware
Rob Amezquita (14:26:20): > thats exactly what this whole thing is about haha
Rob Amezquita (14:26:34): > so the writing you just did is going into that
Rob Amezquita (14:27:07): > steph and i are coordinating that effort, so youll be on that review, hence why advertising your packages is exactly what we want cos youre awesome
Rob Amezquita (14:27:14): > and the world should know it
Federico Marini (14:30:27): > :slightly_smiling_face:
Rob Amezquita (14:32:46): > again, thank you all for your help, we will be sending out the draft to all the authors once its in a more final state and we take everything through a few rounds of edits
Federico Marini (14:35:15): > it was our pleasure to contribute:wink:indeed the time was ripe
Aaron Lun (15:39:57): > all right alll right, I just owke up.
Aaron Lun (15:40:10): > just use my affiliaion from the issee paper
2018-12-17
Kevin Rue-Albrecht (07:23:52): > If anyone has any typo or small stuff to fix, branchvoice_vignetteis open. I’m giving myself the day to think about anything else I wanna throw in, but I’ll probably merge (without version bump yet) tomastertonight
Federico Marini (08:05:40): > quick question about the commands
Federico Marini (08:06:04): > (selection)is in parentheses because it is optional to say?
Kevin Rue-Albrecht (08:06:11): > Yup
Kevin Rue-Albrecht (08:06:23): > Want it to be explicit ?
Federico Marini (08:06:31): > probably clearest?
Federico Marini (08:06:39): > it would just be one line
Kevin Rue-Albrecht (08:07:31): > One line of what?
Kevin Rue-Albrecht (08:08:06): > Btw the reference place to look for the spoken commands ishttps://github.com/csoneson/iSEE/blob/master/inst/www/voice.js - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE
Kevin Rue-Albrecht (15:56:35): > Thanks for the feedback Fede. I’ve just pushed my last commit for this PR. Then i’ll merge
Kevin Rue-Albrecht (15:57:06): > (but not critical enough for a version bump-upstream push)
2018-12-26
Federico Marini (09:09:05): > Looks like we are all nicely christmas-busy
Stephanie Hicks (09:10:04): > :heavy_plus_sign:
Federico Marini (09:10:18): > but nice news are coming - at least advertisement wise:smile:
Federico Marini (09:10:18): > ->https://twitter.com/dataandme/status/1077927603061829632 - Attachment (twitter): Attachment > 🧬 Reproducible, interactive GUI for SummarizedExperiment objs… > :package: “iSEE: Interactive SummarizedExperiment Explorer”by @KevinRUE67, @FedeBioinfo, @CSoneson, & Aaron Lun https://buff.ly/2rUHXjd on @Bioconductor > :sparkles: shiny interactive tour: https://buff.ly/2LGzitX #rstats #dataviz
Federico Marini (09:10:31): > we might get a nice boost in visibility thanks to mara’s megaphone:slightly_smiling_face:
Federico Marini (09:10:51): > I have the feeling I am non-stop cooking since 3 days
Federico Marini (09:12:54): > probably this is not so far from the truth
Federico Marini (09:13:10): > benefit is we can feed on the rests till mid jan 2019:stuck_out_tongue:
Stephanie Hicks (10:57:19): > @Federico Marinivery cool to see the tweet from Mara!
2018-12-30
Evan Biederstedt (14:40:07): > @Evan Biederstedt has joined the channel
2019-01-04
Kevin Rue-Albrecht (12:31:23): > Hey peeps > For those missing our Shiny enthusiasm, I’ve initiated a small app as part of theHancockpackage (see#sc-signature). > To get a prototype started quickly I’ve borrowed a few pieces from the iSEE app. > That said, it got me thinking that we might want toexporta few more of our internal functions that may be handy to kickstart other apps. I’m thinking small stuff like.safe_field_name,.disableButtonIf, etc.
Aaron Lun (22:32:41): > ¯*(ツ)*/¯
2019-01-05
Kevin Rue-Albrecht (10:25:09): > Hancockis officially about to useiSEE::collapseBox- Nice one Aaron:thumbsup:https://github.com/kevinrue/Hancock/pull/32 - Attachment (GitHub): Use iSEE’s collapsible boxes by kevinrue · Pull Request #32 · kevinrue/Hancock
Kevin Rue-Albrecht (10:28:43): > That said, if any C++@Aaron Lun(referring to the literal username) can think of a magical trick to speed up my proof of conceptmakeMarkerProportionScree, I’m all earshttps://github.com/kevinrue/Hancock/blob/master/R/markerDetection-methods.R#L74Basically I want to efficiently compute the progressive combined detection rate of markers: i.e. the proportion of samples with detectable levels of the first marker, both of the first two markers, etc. - Attachment (GitHub): kevinrue/Hancock > Cell signatures, with confidence. Contribute to kevinrue/Hancock development by creating an account on GitHub.
Kevin Rue-Albrecht (10:31:29): > The idea being that I apply a threshold to keep the top N markers that aresimultaneouslydetected in at least x% of the samples
Aaron Lun (10:32:20): > I don’t really understand, but it’s 3 am over here so that’s probably got something to do with it.
Kevin Rue-Albrecht (10:32:53): > ow, sorry, still say 3:33PM (UK time) when I click on your profile
Kevin Rue-Albrecht (10:35:14): > Well go get some sleep then. Not an emergency.
Aaron Lun (23:30:01): > GAH reviewing a paper so bored
2019-01-06
Aaron Lun (01:56:33): > Got nothing to do but review this paper in excruciating detail.
Aaron Lun (02:25:17): > Oh, these poor guys are going to get a super detailed report because I’ve got nothing better to do.
Aaron Lun (03:02:40): > Alright@Kevin Rue-Albrechtwhat the hell do you want. I’m not sure I understand your example.
Aaron Lun (03:03:10): > Well, I don’t even see the point of this. Why are you doing this?
Kevin Rue-Albrecht (04:01:46): > Alright 9am over here. Let me warm up:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (04:07:08): > On my quest for a simple « marker identification » template workflow (input/output), I’ve sorted candidate markers from the most frequently to the least frequently detected
Aaron Lun (04:07:46): > Take this to the other channel.
Kevin Rue-Albrecht (04:07:46): > That, in a logical matrix that declares whether each marker (row) is detected in each sample (column)
Kevin Rue-Albrecht (04:07:54): > Ok
Kevin Rue-Albrecht (05:18:21): > I gotta disappear now, but I’ve just pushed a new branch:https://github.com/csoneson/iSEE/tree/collapseboxI’ve been using it herehttps://github.com/kevinrue/Hancock/blob/master/R/shinyLabels.R#L178and it’d be nice be properly export/import the box function in the NAMESPACE > It probably just need an example for the man page, and good to go. Except if I’m missing anything? - Attachment (GitHub): csoneson/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - csoneson/iSEE - Attachment (GitHub): kevinrue/Hancock > Cell signatures, with confidence. Contribute to kevinrue/Hancock development by creating an account on GitHub.
2019-01-07
Stephanie Hicks (09:03:56): > fwiwhttps://www.biorxiv.org/content/early/2019/01/06/512582 - Attachment (bioRxiv): scSVA: an interactive tool for big data visualization and exploration in single-cell omics > We present scSVA (single-cell Scalable Visualization and Analytics), a lightweight R package for interactive two- and three-dimensional visualization and exploration of massive single-cell omics data. Building in part of methods originally developed for astronomy datasets, scSVA is memory efficient for more than hundreds of millions of cells, can be run locally or in a cloud, and generates high-quality figures. In particular, we introduce a numerically efficient method for single-cell data embedding in 3D which combines an optimized implementation of diffusion maps with a 3D force-directed layout, enabling generation of 3D data visualizations at the scale of a million cells. To facilitate reproducible research, scSVA supports interactive analytics in a cloud with containerized tools. scSVA is available online at https://github.com/klarman-cell-observatory/scSVA.
Aaron Lun (09:06:12): > God their code is ugly.
Aaron Lun (09:06:39): > Perhaps we could sneak in to figure out how they generate ggplots quickly for “hundreds of millions of cells”.
Aaron Lun (09:07:05): > We were certainly never able to do that without some hacks.
Stephanie Hicks (09:09:36): > I am curious about how they did that too….
Aaron Lun (09:09:37): > 3D vis…chuckle.
Aaron Lun (09:10:00): > Good for public outreach and impressing rubes.
Aaron Lun (09:10:51): > If you’re going to dimred to 3D, you might as well knock another dim off and get it to 2D. Y’know, so you can print it out and look at it.
Kevin Rue-Albrecht (09:13:22): > I’m peaking at the occurences ofggplotin their code, nothing magical there, and I don’t feel like deciphering what’s going on upstream in the data preparation, e.g.vals$X[1] <- vals$X[1]-(vals$X[2]-vals$X[1])*0.005
Kevin Rue-Albrecht (09:15:34) (in thread): > Genuinely curious to see how they’ll sell this interactive viewer. Might learn a public outreach trick or two
Aaron Lun (09:16:15) (in thread): > Probably with the full power of the Broad marketing machine.
Aaron Lun (09:16:49) (in thread): > coughENCODEcough
Kevin Rue-Albrecht (09:23:31) (in thread): > No mention toiSEEthough in the preprint:cry:
Aaron Lun (09:24:40) (in thread): > Makes sense, as they’re pretty self obsessed.
Kevin Rue-Albrecht (09:28:12) (in thread): > Indeed, existing tools [2-4] require extensive memory resources to explore and visualize cells and cell features, and are not scaled to massive data volumes > 2. Wolf, F. A., et al. (2018). Genome Biol. 19:15 > 3. Satija, R., et al. (2015). Nat. Biotechnol. 33:495 > 4. Fan, J., et al. (2016). Nat. Methods 13:241
Kevin Rue-Albrecht (09:29:07) (in thread): > Meh. It’s a preprint.. still room for improvement
Aaron Lun (09:29:17) (in thread): > Now that I’m moving to industry, I’ve ceased worrying about the back-slapping and mutual admiration societies that I see in the literature.
Kevin Rue-Albrecht (09:32:48) (in thread): > It’s not as much “back-slapping” that I’m after, as adequate references, especially when they’ve been available for >6 months
Aaron Lun (09:33:21) (in thread): > Well, single-cell is pretty cliquey
Kevin Rue-Albrecht (09:33:37) (in thread): > This preprint introduces an interactive viewer, and only references command-line tools, AFA I’m aware
Aaron Lun (09:35:16) (in thread): > seurat has a shiny interface. pagoda probably has one too. don’t know what scanpy has on this front.
Federico Marini (10:51:27): > > If you’re going to dimred to 3D, you might as well knock another dim off and get it to 2D. Y’know, so you can print it out and look at it. > Why print when you can enter that as VR:stuck_out_tongue:
Aaron Lun (10:52:28): > Sounds like a PR stunt.
Aaron Lun (10:53:32): > Oh wait, my institute got a £20 million grant to do just this. I guess I should say that it’s a good idea, then.
Federico Marini (10:55:08): > wut?
Federico Marini (10:55:14): > virtual reality for dim red?
Aaron Lun (10:58:06): > For 3D exploration of breast tumours.
Aaron Lun (10:58:23): > Guess this is what passes for science these days.
Stephanie Hicks (11:01:16): > @Aaron Lunif you want to really be ahead of the game:wink:, you should submit a paper to nature on 3D ScatterPlotRs as a follow up to BoxPlotR (https://www.nature.com/articles/nmeth.2811) - Attachment (Nature Methods): BoxPlotR: a web tool for generation of box plots > BoxPlotR: a web tool for generation of box plots
Aaron Lun (11:01:32): > Indeed!
Aaron Lun (11:01:45): > Honestly?! Nat Meth for box plots!
Stephanie Hicks (11:01:56): > yup
Kevin Rue-Albrecht (11:01:59): > BoxR was taken? Sounds like a package I could write at home..ahem
Stephanie Hicks (11:02:05): > i’m telling you, we’re doing this all wrong
Stephanie Hicks (11:02:12): > we need to submit papers and packages called ScatterPlotR
Stephanie Hicks (11:02:24): > Pie ChaRt
Stephanie Hicks (11:02:45): > barplotR
Stephanie Hicks (11:03:01): > histogramR
Stephanie Hicks (11:03:11): > we’d be famous:joy_cat:
Charlotte Soneson (11:05:30): > or infamous…
Stephanie Hicks (11:05:56): > :sad-parrot:
Federico Marini (11:06:17): > I remember I laughed and got quite angry when I saw that MS come out
Stephanie Hicks (11:06:48): > you and me both
Aaron Lun (11:09:00): > The part of me that cared about publications died last year (maybe just after the Heidelberg conference?) so I now only look on these things with mild amusement.
Federico Marini (11:09:49): > lucky you Aaron:stuck_out_tongue:
Federico Marini (11:11:26): > is there a part of you still caring about stickers and spreading their love in the bay city?
Stephanie Hicks (11:18:48): > also what happened in the heidelberg conf?
Davide Risso (13:05:58): > what? wait!@Aaron Lunare you moving to industry?? What will happen to Bioconductor?:scream:
Kevin Rue-Albrecht (13:07:47): > It’ll get expensive:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (16:51:44): > Hi, I’ve been sitting down staring at the Bioc2019 workshop template syllabus for a bit now, wondering which section to begin with. It’s kind of intimidating, even (or especially) looking at those from past couple of years for ideas. > Maybe I just need to get warmed up, but I’m also happy for tips and suggestions. > Also, by myself, I’d probably settle for a 1-hour thing, but with some extra hands on deck it could turn into a nice all-round tour of various modalities of genomics data. Any takers?
Kevin Rue-Albrecht (16:52:05): > https://github.com/kevinrue/BioC2019/tree/isee/docs - Attachment (GitHub): kevinrue/BioC2019 > BioC2019: Where Software and Biology Connect. Contribute to kevinrue/BioC2019 development by creating an account on GitHub.
Kevin Rue-Albrecht (17:00:13): > (Draft saved as file ‘README.md’ at the above link, as requested per submission guidelines)
Aaron Lun (18:37:11): > What.
Aaron Lun (18:37:21): > Sounds like a topic for a separate channel.
2019-01-08
Kevin Rue-Albrecht (03:15:30): > Separate channel for an iSEE workshop?
Kevin Rue-Albrecht (03:16:20): > I can move it to#bioc2019, but the place to find help and feedback on this particular subject seems here to me
Aaron Lun (03:24:01): > Oh, didn’t notice that.
Aaron Lun (03:24:11): > thought you were just going on about Bioc2019 in general.
2019-01-09
Federico Marini (04:41:24): > Hi<!channel>, is this something worth a shot?https://blog.rstudio.com/2019/01/07/first-shiny-contest/ - Attachment (blog.rstudio.com): Announcing the 1st Shiny Contest | RStudio Blog
Aaron Lun (04:41:48): > ¯*(ツ)*/¯
Aaron Lun (04:41:57): > Might as well throw our hat in. Get some publicity if nothing else.
Federico Marini (04:42:56): > Shall we put it to “vote”? Or as you said, it would be a bummer to not try it
Federico Marini (04:43:19): > worst case scenario: lots of people stumble around, the app gets viewed
Charlotte Soneson (04:47:23): > Seems worth a shot to me, I guess it shouldn’t be too much work (how was it again, did we manage to deployanythingonshinyapps.io, there were some memory issues, right?)
Kevin Rue-Albrecht (04:48:07): > Sounds nice. Challenge for us was to fit it on a shinyapps free instance (even the Allen dataset), and the other criteria is RStudio cloud, which I haven’t played with either.
Stephanie Hicks (06:22:43): > Looks like a fun contest! What’s the memory limit?
Kevin Rue-Albrecht (06:28:01): > 1GB. I wonder whether we make more copies of the data that we’d expect
Federico Marini (08:44:08): > “Worth a shot” seems the unanymous reaction
Federico Marini (08:47:26): > I have a week quite full of appointments but I’ll try to check whether and how this could be feasible
Federico Marini (08:47:49): > Given the fact that with so many interruptions, it is not so possible to get focused for real
Federico Marini (08:48:06): > plus, I have a small grant application I would like to take care of
Kevin Rue-Albrecht (08:48:32): > I just replied to an email where a PhD student listed the analyses thatheagreed withhisPI thatIwould do, forhislab meeting. Busy times ahead:stuck_out_tongue:
Federico Marini (08:49:06): > oh, somebody with troubles with pronouns:smile:
Kevin Rue-Albrecht (08:50:55): > I don’t know when I’m gonna find the time to prepare that Bioc2019 iSEE workshop syllabus, let alone the workshop itself. Exactly as you wrote: getting focused is a rare commodity.
Federico Marini (10:00:15): > Ah, other one small info: I got my presentation at the DAGStat accepted
Federico Marini (10:00:37): > i.e. the one for which you might have received a notification for being listed as coauthors
Federico Marini (10:01:13): > not knowing whether the acceptance notification also go out to you, I’d say it rather one extra time than one less:wink:
Charlotte Soneson (10:01:35): > :+1:(I did not receive an acceptance notification)
Federico Marini (10:04:39): > ok, so then it was good so
Federico Marini (10:20:09): > Ok, this is the last one for today: a mail just came in from a former colleague of mine, now working at Boehringer Ingelheim
Federico Marini (10:20:50): > no details in the email, but more a heads up -> > > Dear Federico, > first of all a happy new year to you, I hope you had a good start. > > I'd like to e-introduce you to Yang Shen from my team – he's started to use iSEE as the visualization component for our scRNA-Seq pipeline, and we like it quite a bit. He has done a few customizations, which might be interesting to contribute upstream. > > All the best, > Holger >
Federico Marini (10:21:30): > So, if we soon happen to see a fork and/or a PR from him, we should not be surprised
Federico Marini (10:21:42): > I don’t know what his changes would be about
Kevin Rue-Albrecht (10:23:31): > Well it’s pretty positive to see people not scared of diving into the package source code themselves, at least
Federico Marini (10:26:36): > :thumbsup:
Aaron Lun (23:47:33): > Make sure they keep us in the loop, it would be sad for them to create a feature that we have to reject.
2019-01-10
Kevin Rue-Albrecht (03:35:57): > Obviously, it’s better if they communicate with us “human-to-human”, but you can always track their progress and get a good idea of where they’re at by visiting their GitHub branch (https://github.com/shenyang1981/iSEE/tree/enhancedCustomPanel) and if you click on that page the “New Pull Request” button, you’ll even get adiffrelative to ourcsoneson/masterbranch - Attachment (GitHub): shenyang1981/iSEE > R/shiny interface for interactive visualization of data in objects derived from the SummarizedExperiment class - shenyang1981/iSEE
Kevin Rue-Albrecht (03:54:39): > @Charlotte Sonesonabout the email I just sent, I just realized thatcytofkitis deprecated for R-devel. Are you familiar with it or its developers?
Kevin Rue-Albrecht (03:55:12): > (ah. just got you email reply:sweat_smile:)
Charlotte Soneson (03:57:34): > Familiar in the sense that I have met the developer and talked about it, but I haven’t really used it (other people in Mark’s group will probably know more, I remember we discussed it and were a bit surprised that it was being deprecated).
Kevin Rue-Albrecht (04:00:28): > Ok. Well even my colleague in his email pointed out that the package was released in 2016, but that they didn’t maintain it in the meaning of adding new clustering or reducedDim methods
Kevin Rue-Albrecht (04:02:07): > Still, now that I look, “last commit: 2 Mar 2018” (https://github.com/JinmiaoChenLab/cytofkit) - Attachment (GitHub): JinmiaoChenLab/cytofkit > cytofkit: an integrated flow/mass cytometry data analysis pipeline - JinmiaoChenLab/cytofkit
Federico Marini (04:50:03): > Re: this set of changes proposed by Shen, should we invite him over to the channel?
Kevin Rue-Albrecht (04:51:43): > Might be faster to deal with than GH or emails.
2019-01-14
Federico Marini (10:39:30): > quick update on my efforts to get the rstudio cloud instance for iSEE running
Federico Marini (10:40:10): > It is quite hard, if at all possible, to have our data and setup fit in the quite limited resources they are providing
Kevin Rue-Albrecht (10:40:25): > Not sure about ‘update’.. did I forget the ‘introduction’ ?
Federico Marini (10:40:58): > this:https://community-bioc.slack.com/archives/C8BJLSP8T/p1547026884031300 - Attachment: Attachment > Hi <!channel>, is this something worth a shot? https://blog.rstudio.com/2019/01/07/first-shiny-contest/
Kevin Rue-Albrecht (10:42:37): > ahhh right, that does need a rstudio cloud instance
Federico Marini (10:42:39): > You should be able to track my efforts here:
Federico Marini (10:42:40): > https://rstudio.cloud/project/164719
Federico Marini (10:42:46): > this should be set to public now
Federico Marini (10:43:04): > gotta say it is still pretty buggy - but hey, they say it is in alpha
Charlotte Soneson (10:43:11): > that’s a pity. Are you saying that the app itself is too big, and that it won’t help even if we subset the data a lot?
Federico Marini (10:43:17): > I was trying to subset
Charlotte Soneson (10:43:34): > btw, the link takes me to a login page
Kevin Rue-Albrecht (10:43:56): > same here, I’ve signed in using GitHub
Federico Marini (10:44:08): > don’t know if login is required also for viewing
Kevin Rue-Albrecht (10:44:14): > and now I can see an RStudio Server UI
Kevin Rue-Albrecht (10:45:00): > uh.. is that normal that I can do more than just viewing Fede? Even though it’s your instance?
Kevin Rue-Albrecht (10:45:35): > At least, I ’m signed in as me, but at the top left it says “Your Workspace iSEE_rsc Federico Marini”
Federico Marini (10:46:21): > yes. everyone can do read-only, yet modify a copy
Federico Marini (10:46:26): > that does not get saved
Federico Marini (10:46:34): > unless you “fork” it
Kevin Rue-Albrecht (10:46:36): > ow
Kevin Rue-Albrecht (10:46:41): > nice
Charlotte Soneson (10:47:00): > ok, I can see it too:+1:
Federico Marini (10:48:25): > so, subset to 50
Federico Marini (10:48:33): > and with some efforts, I can go up to launching
Federico Marini (10:48:54): > but I can not really see the app running, it says connection refused
Federico Marini (10:49:35): > kinda sucks, still, “overall”
Federico Marini (10:49:45): > the installation procedures are far from bug-free
Federico Marini (10:50:04): > don’T ask me how I got iSEE installed without installing some relevant dependencies:smile:
Federico Marini (10:50:39): > OH
Federico Marini (10:50:41): > now it runs
Federico Marini (10:51:12): > … and it crashed while I was taking a screenshot
Federico Marini (10:51:30): > imma try again
Charlotte Soneson (10:52:45): > what about subsetting the genes
Kevin Rue-Albrecht (10:52:50): > It’s a bit sad to go down to so few samples, but you could give a try to theairwaydataset, as the smallest possible example dataset?
Federico Marini (10:53:38): - File (PNG): Pasted image at 2019-01-14, 4:53 PM
Federico Marini (10:54:27): - File (PNG): Pasted image at 2019-01-14, 4:54 PM
Kevin Rue-Albrecht (11:23:11): > So… how stable is it?
Federico Marini (11:44:40): > not so stable, TBH
Federico Marini (11:44:58): > I fear it would just give frustration to people trying to use it:stuck_out_tongue:
Federico Marini (11:45:05): > and that is not exactly our aim
Kevin Rue-Albrecht (11:46:22): > Is that already after downsampling genes?
Kevin Rue-Albrecht (11:48:01): > For a demo, I doubt anyone would mind playing with no more than 500 genes
Federico Marini (11:48:11): > no, but it is already quite reduced in its size
Federico Marini (11:48:19): > it looks to me “overall quite instable”
Kevin Rue-Albrecht (11:48:23): > ahh
Federico Marini (11:48:37): > feel free to get a minimal running instance
Kevin Rue-Albrecht (11:48:47): > well.. if it’s beyond iSEE, it should affect everyone equally
Federico Marini (11:49:00): > probably it is just this cloud thingy
Kevin Rue-Albrecht (11:49:00): > (i.e., all the other apps too)
Federico Marini (11:49:27): > we have the disadvantage of the dependencies - many other apps dont have that much of a prob
Kevin Rue-Albrecht (11:50:07): > Exactly. Speaking of which I only looked at the first one and it was a simple yet fun concept:https://community.rstudio.com/t/shiny-contest-submission-create-visual-abstracts-for-original-research/21530 - Attachment (RStudio Community): Shiny Contest Submission: Create Visual Abstracts for Original Research > Hello, The idea of ‘Visual Abstracts’ have recently become popular and are particularly efficient in distilling the main findings of original research. I wanted to create an application that enabled users (or non programmers in general) to create nice visual summaries for their research projects they could share over social media. I’ve added a fair bit of control and customization in the application mostly related to layout to aesthetics. Technical pieces: -uses ‘rintrojs’ to orient users to…
Federico Marini (11:56:51): > makes sense
Federico Marini (11:57:03): > I would not be surprised to see such e-posters soon
Federico Marini (11:57:16): > people walk around and get a tour of the findings
Kevin Rue-Albrecht (18:27:06): > happy to hear feedback –>https://github.com/kevinrue/BioC2019/tree/isee/docs
Kevin Rue-Albrecht (18:29:33): > Also, do let me know if any of you is planning to go. I don’t see~m~any workshops in past years that had a single instructor
2019-01-15
Charlotte Soneson (11:02:05): > From a user question - does anyone remember why we don’t allow row/col stat tables to transmit selections to plots? > > row_selectable <- c(.noSelection, link_sources$row_plot) > col_selectable <- c(.noSelection, link_sources$col_plot) > > I guess there is a good reason but I just can’t remember…
Kevin Rue-Albrecht (11:11:45): > I’m also a bit out of practice in terms of detailed implementation, and I think I’m not picturing the question correctly: the (single) selected row of tables can be sent to other panels, no?
Charlotte Soneson (11:13:03): > I don’t think it can. - File (PNG): Screen Shot 2019-01-15 at 17.12.34.png
Charlotte Soneson (11:13:13): > Row statistics table 1is not selectable
Kevin Rue-Albrecht (11:24:18): > ow. > hang on, now, I’m confused with the “coloring a single data point” that is one of the last things I remember adding a couple of months ago, where the selected row was indicated as a bigger colored point. > Where did this feature go? (I’m trying to wrap up something else right now, but I can check later tonight if you can’t find it)
Charlotte Soneson (11:26:45): > Yeah, that works fine - File (PNG): Screen Shot 2019-01-15 at 17.25.11.png
Kevin Rue-Albrecht (11:30:30): > Ah. So it is “Color by”… Isn’t that “almost” the same thing? The only difference that I can see is that the source is a table instead of a dropdown menu, right? If that’s the case there’s probably some code that we can re-use
Charlotte Soneson (11:32:45): > That’s what I felt too, but I wasn’t sure if there was a good reason for not allowing it. The use case in this case is to select a gene to show the expression levels of in a custom panel.
Kevin Rue-Albrecht (11:36:29): > Well, intuitively, Ithinkit makes sense to allow that
Federico Marini (13:40:37): > @Kevin Rue-Albrecht, quick feedback on the Bioc2019 proposal:
Federico Marini (13:40:44): > looks fine to me
Federico Marini (13:41:04): > one sentence I did not catch what you meant isUnderstand the difference between [RStudio Shiny](https://shiny.rstudio.com) and [*iSEE*](http://bioconductor.org/packages/iSEE/) apps
Federico Marini (13:41:16): > iSEEisa shiny app as such
Federico Marini (13:41:52): > so maybe you meant “peculiarities of iSEE”, or something else?
Kevin Rue-Albrecht (13:49:56): > I should probably remove that one, I was brainstorming and stretched it a bit. I was thinking about more novice users: having a tiny introduction to shiny itself (think old faithful demo app) to show what shiny is, before going hardcore into how iSEE built on that
Federico Marini (13:52:28): > ah ok, then maybe the “unique feats of iSEE”?
Federico Marini (13:52:41): > because you are indeed already requesting some degree of familiarity with shiny
Federico Marini (13:52:49): > which again makes totally sense
Kevin Rue-Albrecht (13:54:27): > No problem. Keep throwing ideas how to phrase it, the other story behind the “under the difference…” is that those words were actually part of the template,
Federico Marini (13:55:30): > Will do, as soon as they pop out I’ll report them/suggest the edits
Kevin Rue-Albrecht (13:55:31): > You have no idea how much time it took me to finally rewrite that template for iSEE
Federico Marini (13:55:56): > hm writer’s blockades are common to all of us sometimes
Kevin Rue-Albrecht (13:56:24): > Between that and the past year examples, I felt like nothing was good enough:sweat_smile:
Kevin Rue-Albrecht (13:57:17): > Obviously putting me against final versions was the best way to discourage myself:yum:
Aaron Lun (14:48:01) (in thread): > “Colour by” should already allow table selection, see the second selectize in@Charlotte Soneson’s picture.
Aaron Lun (14:50:23): > The select machinery is not particularly amenable to transmitting data from tables.
2019-01-21
Kevin Rue-Albrecht (04:46:06): > @Federico Marini: from what I can see on GH you’re the one who took care of.snapshot_graph_linkedpanels. I was curious if you were interested in looking into graphs of gene:set relationships forhttps://github.com/kevinrue/unisets?
Kevin Rue-Albrecht (04:50:53) (in thread): > This weekend I’ve added a method that allows stuff likebase_sets <- as(org.Hs.egGO, "BaseSets"), and from the Google doc pinned to the#gseabasechannel, one possibly interesting use would be to plot the relationships as a graph (I’d imagine genes and sets as nodes of different colors, and that’s about it). > If you’re interested, we can move the discussion to#gseabasechannel
2019-01-23
Aaron Lun (05:32:02): > One of my colleagues longed for the days ofscran::selectorPlot, which provides a simple interface for selecting many sets of points in a single plot. I don’t want to restore it inscran(which would undermine my motivation for making iSEE in the first place) but putting it iniSEEwould make a weird appendage to the main app. Thoughts?
Kevin Rue-Albrecht (05:40:29): > It’s really tricky to decide of a return value for the main iSEE app. So much can happen in a single session that we’d also need a way to “clear” the memory of sets selected “so far at any given time”. > As you describescran::selectorPlot(I personally never used it), it would make more sense to have “mini-apps” that handle a single plot. > I hope I’m not going off-track here.
Aaron Lun (05:41:28): > I “want” (not really, but it does seem useful) to prettify the selectorPlot and put it into iSEE.
Kevin Rue-Albrecht (05:42:23): > iSEE: the package, or the main app?
Kevin Rue-Albrecht (05:42:49): > i.e., as a separate app, or as a panel of the main app?
Kevin Rue-Albrecht (05:43:43): > (not even sure whether the latter makes any sense, as there’s probably a need for different selectorPlot behaviours for different types of plots, e.g. rowDataPlot should memorize row selections, while memorize column selection )
Aaron Lun (05:45:30): > the package, not the app.
Aaron Lun (05:45:50): > it would be a drop-in replacement forplot()withxandyvalues.
Aaron Lun (05:47:28): > not designed for deep exploration at all - just intended to provide a quick-and-dirty interface for times when you “just want to know what that group of points were”.
Kevin Rue-Albrecht (05:54:00): > You’re making me wonder whether it’d be possible to “toggle” existing plotting panels into that kind of “selector” mode, i.e. a checkbox that “while TRUE”, records each selection into a list/vector that’s returned to the user session when closing the app. > It would only be recording when checked, and could record each individual selection made a list of character vectors. The list could be named by plotting panel: > > list( > colDataPlot1 = list( > selection1=character(10), > selection2=character(5) > ) > ) >
Kevin Rue-Albrecht (05:56:12): > It’d be pretty straightforward for users tounion(unlist(...))to get their > > selecting many sets of points in a single plot
Aaron Lun (08:35:44): > I suspect that would be pretty gross.
Aaron Lun (08:38:44): > It would be better to have a separate function that creates a colData field of selected points for people to do shit with.
Aaron Lun (08:38:57): > … which actually seems pretty reasonable to me.
Aaron Lun (08:39:33): > This would allow you to interactively create arbitrary metadata fields in one shiny app, that you then use iniSEEitself.
Aaron Lun (08:56:11): > okay, I’ve decided. I’ll do it!
Aaron Lun (13:08:37): > Initial draft onselectorbranch. Run the example, brush or lasso, and add to list. Then when you “return to R”, you will get a factor containing the groupings of each point.
Aaron Lun (13:09:47): > Volunteers requested for beautification of the interface. Most immediately, I would like the function to accept a ggplot object so that we can re-use our “add lasso” functionality.
Aaron Lun (13:12:01): > The idea is to use this app to create column (and/or row) metadata for use iniSEEor elsewhere. For example, you select groups for DE analyses. I don’t think there’s a need to have this functionality in iSEE itself, because the newly created groups aren’t that useful unless you know why they’re different, and for that you’d need to do a DE analysis anyway.
Aaron Lun (15:16:30): > HOLD THE PHONE
Aaron Lun (15:16:40): > Just had an inspiration.
Aaron Lun (16:41:59): > BEHOLD.multiselectbranch. Options in the select box, see if you can figure out what’s going on.
Aaron Lun (16:42:17): > And with that, I’m going home.
2019-01-24
Aaron Lun (03:11:29): > Immediate aesthetic things TODO for this branch: > - Prettify the selection message, in terms of improving spacing above and below the text. I think this is doable with<p>in theHTML()call. > - Co-plot the stored selections on the same plot, with numbering in the center of the space to indicate the selection number.
Aaron Lun (03:13:26): > There is a known issue with the “overwrite” button, in that an active brush is replaced in the app’s memory but not from the plot. That is, the brush in memory (and the “hard” brush in the plot) will not match up to the “soft” brush on the plot. This will require some moderately risky surgery to fix, so I suggest putting this aside for the time being.
Aaron Lun (03:13:57): > Tagging the work crew@Charlotte Soneson@Kevin Rue-Albrecht@Federico Marini. I’ve got a visa interview tomorrow so I can’t do this right now.
2019-01-25
Aaron Lun (13:31:51): > Hello? Anyone there?
Aaron Lun (13:31:56): > I got my visa approved, BTW.
Federico Marini (15:01:08): > Hi Aaron, sorry I missed the last days out
Federico Marini (15:01:28): > I was full head into writing a small proposal
Federico Marini (15:02:06): > Good your visa worked out - all set up for moving?
Aaron Lun (15:59:00): > Got to get through the 4 week notice first.
Aaron Lun (15:59:20): > Oh, and organize all my shit on the other side.
Stephanie Hicks (22:49:01): > Congratulations!!!:tada:
Stephanie Hicks (22:49:20): > Does that mean we can come visit you next we are in San Fran?:slightly_smiling_face:
2019-01-26
Aaron Lun (09:55:08): > I guess you could. I probably won’t have a spare room though.
Aaron Lun (09:55:27): > Too expensive to rent a place with an extra room that won’t be used most of the time.
Aaron Lun (10:02:18): > At any rate, we’re getting off-topic. I need volunteers to help me do the thing I wrote above.
2019-01-29
Kevin Rue-Albrecht (11:27:28): > Sorry about the lack of response. These days are just so infinitely short compared to pile of things that are coming my way. Haven’t even had time topullyour branch and check it out:sweat:
Aaron Lun (12:21:11): > y’know, if you invited me to your wedding, I could’ve helped the organization.
Kevin Rue-Albrecht (12:22:52): > uuuuh I would like to see you debate with Anna and hermamma
Kevin Rue-Albrecht (12:23:40): > (I meant in particular about the wedding organization)
Kevin Rue-Albrecht (12:26:27): > You have no idea how helpful her dad was, to basically book all the B&B’s around their place, because there’s no hotel big enough in their town to host all our guests from abroad
Kevin Rue-Albrecht (12:26:54): > I don’t think there’s any good website or script anywhere to bargain which each place the way he did:stuck_out_tongue:
Aaron Lun (12:26:59): > lol
Aaron Lun (12:27:14): > I guess I could have helped with something simple
Aaron Lun (12:27:18): > like choosing the dress or something.
Kevin Rue-Albrecht (12:28:20): > I wasn’t there but if I heard correctly Anna made the wedding shop employee cry:wink:
Stephanie Hicks (12:28:29): > ….
Stephanie Hicks (12:29:11): > oh my!
Kevin Rue-Albrecht (12:31:18): > she knows what she wants and no one’s gonna tell her otherwise hehehe:stuck_out_tongue_winking_eye:
Stephanie Hicks (12:31:52): > i mean, i do remember how stressful planning our wedding is. Congratulations though!!
Kevin Rue-Albrecht (12:32:36): > Thanks! I’ll take the wishes now or after, because I was told I wouldn’t have access to Slack that day:cry:
Stephanie Hicks (12:33:00): > :thumbsup:sounds more than reasonable
Kevin Rue-Albrecht (12:34:23): > can’t wait:grin:
Aaron Lun (12:35:51): > Personally I’d go for a strapless dress.
Kevin Rue-Albrecht (12:37:31): > Why do you ask me for guest then? You should have asked Anna about being bridesmaid:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (12:38:45): > Anyway, that said, it’s Naples and it’s May, so it’s likely she’s gone for strapless
Stephanie Hicks (12:39:02): > ooh lovely!
Kevin Rue-Albrecht (12:40:56): > Yeah:grin:i keep missing the#meetupsevery time I go there though:stuck_out_tongue:
Aaron Lun (12:43:58): > Yeah strapless FTW!
2019-01-30
Federico Marini (05:50:55): > about the silence in the channel: I have still couple of days before the deadline, then I am happy to take a look
2019-02-02
Aaron Lun (14:31:15): > CRUSHED MY EYE TEST.
2019-02-06
Kevin Rue-Albrecht (04:57:24): > Guys… just got wind ofthat: > -https://isee.cellgeni.sanger.ac.uk/treg-tomas/- from the Treg link of this page:http://www.teichlab.org/data/(haven’t checked yet whether there are other gems hidden there)
Aaron Lun (05:04:30): > Some of the 10x links are broken.
Federico Marini (05:43:28): > yet: way to go:slightly_smiling_face:
Federico Marini (05:43:37): > Did you pitch it to Sarah Aaron?
Federico Marini (05:43:54): > Or is it the convincing action over Vladimir Kiselev?:slightly_smiling_face:
Aaron Lun (05:46:15): > Dunno. Probably Vlad.
Aaron Lun (05:46:31): > As if PIs would know how to set up a Shiny server. I know John doesn’t.
Aaron Lun (05:46:42): > Probably doesn’t even know how to ssh anymore.
Aaron Lun (05:47:00): > If you told him ssh, he’d probably think you were telling him to be quiet.
Federico Marini (05:47:07): > well at least you can convince them that it is useful
Federico Marini (05:47:23): > eheheh
Federico Marini (06:27:58): > I should keep track of these examples out in the wild
Federico Marini (06:28:10): > and maybe link to them for the iSEE instances repo
Kevin Rue-Albrecht (06:28:31): > exactly
Kevin Rue-Albrecht (06:29:20): > Makes me think we could have yet another GH repo:awesome-isee
Kevin Rue-Albrecht (07:31:27): > @Federico Mariniif you’re still into thatpkgdownthing (which I’ve just become a huge fan), I’ve just played with it this morning for a separate project:https://github.com/kevinrue/unisetsYour last suggestion on the related issue (https://github.com/csoneson/iSEE/issues/250) works like a charm, and doesn’t pollute themasterbranch the slighest. > Just make sure you follow exactly the instructions listed here:https://pkgdown.r-lib.org/reference/deploy_site_github.htmlespecially > > git checkout --orphan gh-pages > git rm -rf . > git commit --allow-empty -m 'Initial gh-pages commit' > git push origin gh-pages > git checkout master > - Attachment (pkgdown.r-lib.org): Deploy a pkgdown site on Travis-CI to Github Pages — deploy_site_github > deploy_site_github() sets up your SSH keys for deployment, builds the site with build_site(), commits the site to the gh-pages branch and then pushes the results back to GitHub. deploy_site_github() is meant only to be used by the CI system on Travis, it should not be called locally. See ‘Setup’ for details on setting up your repository to use this.
Federico Marini (07:58:32): > Good to hear Kev
Federico Marini (07:58:48): > I never went all-in in checkign out whether that worked
Federico Marini (07:59:07): > this can be relevant also for@Charlotte Soneson, we chitchatted recently about pkgdown
Federico Marini (08:03:59): > should we set it up for iSEE as well?
Aaron Lun (08:49:46): > I don’t care about what I can’t see onmaster.
Federico Marini (09:08:28): > … which is a perfect non-acronym for yes:slightly_smiling_face:
Kevin Rue-Albrecht (09:26:10): > While I still remember: it will require@Charlotte Sonesonto runtravis::use_travis_deploy()(seehttps://pkgdown.r-lib.org/reference/deploy_site_github.html) if she hasn’t done it already - Attachment (pkgdown.r-lib.org): Deploy a pkgdown site on Travis-CI to Github Pages — deploy_site_github > deploy_site_github() sets up your SSH keys for deployment, builds the site with build_site(), commits the site to the gh-pages branch and then pushes the results back to GitHub. deploy_site_github() is meant only to be used by the CI system on Travis, it should not be called locally. See ‘Setup’ for details on setting up your repository to use this.
Charlotte Soneson (09:27:14): > I haven’t.
Federico Marini (09:27:35): > She might be experimenting on the soon to be on our screensalevinQC?
Federico Marini (09:27:37): > oh
Federico Marini (09:27:41): > ok:smile:
Charlotte Soneson (09:28:07): > I don’t think so at least - should I have done so?
Federico Marini (09:28:40): > as Aaron & Kev combinedly said, it is the way to keep the master as immaculate as possible
Charlotte Soneson (09:29:19): > Sure - but I mean, should I have already done this for another reason?
Federico Marini (09:29:55): > it is not mandatory, as long as you’Re fine in usingdocsinmaster
Kevin Rue-Albrecht (09:30:02): > well, no harm isnothaving done it yet; plus, as far as I read, it will apply account-wide on GH, not justiSEE, so up to you
Kevin Rue-Albrecht (09:32:16): > I just don’t mind signing away automated access to my GH, my right to an attorney and my first-born child:stuck_out_tongue_winking_eye:https://www.independent.co.uk/life-style/gadgets-and-tech/news/londoners-agree-to-give-up-first-born-child-in-exchange-for-free-internet-9764291.html - Attachment (The Independent): Londoners agree to give up first-born child for free net use > Experiment with ‘Herod clause’ exposed lax security with open wi-fi networks
Charlotte Soneson (09:33:55): > :scream:ok - let me have a look, I heard about this thing but so far I only tried thedocs/way
Kevin Rue-Albrecht (09:39:06): > @Federico MariniJust to clarify > > git checkout --orphan gh-pages > git rm -rf . > git commit --allow-empty -m 'Initial gh-pages commit' > git push origin gh-pages > git checkout master > > is important to make a blank slate on thegh-pagesbranch, because > > deploy: > provider: script > script: Rscript -e 'pkgdown::deploy_site_github()' > skip_cleanup: true > > will raise an error if the content at the root of the repository (ongh-pagesbranch) is not either empty or the output of an earlier deployment.
Kevin Rue-Albrecht (09:39:45): > ….learn from my mistakes… I initially branchedgh-pagesfrommasterand that didn’t end well:wink:
Federico Marini (15:31:49): > I was in a meeting today and had to hold my laugh brakes a lot when one very pompous professor was expecting more methods for so calleddata driven analysis
Aaron Lun (17:19:25): > What do they think we do? Theoretical analysis?
Kevin Rue-Albrecht (17:20:25): > > setMethod("plot", c("scRNAseqDataSet"), function(x) { plotTSNE(...) }) >
Aaron Lun (17:21:04): > I can only hope you’re being sarcastic there.
Aaron Lun (17:21:21): > Because, obviously, the future is UMAP.
Aaron Lun (17:21:28): > Duh.
Federico Marini (17:21:36): > the new sheriff in town
Federico Marini (17:21:57): > puh
Federico Marini (17:22:17): > if it was the sheriff new in town, the acronym would dangerously be…
Federico Marini (17:22:23): > tSNiT
2019-02-07
Aaron Lun (13:35:25): > Was so bored:https://github.com/LTLA/MoeDCPLogo
Federico Marini (14:33:37): > This should actually deserve its own channel:stuck_out_tongue:
Federico Marini (14:33:54): > Aaron boy, did you settle in already in SF?
Federico Marini (14:34:07): > or what keeps you so idle at the moment?:stuck_out_tongue:
Aaron Lun (16:42:17): > I killed all my projects, just waiting to die here.
Aaron Lun (16:43:27): > Feel free to tweet it
2019-02-08
Federico Marini (02:56:14): > oh no come on
Stephanie Hicks (16:46:07): > LOL
2019-02-11
Federico Marini (08:27:35): > You might have seen this already around but yet:
Federico Marini (08:27:36): > https://github.githistory.xyz/csoneson/iSEE/blob/master/R/iSEE-main.R
Federico Marini (08:27:57): > .com -> .githistory.xyz, for any github-based url
Aaron Lun (08:29:16): > What’s the point?
Federico Marini (08:30:20): > a graphical display of the history of any file, FWIW
Federico Marini (08:30:45): > (more useful but less funny than starlogs, if you ask me:slightly_smiling_face:)
2019-02-14
Aaron Lun (14:30:46): > I just put the poster back up. Bit worse for wear after SCG2018, but whatever. Yes, that is John’s face in the bottom right. - File (JPEG): 20190214_192907.jpg
Aaron Lun (14:31:23): > Yes, those QR codesarehand drawn.
Aaron Lun (14:31:27): > 3 hours each one.
Aaron Lun (14:31:44): > Yes, those are my passport photos. Had some leftover and I couldn’t be bothered drawing my own face.
Aaron Lun (14:32:19): > And no, I didn’t hand-draw the t-SNEs.
Federico Marini (14:34:36): > about the QR code, I do recall
Federico Marini (14:34:46): > :stuck_out_tongue:
Federico Marini (14:34:53): > what’S in the bottom right one?
Aaron Lun (14:36:17): > Oh, it has a clear holder that holds various stickers for people to take. I ddint’ have iSEE stickers but@Matt Ritchiesent me some scran & dropletutils & edgeR stickers.
Aaron Lun (14:36:35): > Now it just contains random shit e.g., burger king salt
Aaron Lun (14:36:39): > copies of John’s face
Aaron Lun (14:36:51): > some plastic flower ring stuff
Federico Marini (15:11:56): > definitely a bait for the peeps
2019-02-16
Aaron Lun (08:35:00): > ANYONE AROUND
Aaron Lun (08:53:55): > I have some tasks for people.
Aaron Lun (10:07:46): > Multi select enabled. - File (PNG): Pasted image at 2019-02-16, 3:07 PM
Aaron Lun (12:35:36): > Hello? Anyone there?
2019-02-17
Kevin Rue-Albrecht (10:22:58): > I’m around, but currently lagging through package updates, slowed down by required compilation
Kevin Rue-Albrecht (10:23:45): > I just glimpsed at your commit messages. Interesting and scary, to handle multiple selections
Aaron Lun (11:35:18): > It’s pretty fucked up. For starters, we should get it passing CHECK again. I need to do some more refactoring, the entire transmission machinery has gotten out of hand.
Aaron Lun (11:35:50): > I’m out of steam for today… will think a bit more about the necessary refactoring.
Kevin Rue-Albrecht (11:40:26): > It’s just so hard to handle refactoring jobs in the middle of other things. It’s just a shame that work comes in the way:wink:I’m spread too thin these days. I know the “out of steam” feeling. Time for a bit of Netflix again, I guess
Aaron Lun (11:42:04): > I’ll deal with the refactoring and the other features. Just grind through some of the tests when you have some time. Or the aesthetics, which are pretty easy.
Kevin Rue-Albrecht (11:42:20): > true
Kevin Rue-Albrecht (11:45:08): > Plus, I need to seriously get thinking about that iSEE workshop. I’ve submitted the form for bioc2019, but for now I’m still waiting on the technical guidelines for preparing it
2019-02-18
Federico Marini (04:10:52): > talking briefly about workshops and conferences, is anyone else also coming to useR (Toulouse)?
Charlotte Soneson (04:14:22): > Likely not
Federico Marini (04:14:30): > The abstract submission deadline is in 2 weeks or so, and I think we should not miss the chance
Federico Marini (04:15:01): > something on the line of EuroBioc would be a nice eye-catcher I gues
Federico Marini (04:15:01): > s
Federico Marini (04:15:14): > (plus, we have Martin keynote-ing)
Kevin Rue-Albrecht (04:43:51): > Very unlikely that I convince Steve to let me go to both Bioc2019 in June and useR in July (and my wedding in May). Priorities are made:wink:
Federico Marini (04:44:14): > :slightly_smiling_face:
Federico Marini (04:44:39): > I am struggling to fit the abstract in their limitation of 1200 characters
Federico Marini (04:44:57): > which is quite restrictive
Federico Marini (04:45:56): > (the work in progress is here:https://docs.google.com/document/d/1DIFhs0cR6u94B2fZ6SasupriNAFY7GVQ3TZSkMAW_c8/edit?usp=sharing)
Charlotte Soneson (04:48:34): > Is that 1200 with or without spaces?
Federico Marini (04:48:48): > good question
Federico Marini (04:48:58): > an abstract (in plain text — no HTML, TeX or MarkDown formatting is accepted, 1200 characters at maximum)
Charlotte Soneson (04:49:49): > So maybe with, if it has to fit in some kind of box?
Federico Marini (04:50:34): > seems so
Federico Marini (04:50:44): > I am going to ask in an email
Federico Marini (04:58:14): > probably meaningful to wait for them to reply, the whitespaces are like a VAT of ~15% extra
Federico Marini (05:00:29): > yes, the limitation includes all characters including whitespaces.
Federico Marini (05:00:36): > that was fast:slightly_smiling_face:
Federico Marini (05:02:05): > (I hope now they don’t count title and authors as well, otherwise the abstract will be pretty much “hey iSEE is cool, come check it out”)
Charlotte Soneson (05:02:51): > Short and to the point:slightly_smiling_face:
Federico Marini (05:03:32): > dammit why is the class name so long
Federico Marini (05:03:33): > :smile:
Federico Marini (05:07:25): > plus: the british-affine speakers will horrify, but I am happy to cut out the full stops out of the items in the list
Federico Marini (05:24:40): > Sorry for the first chaotic version. Page 2 of the GoogleDoc is the shortened up version
Federico Marini (05:24:57): > I hope the title does not count in that
Federico Marini (05:25:41): > Any feedback welcome if you get more compact/knacky formulations
Kevin Rue-Albrecht (05:37:08): > @Federico MariniDoes this help?:stuck_out_tongue_winking_eye:https://kevinrue.shinyapps.io/abstract/
Federico Marini (05:41:31): > eheh
Federico Marini (05:41:39): > there’s an app for everything
Federico Marini (05:53:14): > Funny but not funny: the website for submission seems to not count whitespaces
Federico Marini (05:53:41): > And even without them, the count is different from what Word says:stuck_out_tongue:
Federico Marini (05:53:47): > OH WELL
Aaron Lun (06:04:27): > Round 1 of some brutal refactoring done. Play around withmultiselectand report any bugs you find.
Aaron Lun (06:13:25): > To re-iterate: the “Selection parameters” now has a Save button, which will save the active selection into the panel’s history. There is also a Delete button that will remove the last saved selection (first-in-last-out). Receiving panels can now choose to use the active selection, one of the saved selections, or a union of all active and saved selections in the transmitting plot.
Aaron Lun (06:13:54): > This is designed to be especially powerful when combined with custom panels, as you can now do what everyone wants, i.e., DE between two (or more!) selected groups. <- haven’t gotten around to this, though.
Aaron Lun (11:46:29): > Three hours of my life I’m not getting back:https://github.com/csoneson/iSEE/pull/267#issuecomment-464805879
Federico Marini (12:06:17): > oh jeez
Federico Marini (12:06:39): > I can feel that the reaction to that bizarre behavior went into the name of the UI object
Federico Marini (12:08:13): > Since it is only 10 days ahead: do we want to pursue (should I pursue) the attempt for the shiny contest? Keeping in mind that this rstudio cloud is not really the most stable thing even for our subset
Federico Marini (12:08:29): > (and that I never tried the wholeshinyapps.iodeployment)
Aaron Lun (12:08:30): > ¯*(ツ)*/¯
Aaron Lun (12:08:38): > You can do what you like, but don’t expect any help from me.
Federico Marini (12:08:43): > I was counting on at least a shrug:stuck_out_tongue:
Federico Marini (12:10:34): > I had the feeling it would be a nice spotlight, but I cannot follow that up at full throttle as of now
2019-02-19
Kevin Rue-Albrecht (09:25:37): > tada?https://kevinrue.shinyapps.io/project/
Kevin Rue-Albrecht (09:28:36): > I’ve completely bypassed theallendataset which seems to be the cause of constant crashes for “RStudio Cloud”.@Aaron LunWould you mind whipping up an example with some substructure, like you did for that “antimagic” repo? That could make the app a bit more exciting without requiring data from a “real” experiment. > Code is here:https://rstudio.cloud/project/224584
Aaron Lun (09:28:55): > Knee deep in the brush shit right now.
Kevin Rue-Albrecht (09:30:23): > Whenever you can make time.~Also I probably need to push it to GH first anyway to give us a shared “workspace”~
Kevin Rue-Albrecht (09:31:27): > GH is only to clone a project in the workspace. After that it sounds like they want the project to live in their cloudhttps://docs.google.com/document/d/1p-5Ls2kEU9TUoUTQfBNqwEMPoAL0eNHceKRXDZ1koXc/edit - File (Google Docs): RStudio Cloud instructions for the Shiny Contest
Aaron Lun (09:54:08): > I’m not seeing any plots.
Aaron Lun (09:54:33): > In any case, why can’t you take a subset of the data?
Aaron Lun (10:03:23): > Also, crazy brush bug “fixed”. Mostly by running away from the problem.
Kevin Rue-Albrecht (10:04:08): > the RStudio Cloud session keeps crashing when I try to do anything with theallenobject, including setting assays toNULL(in an attempt to reduced memory usage).
Kevin Rue-Albrecht (10:04:38): > it seems like it’s easier to build an SCE “up” than to trimallen“down”
Aaron Lun (10:04:42): > Hm. So even loading it is a problem.
Kevin Rue-Albrecht (10:04:55): > actually, no, that’s the funny thing
Aaron Lun (10:04:56): > Okay, well, just make up some shit. Steal some code from somewhere/
Kevin Rue-Albrecht (10:05:02): > data(allen)works just fine
Kevin Rue-Albrecht (10:05:24): > but if I follow it immediately byassay(allen, ...) <- NULL, it crashes
Kevin Rue-Albrecht (10:08:08): > I guess it’s this whole “cloud” buggy behaviour that Fede faced too. I haven’t figured out hw to avoid it. > Update here:https://community.rstudio.com/t/killed-error-during-installation-for-many-packages/24239/3 - Attachment (RStudio Community): “Killed” error during installation for many packages > This indicates to me that loading the package is exceeding the 1 GB limit to memory that currently exists in rstudio.cloud. However, the package is actually installed, and can be loaded by library(iSEE). What does the package iSEE do when it is loaded? The message indicates that BiocManager is trying to update/install R system library packages, which are not, in fact, updatable by users. It seems to be an informative warning instead of an outright error though.
2019-02-20
Kevin Rue-Albrecht (03:25:47): > @Federico MariniI got a subset of theallenworking for the Shiny contest:https://kevinrue.shinyapps.io/project/Wanna have a look and polish anything? The associated project ishttps://rstudio.cloud/project/224584
Kevin Rue-Albrecht (03:29:15): > I’ve subsetted totophat_countsand the 500 most variable genes. That “could” be relaxed, but not the immediate point of the Shiny contest… plus I’m too lazy to optimize a test case against arbitrary memory limits:wink:
Kevin Rue-Albrecht (03:29:38): > Otherwise all that’s left would be to write the post to enter the contest
Federico Marini (03:34:35): > On it soon:wink:
Aaron Lun (04:54:01): > Numbered multiple selections. - File (PNG): Pasted image at 2019-02-20, 9:53 AM
Kevin Rue-Albrecht (04:54:46): > I move my pawn in A2
Kevin Rue-Albrecht (04:55:56): > but… wow:clap:
Aaron Lun (04:56:08): - File (PNG): Pasted image at 2019-02-20, 9:56 AM
Aaron Lun (04:58:38): > The unnumbered selection is the currently active selection.
Kevin Rue-Albrecht (05:03:31): > I guessed that one:wink:My question is rather: how do you use the other selections, then?
Kevin Rue-Albrecht (05:03:53): > Are they added as extra “colData”?
Aaron Lun (05:05:50): > Well, why don’t you try it out?
Kevin Rue-Albrecht (05:07:36): > Fair enough. I’ll sneak that during my N+1’th debugging notebook for that lincRNA project that just won’t die
Kevin Rue-Albrecht (05:52:27): > We’ll need to prefix the diagnostics with the selection number, e.g.[1] 29 of 379 points in saved selection 3 (7.7%)
Kevin Rue-Albrecht (05:53:53): > But wow that looks cool
Aaron Lun (05:55:55): > Don’t understand. Selection 3 IS the selection number.
Kevin Rue-Albrecht (05:57:08): > :sweat:I’m blind. I read too fast and didn’t spot the differencesaved ... 3
Kevin Rue-Albrecht (05:59:23): > in that case, I guess the point that I accidentally made was that intuitively I would have looked for the selection ID highlighted at the start of the text rather than further down in the text. Other personal preferences?
Aaron Lun (06:00:17): > I don’t think it’s worth changing, unless you want to change all the sentences.
Aaron Lun (06:00:26): > Look, there’s bigger problems, namely the tests.
Kevin Rue-Albrecht (06:01:04): > Absolutely. No worries. I know where to look, and that one can’t drink and use iSEE:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (06:01:36): > Skipped my coffee this morning, explains things
Aaron Lun (06:01:48): > Anyway, do me a favour and starhttps://github.com/LTLA/MoeDCPLogo. I want to push this to the top ofhttps://github.com/topics/human-cell-atlas.
Aaron Lun (06:03:53): > Same goes for you@Federico Marini
Kevin Rue-Albrecht (06:04:37): > Done. And I’m watching you:nerd_face:
Federico Marini (06:08:03): > make place SeuratS, an army of moes will bury you
Federico Marini (06:08:11): > btw
Federico Marini (06:08:15): > we crossed 70 stars
Aaron Lun (06:08:41): > Just need another 300 stars.
Federico Marini (06:08:44): > and we did increase again the dl numbahs
Rob Amezquita (10:29:22): > any chance that iSEE could incorporate non-default color palettes?:slightly_smiling_face:
Rob Amezquita (10:30:00): > although thats a very slippier slope for aesthetics..
Federico Marini (10:31:32): > Check the ecm-dedicated vignette Rob
Federico Marini (10:31:43): > the colormaps are what could do the job for you
Federico Marini (10:32:29): > I like for example > > colpal <- colorRampPalette(RColorBrewer::brewer.pal(n= 9,name = "YlOrRd")[2:9]) > ecm <- ExperimentColorMap( > all_continuous=list( # shared > assays=colpal, > colData=colpal > ), > global_continuous=colorRampPalette(RColorBrewer::brewer.pal(n= 9,name = "YlOrRd")[2:9]) # global > ) > ecm >
Federico Marini (10:33:01): > (in brief, expr values on Tsne are somewhat easier to discern)
Rob Amezquita (10:36:34): > thanks federico!
Aaron Lun (13:55:26): > UNIT TESTS UNIT TESTS UNIT TESTS UNIT TESTS
Aaron Lun (13:55:39): > multiselectNEEDS to get into the next release.
Kevin Rue-Albrecht (13:58:48): > Yup yup yup… ok so we’re end of february… I guess a couple of nights and it should be in shape
Aaron Lun (13:59:03): > pretty much
Aaron Lun (13:59:15): > I want to go to sleep right now. But the work doesn’t end.
Kevin Rue-Albrecht (14:00:22): > do we have the next release schedule (browsing the bioc site right now)
Kevin Rue-Albrecht (14:00:49): > ah no it’s still the previous schedule
Kevin Rue-Albrecht (14:02:17): > Shame Anna’s hen party is end of April… it’ll be borderline late… hackathon for the stag?
Kevin Rue-Albrecht (17:34:22): > test-plotting.R: 1,962 lines of unit tests … > … the entireR/directory is 12,442 lines
Kevin Rue-Albrecht (18:20:47): > wake me up when the Travis builds ends:notes:
2019-02-21
Kevin Rue-Albrecht (02:54:33): > Damn I missed that tracker unit test. I’ll finish it with coffee
Aaron Lun (02:58:32): > Note thatget_selection_dependentsshould no longer be in use, testing should be done onget_direct_children(which is much, much simpler anyway).
Kevin Rue-Albrecht (03:04:46): > Thanks. Hadn’t met that one yet, I did see.any_point_selection->.any_active_selectionlast night.
Aaron Lun (03:05:57): > get_selection_dependents still exists but should be unused (double-check this).
Kevin Rue-Albrecht (03:07:46): > ok.. but before that the last failing test is complaining about.track_selections_onlyhaving disappeared. I’ll fix that one first and then triple check.get_selection_dependentsas an extra
Aaron Lun (03:10:02): > Oh yeah I got rid of the track selections.
Aaron Lun (03:10:07): > Couldn’t be bothered updating it.
Aaron Lun (03:10:16): > And no one really uses it anyway.
Kevin Rue-Albrecht (03:19:59): > mhh.. I’ve updatedtrack_selections_onlyto.track_selection_code, but this couple of lines still confuse mehttps://github.com/csoneson/iSEE/blob/multiselect/tests/testthat/test_track_code.R#L156when i look into theoutobject, there are no mentions of “Column data plot”. > I need to check if that is related to you”direct children” update
Aaron Lun (03:21:32): > don’t think so.
Aaron Lun (03:21:46): > Though it might be.
Kevin Rue-Albrecht (03:22:29): > just for my own record: that’s the state of the selection graph - File (PNG): Pasted image at 2019-02-21, 8:22 AM
Kevin Rue-Albrecht (03:23:03): > so the colData’s are direct children of the redDim
Aaron Lun (03:23:19): > it should work.
Aaron Lun (03:23:33): > okay, gotta read mike’s fellowship applicatio before he gets in.
Kevin Rue-Albrecht (03:24:46): > can’t wait to start writing my first one… someday…
Kevin Rue-Albrecht (03:24:55): > have fun
Kevin Rue-Albrecht (03:41:41): > Whenever, please just check outhttps://github.com/csoneson/iSEE/pull/267/commits/362724050e39a7a5af47d5a94e089286ed81e066to check if I wrote anything stupid. > I’m going to be a pain again, but the internal functions deserve a bit more documentation (in this case?INTERNAL_track_it_all). I don’t mind running the code to see what it does and learn what to expect, but it’ll never be as good as having a reference description of what to expect from the developer’s perspective.
Federico Marini (03:41:48): > iSEE we left a trace:stuck_out_tongue:->https://marionilab.cruk.cam.ac.uk/MouseGastrulation2018/
Federico Marini (03:42:04): > the tour part is a nice cherry on top
Kevin Rue-Albrecht (03:45:17): > the last step of the tour jump to the topleft corner. Who do I email about it?
Aaron Lun (04:09:39): > I can go over and nag him
Aaron Lun (04:10:04): > Anyway, re docs: you’ve picked the few examples where I couldn’t be bothered documenting individual functions.
Aaron Lun (04:10:20): > Most other internals are doc’d quite exhaustively.
Aaron Lun (04:11:05): > Well, while you’re there, can you killget_selection_dependentsand re-use its docs forget_direct_children?
Aaron Lun (04:11:17): > The difference in the names should tell you everything about the difference in the functions.
Kevin Rue-Albrecht (04:12:23): > I know that I’m being a pain - and I would have probably done the same myself. It’s just in those moments that I realize how the (internal) docs are a fixed reference point, in contrast to discussion on Slack which are helpful to clarify things on a shorter term
Kevin Rue-Albrecht (04:13:47): > Let me just clear other stuff this morning while Travis completesR CMD check(doing it right now after 30 min of package updates)
Kevin Rue-Albrecht (04:14:09): > I’ve made a note ofget_selection_dependents. Worst case I get to it tonight.
Kevin Rue-Albrecht (04:14:39): > If we can get at least Travis happy again, it’ll cache the updated packages, and future builds will take less than 15 min again
Aaron Lun (05:05:09): > Is brushing working for people? Is it persistent? I had to hack it, check out the comments in the brush observer iniSEE-main.R.
Aaron Lun (06:04:05): > Anyway,get_selection_dependentsis done. You can knock off a few of the coverage losses if you like.
Kevin Rue-Albrecht (07:27:58): > Ok, thanks for taking care of it. > Now that Travis completed again, I’ll check out codecov (https://codecov.io/gh/csoneson/iSEE/pull/267) to restore the missing coverage, hopefully this evening. - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Kevin Rue-Albrecht (07:30:21): > On a separate note, 2 birds 1 stone: using iSEE for some local stuff, I realized again that our discrete axes do not show ticks forNAvalues. However, thecoloraesthetic does showNA(as grey). ggplot can be told to showNAon discrete axes, should we? I vaguely remember that we had a discussion about that a while back, I just can’t remember what our reasons were for either choice
Kevin Rue-Albrecht (07:32:36): > shapeshows theNAlabel in the legend, but without a symbol (basically the data point are not shown in the plot)
Aaron Lun (07:35:37): > ¯*(ツ)*/¯
Aaron Lun (07:35:52): > I guess one could.
Kevin Rue-Albrecht (07:36:16): > Thinking of it, it might have been something to do with transmitting selections that contain NAs
Kevin Rue-Albrecht (07:38:05): > Anyway. Better to investigate this as part of an independent PR. Let’s finalize this one first so it can be merged before I start messing with unrelated bits of code
Aaron Lun (08:17:20): > Differential expression with custom selections. - File (PNG): Pasted image at 2019-02-21, 1:17 PM
Aaron Lun (08:17:35): > active vs saved
Federico Marini (08:18:12): > whoooohaaa:party_parrot:
Federico Marini (08:18:26): > gogo custom panels
Aaron Lun (08:18:59): > :w
Aaron Lun (08:19:02): > whoops
Aaron Lun (08:19:08): > Still working on the custom plots.
Kevin Rue-Albrecht (08:19:26): > I was waiting for that kind of custom panel the moment i saw the first multiselect screenshot:heart_eyes:
Aaron Lun (08:40:33): > give me minimum code to make a ggplot heatmap from an input matrix of log-fold changes…. starting NOW.
Kevin Rue-Albrecht (08:44:05): > wait wait wait, that matrix is rows=genes, but columns=(?) > contrast?
Kevin Rue-Albrecht (08:46:34): > > require(reshape2) > nr <- 100 > nc <- 4 > rn <- paste0("gene", seq_len(nr)) > cn <- paste0("constrast", seq_len(nc)) > matFC <- matrix(rnorm(nr*nc), nr, nc, dimnames = list(gene=rn, contrast=cn)) > dfFC <- melt(matFC) > ggplot(dfFC, aes(contrast, gene)) + > geom_raster(aes(fill = value)) >
Aaron Lun (08:52:40): > yes.
Aaron Lun (09:13:06): > core dumped.
Aaron Lun (09:42:28): > fuck ggplot and its stupid promise-based evaluation
Aaron Lun (09:42:38): > why can’t I just plot the x- and y-axes?
Aaron Lun (09:42:40): > DAMMIT
Aaron Lun (09:42:51): > How do you do it withoutcontrast?
Aaron Lun (09:43:24): > this shit is so stupid it hurts.
Kevin Rue-Albrecht (09:45:12): > did it get confused with thecontrastfunction? Does it work if you renamed the field to something less ambiguous (i.e., not a function name)?
Aaron Lun (09:46:17): > And on the sixth day - File (PNG): Pasted image at 2019-02-21, 2:46 PM
Kevin Rue-Albrecht (09:46:57): > :angel:
Kevin Rue-Albrecht (09:47:42): > do me a favor and add+ cowplot::theme_cowplot()I just can’t stand that grey background
Kevin Rue-Albrecht (09:49:16): > but damn that custom panel is awesome
Aaron Lun (10:02:47): > New check fails probably imminent.
Kevin Rue-Albrecht (12:11:59): > I only appreciate@Aaron Lun’s “devel” update of managing the UI layout when I have to work with the release version of iSEE. That dropdown/button way just keeps annoying me
Aaron Lun (12:56:15): > <!channel>multiselectshould now be more-or-less functional. Seevignettes/custom.Rmdfor an example of its usage.
Aaron Lun (12:57:45): > If it’s mostly bug-free and once it passesCHECK, ready to merge.
Kevin Rue-Albrecht (12:57:52): > I’ll bite the bullet and deal with the unit tests and my own round of interactive testing this evening
2019-02-22
Kevin Rue-Albrecht (03:15:50): > https://twitter.com/thomasp85/status/1098536536675700736?s=12Neat - Attachment (twitter): Attachment > Yas!!! The geom_mark family in ggforce gets automatically placed labels (+description) for supercharged annotations in #ggplot2 #rstats https://pbs.twimg.com/media/Dz7JEByX0AAeQT5.jpg
Federico Marini (03:44:32): > some potential for annotating clusters with title and caption?
Kevin Rue-Albrecht (04:10:26): > yup. I’ll definitely be all over that inhancock
Kevin Rue-Albrecht (04:11:25): > oh… and yes, we are at 100% coverage:party_parrot:
Aaron Lun (04:33:45): > Merge when ready.
Kevin Rue-Albrecht (04:40:14): > Yup, I’ll look at that last “Disable Delete button” later this morning. Seems to be the last item
Aaron Lun (04:41:05): > Well, it’s not a big deal, but if you do it, then get rid of theshowNotificationsbecause they shouldn’t be necessary.
Aaron Lun (04:41:57): > Be aware thatdisablingthe button is a bit of a pain, because it can come via inputs other than “Delete” (e.g., in.regenerate_unselected_plot).
Federico Marini (09:49:49): > -> is this one already circulated?http://www.teichlab.org/singlecell-treg
Federico Marini (09:49:59): > plenty of instances of iSEE:love_letter:
Kevin Rue-Albrecht (10:15:36): > Yup. It’s funny that it came back to me from Anna’s colleagues, considering that there are a few authors from the Kennedy xD
Kevin Rue-Albrecht (10:18:03): > Btw, makes me think that we could have made the app open the “About iSEE” on startup. If we don’t get the citations through publications…
Aaron Lun (10:18:42): > I think the name at the top of the bar is already sufficient.
Federico Marini (10:19:12): > watermarks in the plots, that’s what we want
Federico Marini (10:19:23): > or we go MSigDB style
Federico Marini (10:19:38): > plots and code you get them only after providing your email address
Federico Marini (10:19:39): > :smile:
Kevin Rue-Albrecht (10:22:03): > Download stats, that’s the metric for us, I guess, more than citations.
Kevin Rue-Albrecht (10:23:26): > The “hey I want some of that too” feeling that will come from showcasing data sets like those Tregs
Federico Marini (11:38:31): > quick update on the abstract for useR: thanks for the Rückmeldungen:slightly_smiling_face:
Federico Marini (11:38:48): > it is in word about 1300 chars
Federico Marini (11:39:26): > yet their submission system says 1102
Federico Marini (11:40:13): > even less than Word’s count without whitespaces
Federico Marini (11:52:51): > submission’s finalized, in case you get an email, don’t be surprised:wink:
Kevin Rue-Albrecht (12:06:22): > just one more local round of CMD check, and then I’ll push the* Disable Deletefeature
Kevin Rue-Albrecht (12:12:11): > 0 errors:heavy_check_mark:| 0 warnings:heavy_check_mark:| 2 notes:heavy_multiplication_x:
Kevin Rue-Albrecht (12:20:09): > I tested it interactively, but please someone else tries to break it before we merge. Namely: > - the “clear history” button initialises with “No history” (actually, I need to check what happens when the app is initialized with a history) > - emptying the history disables the button > - saving a selection enables the button
Federico Marini (12:28:31): > I’ll test it a little after dinner
Kevin Rue-Albrecht (12:52:23): > mhh..@Aaron Lunwhat am i missing to launch the app with a panel having a preconfigured selection history: > > rd <- redDimPlotDefaults(sce, 5) > rd[["MultiSelectHistory"]][[1]] <- list(list(xmin = -4.6466447142515, xmax = 0.65560197124888, ymin = 12.923854755961, > ymax = 16.203276392259, coords_css = list(xmin = 155.80624961853, xmax = 221.80624961853, > ymin = 27.4312438964844, ymax = 78.4312438964844), coords_img = list(xmin = 311.617964038229, > xmax = 443.620278944582, ymin = 54.8638573802288, ymax = 156.866403708442), > img_css_ratio = list(x = 2.00003507433869, y = 2.00004992800418), mapping = list( > x = "X", y = "Y"), domain = list(left = -14.0531423894257, right = 10.9153021971093, > bottom = -12.0002389472417, top = 16.4373197678157), range = list(left = 77.4380886130137, > right = 699.041095890411, bottom = 932.100973886986, top = 47.5842139230505), > log = list(x = NULL, y = NULL), direction = "xy", brushId = "redDimPlot1_Brush", > outputId = "redDimPlot1"), list(xmin = 2.6640287460596, xmax = 5.7971745147644, > ymin = 7.2395233310055, ymax = 10.969061662482, coords_css = list(xmin = 246.80624961853, > xmax = 285.80624961853, ymin = 108.831253051758, ymax = 166.831253051758), > coords_img = list(xmin = 493.621155803049, xmax = 571.622523702258, ymin = 217.667939830773, > ymax = 333.670835655016), img_css_ratio = list(x = 2.00003507433869, y = 2.00004992800418), > mapping = list(x = "X", y = "Y"), domain = list(left = -14.0531423894257, right = 10.9153021971093, > bottom = -12.0002389472417, top = 16.4373197678157), range = list(left = 77.4380886130137, > right = 699.041095890411, bottom = 932.100973886986, top = 47.5842139230505), > log = list(x = NULL, y = NULL), direction = "xy", brushId = "redDimPlot1_Brush", > outputId = "redDimPlot1")) > > app <- iSEE(sce, redDimArgs = rd) >
Kevin Rue-Albrecht (12:53:19): > I can’t believe I’m the one who wrote the unit tests, and I can’t figure that out
Kevin Rue-Albrecht (12:59:29): > Uh. We need to investigate that I guess. I also tried pasting the whole block of code form the tracker, but none of the preconfigure selections are shown.
Aaron Lun (13:06:53): > Probably isn’t transferred.
Aaron Lun (13:21:09): > Well, for starters, yourignoreNULL=FALSEis causing the observer to fire multiple times during initialization.
Aaron Lun (13:21:46): > Don’t setignoreNULL=FALSE, this is usually a bad idea.
Aaron Lun (13:35:14): > Multifiring of the delete observer wipes out the memorized brushes.
Kevin Rue-Albrecht (13:58:38): > Ugh. Sorry. I’ll think more carefully of another way to check upon initialization whether the button should be disabled. i..e I broke one thing while trying to “fix” another
Aaron Lun (14:05:28): > use start it enabled or disabled by modifying the UI elements.
Kevin Rue-Albrecht (14:06:46): > Not sure what you mean by this last one.
Aaron Lun (14:07:34): > shinyjs allows you to start things enabled or disabled.
Aaron Lun (14:07:45): > I used to use it for the up/down buttons on the panel organization bar.
Kevin Rue-Albrecht (14:07:50): > That said, I can initialize with a history of brushes now, withoutignoreNULLas you said. I’ll fix the starting state of the button after dinner
Kevin Rue-Albrecht (14:08:09): > Cool, thanks for the tip. We’re almost there
Kevin Rue-Albrecht (17:01:42): > argh.. racking my brain about starting that button in a state that depends on the presence of a selection history,
Kevin Rue-Albrecht (17:02:04): > actually. hang on
Kevin Rue-Albrecht (17:24:08): > There. I’m always amazed how the shortest solutions take so much time to identify
Kevin Rue-Albrecht (17:26:39): > I don’t think the “save” button needs any similar treatment? I mean onecoulddisable it when there is no active selection (either brush or lasso). But for now there is the friendly notification to tell users.
Rob Amezquita (18:18:11): > hi#iseeteam - i am in the process of doing editing, and wanted to circle back on the paragraph you guys helped write. i have tailored the paragraph to more heavily feature iSEE, and would love your feedback when you get a chance on the below:
Rob Amezquita (18:18:38): - File (LaTeX/sTeX): iSEE paragraph v2
Rob Amezquita (18:19:23): > please feel free to add/subtract as you see fit, and thank you in advance!
Rob Amezquita (18:20:07): > i will also be making a cartoon diagram of the iSEE interface and will share that with y’all when i get to it - but if any of you have a cartoon schematic of the iSEE interface please share it with me so it can go into the review:slightly_smiling_face:thanks!
2019-02-23
Charlotte Soneson (01:40:37): > Hi@Rob Amezquita, thanks for sending this! I moved the text temporarily to a Google Doc and added a couple of small suggestions:https://docs.google.com/document/d/1IIs2BcITiihscZg_R-5nD4vl42HfuZUqFhvrw7oSRcI/edit?usp=sharing. For the diagram - from what I can remember, so far we’ve mostly been using direct screen shots of iSEE. It may be worth noting here that the interface looks a little bit different in the devel version.
Kevin Rue-Albrecht (02:40:57): > which reminds me that before we merge tomasterwe should do a round in the commits and PRs to identify stuff new features worth mentioning in theNEWSfile. That sort of review is where theNEWSfile can help quickly identify features and updates
Kevin Rue-Albrecht (03:28:36): > I’ve edited the paragraph on GDoc. Feel free to keep/discard anything.]
Kevin Rue-Albrecht (03:29:37): > For the PR, now i also enable/disable the “Save” selection button. I updated the NEWS (we skipped version 1.3.5 for some reason) and I think we’ll be ready to merge when people have a last go
Aaron Lun (07:02:34): > PR plays fine to me. Ready to merge once@Federico Mariniand@Charlotte Sonesongive the green light. But make sure you play around with the custom vignette, this was the whole motivation for the multiple selections in the first place, so it had better work.
Aaron Lun (07:05:42): > I can’t read the doc with all the strikethroughts. Someone just accept all the changes.
Kevin Rue-Albrecht (07:36:32): > In the part of the vignette where you wrote “IfcustomSendAll=FALSE(the default)“, would it be clearer to write “IfiSEE(customSendAll=FALSE)(the default)” ?
Kevin Rue-Albrecht (07:38:39): > I don’t really have a clear suggestion for a better phrasing, but for a moment there I read it as if the custom functions were the ones to deal with thecustomSendAllargument
Kevin Rue-Albrecht (07:45:36): > mhh… actually, instead of 1 group of 3 bullet points, how about: > - ASummarizedExperiment… > IfiSEE(..., customSendAll=FALSE)(the default): > - a vector of row names as second argument > - a vector of column names as third argument > Does this read better? - File (PNG): Pasted image at 2019-02-23, 12:45 PM
Aaron Lun (08:21:56): > I don’t understand the problem.
Kevin Rue-Albrecht (08:33:02): > Hang on I’ve almost finished writing edits. I’ll push and you can review
Aaron Lun (08:38:44): > No, as in, I think it’s fine.
Aaron Lun (08:44:13): > LOL:https://bioconductor.org/packages/release/bioc/vignettes/BioNetStat/inst/doc/BNS_tutorial_by_command_line.html
Kevin Rue-Albrecht (08:58:33): > Oh yeah, that clarifies it
Kevin Rue-Albrecht (08:58:48): > After BiocStyle… BiocLocale ?
Kevin Rue-Albrecht (09:12:02): > i pushed the proposed edits, let me know what you think
Kevin Rue-Albrecht (09:12:32): > a minor update is that i consistently put everything to present tense, it reads better, I think
Aaron Lun (09:13:34): > Commented.
Kevin Rue-Albrecht (09:19:11): > ok i’ll deal with it ASAP, I’m just balancing this with stuff to do around the house.
Aaron Lun (09:19:22): > chores.
Aaron Lun (09:19:23): > pfft.
Aaron Lun (09:19:27): > just hire a housekeeper.
Aaron Lun (09:19:44): > ah crap, I’m thinking like I’m rich already.
Kevin Rue-Albrecht (09:20:30): > Just to be clear about the “talk about iSEE(…, customSendAll=TRUE) later”. I understand this as just dropping the “In this case, thesecondandthirdarguments of the user-defined functions must each accept a named list of 2 items,"active"and"saved", respectively.” part. I can keep the “Conversely,…” and “This feature is described in further detail and demonstrated in a later section.” parts, right?
Kevin Rue-Albrecht (09:23:04): > I’m barely allowed to touch anything in Anna’s~house~territory..:wink:I wouldn’t give much of a chance to a housekeeper
Aaron Lun (09:24:08): > Yes.
Kevin Rue-Albrecht (10:04:37): > No one freaks out please - I’m going to create a R-3.5 branch again, with the requirement dropped in DESCRIPTION. It’s just so that I can install the latest version of the package on the RStudio Cloud whose latest version is 3.5
Kevin Rue-Albrecht (10:05:46): > so long as we enter the contest, we might as well blow a few minds with the multiselect functionality
Aaron Lun (10:05:50): > Y’know, we don’t even need an R version requirement.
Kevin Rue-Albrecht (10:05:57): > oh
Aaron Lun (10:05:59): > We don’t have any Rds files.
Kevin Rue-Albrecht (10:06:39): > well then, I guess we could drop it altogether, and at least one of the package that we depend on will set that version requirement anyway
Aaron Lun (10:06:57): > pretty much.
Kevin Rue-Albrecht (10:08:57): > so.. what’s the practice in this case, dropping the whole thing? or just the version (i.e., keeping “Depends: R, SummarizedExperiment”). > I guess keeping the “R” is useless right?
Aaron Lun (10:09:04): > Pretty much.
Kevin Rue-Albrecht (10:45:44): > mhh. RStudio cloud is happy to install the package from themultiselectbranch, butrsconnect::deployApp()doesn’t seem to like it when it assembles the bundle to push toshinyapps.ioHappy to merge the PR anytime, so that I can just do a regular install from themasterbranch
Aaron Lun (10:50:37): > Do it.
Kevin Rue-Albrecht (10:51:34): > Ok. It’s not like opening new branches costs much, if we need to fix anything later
Kevin Rue-Albrecht (10:51:57): > Oh wait. One thing though: do I simultaneously pushupstream?
Aaron Lun (10:52:20): > Wait for comments from@Federico Mariniand@Charlotte Soneson.
Kevin Rue-Albrecht (10:52:50): > I’d say “no” to wait for them indeed > But also, I’d say “yes” so that what’s on GHmasteris the same as what’s on Bioc-devel > Let’s say “no” then
Aaron Lun (10:53:07): > Well, if they get to it tonight, it should be fine.
Kevin Rue-Albrecht (10:53:19): > Done.
Kevin Rue-Albrecht (10:53:27): > (master, not Bioc)
Aaron Lun (11:02:42): > :+1:
Aaron Lun (11:02:56): > If they give you the green light, go ahead and push upstream.
Kevin Rue-Albrecht (11:05:53): > damn. merging to master didn’t help RStudio cloud. Looks like an issue betweendevtoolsand system libraries: > > building: Building package: iSEE > ################################ Begin Task Log ################################ > [2019-02-23T16:01:53.192831920+0000] Execute script: packages/build/sys.sh > + set -e > + apt-get update -qq > + apt-get install -y libapparmor-dev > Reading package lists... > Building dependency tree... > Reading state information... > The following NEW packages will be installed: > libapparmor-dev > 0 upgraded, 1 newly installed, 0 to remove and 29 not upgraded. > [...] > Preparing to unpack .../libumfpack5.7.1_1%3a4.4.6-1_amd64.deb ... > Unpacking libumfpack5.7.1:amd64 (1:4.4.6-1) ... > Selecting previously unselected package libspqr2.0.2:amd64. > Preparing to un################################# End Task Log ################################# > Error: Unhandled Exception: Child Task 589830917 failed: Error building image: Error fetching iSEE (1.3.6) source. Error accessing GitHub repository csoneson/iSEE. Unauthorized >
Kevin Rue-Albrecht (11:06:02): > (https://rstudio.cloud/project/224584)
Kevin Rue-Albrecht (11:06:18): > ¯*(ツ)*/¯
Aaron Lun (11:07:41): > Can you pull stuff from Bioc-devel?
Kevin Rue-Albrecht (11:12:32): > mhh… maybe, but not until I push the dropped R requirement upstream
Aaron Lun (11:13:08): > As in, any random stuff, just to test it.
Kevin Rue-Albrecht (11:13:19): > > > BiocManager::install("iSEE", version="devel") > Error: Bioconductor version '3.9' requires R version '3.6'; see[https://bioconductor.org/install](https://bioconductor.org/install) >
Aaron Lun (11:13:33): > Well, so much for that.
Aaron Lun (11:14:08): > Probably want to complain to Rstudio about their fucked up cloud, then.
Aaron Lun (11:14:15): > Does it work for other GH repos?
Kevin Rue-Albrecht (11:14:18): > Well, we still have a bit of time to enter the Shiny contest. Today I just want to start writing the post that we’ll submit to enter the contest
Kevin Rue-Albrecht (11:14:37): > Good point, let me try another repo
Kevin Rue-Albrecht (11:15:27): > although, if you have time to kill and want the full log of the failed installation (I posted only part of it above), go tohttps://rstudio.cloud/project/224584and runhttps://rstudio.cloud/project/224584
Aaron Lun (11:17:18): > I don’t have a login.
Aaron Lun (11:17:24): > and I CBFd.
Aaron Lun (11:17:37): > Get@Federico Marinito look at it, he’s the other contest fan.
Aaron Lun (11:17:49): > Where are those guys anyway?
Aaron Lun (11:18:16): > All these people hanging out with family instead of coding.
Kevin Rue-Albrecht (11:18:22): > … having a life?:wink:
Aaron Lun (11:18:23): > Makes me:face_vomiting:
Kevin Rue-Albrecht (11:19:13): > I get you… I day without a commit in not worth living
Aaron Lun (11:19:36): > damn straight
Aaron Lun (11:19:52): > my last non-commit day was april last year
Aaron Lun (11:20:21): > and that was because I was in a hotel room without wifi, and I was physically too ill to get out of bed to commit anything.
Kevin Rue-Albrecht (11:20:43): > well,devtools::install_github("kevinrue/sqlighter")works (on RStudio cloud)
Kevin Rue-Albrecht (11:21:33): - File (Shell): Untitled
Aaron Lun (11:27:48): > I should point out that BioC’s iSEE is 1.3.7 and we are 1.3.6.
Kevin Rue-Albrecht (11:28:42): > ?!
Kevin Rue-Albrecht (11:29:13): > what the hell
Aaron Lun (11:29:39): > Well, just kick it to 1.3.8, along with any other cosmetic changes.
Kevin Rue-Albrecht (11:29:55): > i didn’t even check that because the NEWS file was missing 1.3.5, and the last PR was a bump up to 1.3.6
Aaron Lun (11:30:44): > maybe it was one from the Bioc folks about immunooncology or something.
Aaron Lun (11:31:04): > yeah, that’s probably it. That wouldn’t have gone via GH.
Kevin Rue-Albrecht (11:31:12): > ow
Aaron Lun (11:31:29): > they sent out emails but those would have gone tot he fofficial maintainer.
Aaron Lun (11:31:36): > So I got a whole bundle for other packages.
Kevin Rue-Albrecht (11:31:48): > oh wait so that means we would have had tofetch upstream
Kevin Rue-Albrecht (11:32:12): > otherwise we’ll never see the changes. i’ll check that
Kevin Rue-Albrecht (11:32:24): > ugh
Aaron Lun (11:32:48): > Well, these types of changes are pretty rare, so I’m down with that.
Aaron Lun (11:33:01): > Usually when data structures get updated.
Kevin Rue-Albrecht (11:34:04): > oh yeah i get it, no problem, it’s just that it would be nice ifgithad a little bell to indicate “new commits available on some remotes”
Kevin Rue-Albrecht (11:34:13): > yup, i got the update
Kevin Rue-Albrecht (11:35:20): > ok, so you should see now a branchbiocupdatethat I created
Kevin Rue-Albrecht (11:35:30): > (can’t push straight tomaster)
Aaron Lun (11:36:30): > I’m going for a walk. But I will be back to do some edits.
Kevin Rue-Albrecht (11:37:04): > I’ll track down the updates for the last few version bumps, to mention everything in the right places
Aaron Lun (12:20:53): > The date is fucked up.
Kevin Rue-Albrecht (12:22:15): > ugh
Kevin Rue-Albrecht (12:24:50): > Aside from the back to the future date, i recently noticed that this affects thecitation(). Isn’t there an argument for setting this date to the first commit, in that case?
Aaron Lun (12:25:33): > What. How? The citation should be the f1000 paper.
Kevin Rue-Albrecht (12:27:32): > oh right, in this case, we’ve overridden that
Kevin Rue-Albrecht (12:28:12): > i was talking about the citation that’s built from the DESCRIPTION file if you don’t provide your own citation
Kevin Rue-Albrecht (12:33:06): > see branchdate
Aaron Lun (12:35:50): > Yep.
Aaron Lun (13:34:48): > I’m done with the paragraph@Rob Amezquita
Aaron Lun (13:35:07): > I pretty much mauled everyone else’s changes.
Aaron Lun (13:37:33): > off to dinner.
Aaron Lun (13:37:42): > @Kevin Rue-AlbrechtI don’t know what you want me to do with the template.
Kevin Rue-Albrecht (13:38:17): > At the moment, it’s probably still too raw for you to bother reading, but ultimately, the same thing that you did for Rob’s paragraph
Kevin Rue-Albrecht (13:39:10): > Essentially it’ll be a mashup of the F1000 article, GitHub README, and package doc
Kevin Rue-Albrecht (13:39:34): > I don’t want it to be too long, but right now I’m just collecting material, I’ll trim down later
Kevin Rue-Albrecht (13:58:46): > @Aaron Lunit’s taking shape
Kevin Rue-Albrecht (14:00:14): > at this point, all i’d ask is a review of the brief description/introduction, and filling in the 3 sets of bullet points to list key features of iSEE, in case I missed any important one
Kevin Rue-Albrecht (18:37:38): > Next PR is likely to cross the 2,000 commit mark, if anyone’s up for a challenge. > There is always the possibility of extending the set of vocal commands supported:slightly_smiling_face:
Kevin Rue-Albrecht (18:50:25): > Alright, I’m done with it for now. Here is the link again:https://docs.google.com/document/d/1ZVyxnf_4IW-FjTmlGVUqcYte2fLX0o3X59DziIVv1-Y/edit#heading=h.n072rajuky83It starts on page 2 (page 1 are just a copy of submission guidelines) - File (Google Docs): iSEE shiny contest
2019-02-24
Federico Marini (05:27:57) (in thread): > I could say “been on a train, will be on a train”:stuck_out_tongue:
Federico Marini (05:28:19) (in thread): > touch & go at my parents, to drop the kids (with care)
Federico Marini (05:36:38): > I’ll have a look on the train:wink:
Federico Marini (05:51:37): > re: paragraph for the manuscript, I am fine with the whole content
Federico Marini (05:51:53): > just added a comment to eventually add/use/cite tSNE and UMAP
Federico Marini (05:52:36): > on the like “everyone knows the name (not the algorithm/principle, but hey)”
Charlotte Soneson (09:01:57): > Ok, I played around with the currentmasterbranch with the multi-selection - looks really nice and seems to do what it should! A couple of things: > • I am trying to make this reproducible, but so far unsuccessfully - it happens only occasionally: sometimes when I make a point selection, and then make another one (without saving the first one), the first one stays around, but without the strong border (see leftmost rectangle in screen shot below), and it does not disappear. > • I am no longer able to drag the selections around (maybe this was discussed at some point, can’t remember now) > • minor detail: in theCUSTOM_HEATfunction in the vignette, the[1:50]subsetting means that if one selects less than 50 genes, there will be a row of NAs in the heatmap… - File (PNG): Screen Shot 2019-02-24 at 14.40.01.png
Aaron Lun (11:15:06): > First point: yes, I don’t know why it happens either.
Aaron Lun (11:16:44): > Second point: This is because the Javascript brush is destroyed when the plot is recreated, so you never get a persistent javascript-side brush. This technically didn’t need to happen, but I made it so it’s one less thing to worry about. Now the brush that’s in iSEE’s memory is the only version of the brush that we ever need to care about.
Aaron Lun (11:16:57): > Third point: Make a PR.heador something.
Aaron Lun (11:55:24): > getting my ass kicked by rewriting a local alignment algorithm
Aaron Lun (11:55:29): > it hurts so much
Aaron Lun (13:19:01): > Ugh
Aaron Lun (13:19:03): > out of juice
Charlotte Soneson (15:14:59): > Anyone has an intuition of where this error (subscript out of boundsif more than one panel of a given type ininitialPanels) might be triggered (or which of the recent changes could have induced it)? > > init <- DataFrame(Name = sprintf("Reduced dimension plot %i", 1), > Width = 4) > iSEE(sce, initialPanels = init) ## -> everything ok > > init <- DataFrame(Name = sprintf("Reduced dimension plot %i", 1:2), > Width = 4) > iSEE(sce, initialPanels = init) > > ## -> Warning: Error in [[: subscript out of bounds > ## [No stack trace available] >
Aaron Lun (17:32:25): > Aw gawd
Aaron Lun (17:32:32): > I spent three hours tracking down one of those.
Aaron Lun (17:33:53): > Start by adding print statements to the UI renderers.
Aaron Lun (17:34:13): > Then flood the observers with more print statements.
Kevin Rue-Albrecht (18:35:49): > I’ve just integrated the changes for the shiny contest. The deadline for the submission is 8th March, so we still got ourselves a bit of time, but I’d like to submit by next weekend. Feel free to chime in more edits in the meantime
2019-02-25
Charlotte Soneson (04:07:19): > Ok, the direct source of the error seems to be the use of[[id]]here (theparam_choicesis already subset to an ID):https://github.com/csoneson/iSEE/blob/4598b8c45e6684e23a3731188a0c1155a0d1bd85/R/dynamicUI.R#L1149-L1157.@Kevin Rue-Albrecht- suggestions for the best way to fix it? Changingid -> 1fixes theinitialPanelsissue, but I haven’t tried yet what will happen if there are saved selections…
Aaron Lun (04:12:18): > 1L should be fine.
Aaron Lun (04:12:26): > change the 0 to a 0L as well.
Aaron Lun (04:12:31): > oh wait, that’s already the case.
Aaron Lun (04:12:42): > Probably just use==instead ofidentical.
Aaron Lun (04:12:57): > And put a newline after 1152.
Kevin Rue-Albrecht (04:14:17): > identicalis fine, indeed, it should be1Linstead ofid
Charlotte Soneson (04:17:52): > ok, PR opened
Kevin Rue-Albrecht (04:24:33): > Thanks - have you tested saved selections or would you like me to do it? My reply was theoretical, even though I’m confident
Charlotte Soneson (04:30:01): > Yeah, it seems to work fine.
Charlotte Soneson (04:30:17): > The buttons appear when they should
Kevin Rue-Albrecht (04:39:40): > Great. Thanks! I might give it a try as soon as I get some stuff done this morning
Kevin Rue-Albrecht (05:06:35): > Confirmed. It looks all fine to me with preconfigured saved and active selections
Charlotte Soneson (05:07:25): > :+1:
Kevin Rue-Albrecht (05:14:25): > @Aaron Lunwhere I think your help would make a difference for the shiny contest, is the ” Briefly explain key technical details (how the customization or extension was achieved).” section of the post (last section of the GDocs)https://docs.google.com/document/d/1ZVyxnf_4IW-FjTmlGVUqcYte2fLX0o3X59DziIVv1-Y/editYou’re better at cutting the crap and getting to the point, from what I wrote down so far. - File (Google Docs): iSEE shiny contest
Aaron Lun (05:54:06): > https://kevinrue.shinyapps.io/isee-shiny-contest/doesn’t show any plots.
Kevin Rue-Albrecht (05:56:59): > Did you give it time? it does for me.
Aaron Lun (05:57:13): > I mean, how much time? A minute?
Aaron Lun (05:57:24): > it’s just been sitting there.
Kevin Rue-Albrecht (05:57:36): > 5-10s tops for me
Aaron Lun (05:57:50): > Nope, still just sitting there.
Kevin Rue-Albrecht (05:57:52): > I literally just clicked on your link
Kevin Rue-Albrecht (05:57:59): > … refresh the page?
Aaron Lun (05:58:23): > nope.
Kevin Rue-Albrecht (05:59:22): > from right now - File (PNG): Pasted image at 2019-02-25, 10:59 AM
Charlotte Soneson (05:59:25): > I also don’t see any plots (just panels)
Aaron Lun (05:59:39): > firefox 65.0.
Charlotte Soneson (05:59:51): > chrome
Aaron Lun (06:00:07): > same problem for me on chrome.
Kevin Rue-Albrecht (06:00:35): > Ok, I have the issue with Firefox indeed
Kevin Rue-Albrecht (06:00:55): > Obviously the screenshot above says that Safari works
Kevin Rue-Albrecht (06:02:21): > same problem with Chrome here too. damn
Kevin Rue-Albrecht (06:02:56): > so basically only Safari works for our app onshinyapps.io? ugh. where the hell could that bug be coming from
Aaron Lun (06:03:44): > try without the tour.
Kevin Rue-Albrecht (06:03:52): > @Federico Mariniyou mentioned something about clearing the cache? did that work?
Kevin Rue-Albrecht (06:03:55): > oh yeah, maybe
Federico Marini (06:04:56): > I had it working on everything but safari
Aaron Lun (06:05:19): > anyway, I’m done with your text.
Kevin Rue-Albrecht (06:05:46): > lol - just the opposite of what we’re seeing today then@Federico Marini?
Federico Marini (06:06:11): > testing again now on both machines
Kevin Rue-Albrecht (06:06:20): > anyway, i’m redeploying without the tour right now, will take a a few minutes > thanks@Aaron Lun
Federico Marini (06:09:37): > confirmed here: no chrome, no firefox (neither tour nor plots) - safari like a charm
Federico Marini (06:09:57): > (and no I am not testing on Edge:stuck_out_tongue:)
Kevin Rue-Albrecht (06:10:08): > Opera?
Federico Marini (06:10:16): > me do not have that
Kevin Rue-Albrecht (06:10:41): > lol, i just remember it from a tablet that someone has in my family
Kevin Rue-Albrecht (06:11:56): > damn i’m stupid, I opened the wrong RStudio cloud project today.. redeploying the contest one now
Kevin Rue-Albrecht (06:14:23): > > > rsconnect::deployApp() > Preparing to deploy application... > Update application currently deployed at[https://kevinrue.shinyapps.io/isee-shiny-contest/](https://kevinrue.shinyapps.io/isee-shiny-contest/)? [Y/n] y > DONE > Uploading bundle for application: 735398...DONE > Deploying bundle: 1897678 for application: 735398 ... > Waiting for task: 590235571 > building: Parsing manifest > building: Building image: 1980234 > building: Installing system dependencies > building: Fetching packages > building: Installing packages > building: Installing files > building: Pushing image: 1980234 > deploying: Starting instances > rollforward: Activating new instances > terminating: Stopping old instances > Application successfully deployed to[https://kevinrue.shinyapps.io/isee-shiny-contest/](https://kevinrue.shinyapps.io/isee-shiny-contest/) >
Kevin Rue-Albrecht (06:16:01): > Still having the same issue on both Firefox and Chrome, even without the tour opening
Aaron Lun (06:20:10): > If I inspect the element in Firefox, I getTypeError: input.redDimPlot1_VisualChoices is undefined[Learn More]
Kevin Rue-Albrecht (06:24:55): > I’ve just tried another thing: launching the app from the RStudio Cloud session in Firefox, there it works
Aaron Lun (06:25:41): > ¯*(ツ)*/¯
Aaron Lun (06:41:52): > Fixed a very minor js error injsfix.
Aaron Lun (06:46:14): > Reread the code ininst/www/collapseBox.js. Understood NONE of it.
Aaron Lun (06:46:19): > How did I ever manage to write that?
Aaron Lun (06:46:41): > I only remember there was a lot of guesswork involved.
Kevin Rue-Albrecht (06:49:35): > lol, sounds like me after each refactoring
Kevin Rue-Albrecht (06:51:18): > … that, andprintstatements all over the place ^^
Kevin Rue-Albrecht (07:39:22): > Ok.. i’m giving up for now, but I actually see the same errorTypeError: input.redDimPlot1_VisualChoices is undefinedin the Safari console. But somehow that’s the only browser that probably reevaluates the statement again later, when the value is defined.
Kevin Rue-Albrecht (07:53:14): > Just before we waste (more) hours on that, I just wonder whether this could be something that we’ve fixed in the last 6 months. I always struggle to keep in mind that the RStudio Cloud is using thereleaseversion of iSEE (1.2.1). I’ll have to try on a 3.5 installation locally using Firefox to check that possibility
Aaron Lun (08:07:22): > The typeerror is fixed, if that’s what you’re saying.
Kevin Rue-Albrecht (08:10:53): > If it was fixed between the October release and now, that is what I was saying.
Kevin Rue-Albrecht (08:12:27): > Which then means that firefox and chrome won’t work for the shiny contest app until I can install an R3.5-compatible updated version
Kevin Rue-Albrecht (09:28:33): > I’ve just tried both 1.2.1 on R-3.5 and 1.3.8 on R-3.6, and I can see on both cases theTypeError: input.redDimPlot1_VisualChoices is undefined(in Firefox, haven’t checked the others). Do you remember when that was fixed?
Aaron Lun (10:02:54): > I just did it,jsfix.
Kevin Rue-Albrecht (10:48:38): > ah that’s what you meant. sorry i didn’t look at the fix yet
Rob Amezquita (11:46:19): > a quick suggestion/thought - i share iSEE with collaborators and they often have trouble running it..i point them to section 3 of the current vignette but often they got stuck on the technical details in the section before. itd be really useful to have either a separate vignette that has just the essentials (loading an SCE object into R, getting iSEE running in one/two lines) or at the top of the current one a “quick start” section. that way i can just link to the vignette vs rewriting how to run iSEE on the sce object i share every time with my non-bioinformatician colleagues:stuck_out_tongue:
Aaron Lun (11:46:45): > Uh. Isn’t?iSEEenough?
Rob Amezquita (11:46:52): > they often dont even know how to do that
Rob Amezquita (11:46:57): > they just want to point and click
Aaron Lun (11:47:11): > Well. That’s going to be a problem.
Aaron Lun (11:47:28): > I mean, if they can’t use R, then there’s not really any choice but to do a web deployment.
Rob Amezquita (11:47:32): > they know enough to start R, and cut and paste
Aaron Lun (11:47:53): > Minimal commands would be: > > library(iSEE) > example(iSEE, ask=FALSE) >
Aaron Lun (11:48:16): > If you provide them with a SCE object, then they can just do: > > library(iSEE) > sce <- readRDS("my_sce.rds") > iSEE(sce) >
Aaron Lun (11:48:32): > But anything more customized is going to require some programming skill.
Rob Amezquita (11:48:32): > yep! exactly
Rob Amezquita (11:48:38): > yeah no need for customizations
Rob Amezquita (11:49:21): > im just saying that a separate vignette withjustthat or a quick start section might be something that would be worth adding, thought id add my $0.02 since this is a frequent issue with experimental collaborators, and id love to point them to an official vignette that has that
Aaron Lun (11:49:39): > Well. I guess we could.
Federico Marini (11:50:14): > What they might like is also to have the panels set up on launch, I guess
Federico Marini (11:50:35): > One way I have been toying around is to launch it as default first
Federico Marini (11:50:38): > then setup the panels
Federico Marini (11:50:51): > cut out the code for the panel setup
Federico Marini (11:51:01): > and then customize the call + deploy that version
Aaron Lun (11:52:08): > I wouldn’t rely on biological collaborators to be able to do that.
Aaron Lun (11:52:21): > They just want:
Aaron Lun (11:52:22): > - a script
Rob Amezquita (11:52:24): > yeah, thats way too much. they literally just need to see what i put into the SCE object
Aaron Lun (11:52:28): > - open Rstudio.
Aaron Lun (11:52:32): > - press “Run”.
Rob Amezquita (11:52:35): > yes!
Aaron Lun (11:52:42): > No reading of the vignette is required here. You just tell them what to do.
Rob Amezquita (11:54:00): > im just thinking theres a fair amount of boilerplate in the current vignette in section 2, enough that its daunting to a novice, and that section 3 could be replicated very shortly at the top with the assumption a person has an SCE object already thats ready to go (with reducedDim, normalized, etc.)
Aaron Lun (11:54:32): > Hold on, hold on. Are you talking about section 2 of vignette 1, or vignette 2?
Federico Marini (11:54:38): > that would just be the elaborate script I would be giving them
Rob Amezquita (11:54:42): > https://bioconductor.org/packages/release/bioc/vignettes/iSEE/inst/doc/basic.html#3_launching_the_interface
Aaron Lun (11:54:47): > So vignette 1.
Rob Amezquita (11:54:54): > yep! that one
Kevin Rue-Albrecht (11:58:55): > btw, if we’re considering a new vignette specifically for this “get started” purpose, i recommend that the file is called “iSEE.Rmd”.https://pkgdown.r-lib.org/articles/pkgdown.html#navigation-bar > > By default, the top navigation bar (the “navbar”) will contain links to: > > - “Get Started”, if you have an article with the same name as the package (e.g., vignettes/pkgdown.Rmd).
Kevin Rue-Albrecht (11:59:20): > for when we deploy the pkgdown website
Aaron Lun (11:59:22): > Well. I just did it,jsfix``b4a499f.
Kevin Rue-Albrecht (12:00:31): > ah ok, but not as a separate extra vignette, so my point is mute for now
Aaron Lun (12:00:48): > moot
Kevin Rue-Albrecht (12:01:13): > I’m French, I’m allowed to do mistakes:stuck_out_tongue:
Kevin Rue-Albrecht (12:01:23): > do we expect more fixes, considering that you haven’t opened a PR yet?
Aaron Lun (12:01:30): > No, not really.
Aaron Lun (12:01:37): > Well, while I”m here, I might as well touch up some other things.
Aaron Lun (12:11:30): > Okay, I’m done.
Rob Amezquita (12:16:40): > submitted a PR to master with slight edits to jsfix@Aaron Lun
Aaron Lun (12:19:40): > Hey! I like my snark!
Aaron Lun (12:20:03): > Being able to insult people is the only thing that gets me out of bed in the morning
Federico Marini (12:27:20): > appreciating the half insult:slightly_smiling_face:
Rob Amezquita (12:28:59): > im just gonna push back on it a bit aaron - i know it was in jest (or at least half), but im writing this review on Bioconductor tools and featuring iSEE pretty heavily in its own section as a tool for experimentalists who dont have much expert knowledge of R. having been on and seeing both sides, i just dont want the docs to come off as contemptuous to folks who are earnestly trying to understand their data and need help doing so
Aaron Lun (12:29:32): > well, I think the current version is fine.
Aaron Lun (12:32:18): > Fine. Fine.
Aaron Lun (12:32:22): > I’ve halved it again.
Aaron Lun (12:32:25): > Now it’s only a quarter insulting.
Aaron Lun (13:50:16): > guh. Mentally checked out.
Aaron Lun (13:55:30): > @Kevin Rue-Albrechtdid the update fix anything?
Kevin Rue-Albrecht (14:07:23): > Will check now
Kevin Rue-Albrecht (14:07:41): > Been badly multitasking all day as usual
Kevin Rue-Albrecht (14:15:35): > tbh, I thought you tested it before you pushed. Now I remember why the expression “can’t be bothered” already sounded familiar:yum:
Aaron Lun (14:17:39): > Huh? It doesn’t work?
Kevin Rue-Albrecht (14:17:55): > hang on i just launched
Aaron Lun (14:17:58): > It works for me, just not on the shiny apps
Kevin Rue-Albrecht (14:18:18): > the remark was just teasing unrelated to the functionality
Aaron Lun (14:18:45): > geez gave me heart palpitations:flag-nz:
Kevin Rue-Albrecht (14:19:18): > it’s just that the way you asked me sounded like you hadn’t tested it yourself
Kevin Rue-Albrecht (14:19:55): > it’s all fine on my side, UI loaded and no error in console
Kevin Rue-Albrecht (14:21:05): > (Firefox, Safari, and Chrome)
Kevin Rue-Albrecht (14:22:29): > If you’re bored, Chrome does raise a few warnigs: > > The specified value "-Inf" is not a valid number. The value must match to the following regular expression: -?(\d+|\d+\.\d+|\.\d+)([eE][-+]?\d+)? jquery.min.js:3 > The specified value "Inf" is not a valid number. The value must match to the following regular expression: -?(\d+|\d+\.\d+|\.\d+)([eE][-+]?\d+)? jquery.min.js:3 >
Kevin Rue-Albrecht (14:22:37): > but it’s the only one (and the warnings do not seem to be causing any issue from a user experience)
Aaron Lun (14:25:19): > I would guess that’s a heatmap thing? It’s the only place where Infs could occur.
Aaron Lun (14:42:20): > CHECKING OUT
Kevin Rue-Albrecht (15:19:53): > aside from that (minor possible heat map thingy),@Charlotte Soneson@Federico Marinihow do we stand on thepush upstreampoint, btw?
Charlotte Soneson (15:20:41): > To me it looks fine, everything worked as expected except theid->1Lthing
Kevin Rue-Albrecht (15:21:25): > .. which is now fixed. Cool, thanks!
Kevin Rue-Albrecht (15:22:05): > I have to admit that triple-checking interactive apps is another level of package maintenance
Federico Marini (16:04:55): > I’ve also been experimenting around with the latest master, and found no bad surprise.
Federico Marini (16:05:48): > Re: contest, I still get only safari running. Which again makes me wonder, whether this is deterministic or not
Federico Marini (16:06:00): > cache cleared by both ff and chrome
Kevin Rue-Albrecht (16:07:22): > don’t bother with the cache, the contest app won’t be fixed until I can install the fixed version of iSEE from Bioconductor
Kevin Rue-Albrecht (16:09:18): > When I install iSEE from GitHub using devtools, the rsconnect::deployApp function crashes because of some really weird errors about installing system libraries
Kevin Rue-Albrecht (16:10:26): > So, in a way this week I’m pushing topush upstreamso that I can at leastBiocManager::install("iSEE", version="devel"), hoping that I can deploy [the contest app] using that version
Kevin Rue-Albrecht (16:11:08): > but on the other side I’d hate to rush a “push upstream” if it means releasing any issue we could have fixed
Federico Marini (16:25:41): > I’d say we 4x-checked enough
Kevin Rue-Albrecht (16:34:00): > Well, I’ll wait another couple of days to see if the Aaronmaster figures out the heat map thingy, or perhaps I’ll track that down myself, just not tonight
Rob Amezquita (17:02:46): - File (PNG): Screen Shot 2019-02-25 at 1.53.29 PM.png
Rob Amezquita (17:03:26): > dont share this/post this quite yet, but this is the explainer figure for iSEE that ive been drafting up today to go along with the review text re: iSEE, if you have feedback always appreciated
Rob Amezquita (17:04:01): > (the sce object is explained in more detail in an earlier figure so its only the bare essentials in this one)
Rob Amezquita (17:06:25): > just ping me in a message if you do, thanks!
Rob Amezquita (17:06:32): > @Rob Amezquita has left the channel
Aaron Lun (17:40:40): > I would say it looks neat, but he left.
Aaron Lun (17:43:43): > hold on@Kevin Rue-AlbrechtI thought there were problems with using Bioc-devel on the shiny apps server.
Stephanie Hicks (20:28:49): > I’ll tell him@Aaron Lun:wink:
2019-02-26
Kevin Rue-Albrecht (02:07:59): > @Aaron LunThe only problem I have with bioc-devel at the moment is that iSEE requires R-3.6, which means I can’t install and test it on RStudio cloud, let alone deploying it toshinyapps.io. That’s why I’m hoping that pushing 1.3.8 where we dropped that requirement could help
Kevin Rue-Albrecht (02:14:05): > You may have that mixed up with the fact that I’ve also trieddevtools::install_github, which allows me to install the latest versions on RStudio Cloud, but then I get errors about system libraries when I try to deploy toshinyapps.io
Charlotte Soneson (02:36:35): > Not sure if@Rob Amezquitawill be notified if I tag him here, let’s see:slightly_smiling_face:I think the schematic above looks very nice - one thing I would do though is to replace the figures by plots actually generated from iSEE - in particular, since (at least as far as I know, maybe I have missed something crucial, that would be embarrassing:flushed:) our default feature assay plots don’t actually show multiple genes in a single panel.
Kevin Rue-Albrecht (02:40:34): > oh you’re right i didn’t spot that, we can facet, but not by gene indeed
Kevin Rue-Albrecht (02:41:15): > that’d be problematic for the y-axis range i guess (although we could allow “free_y”)
Charlotte Soneson (02:42:25): > yeah, I guess we’d need to have a gene selector similar to the one for the heatmap panel
Kevin Rue-Albrecht (02:52:40): > oh right i’m stupid - we can’ facet, potentially 10k genes, or at least we’d subset to a handful of genes selected by the user
Kevin Rue-Albrecht (02:54:57): > But that’d be a nice complement to the current situation, which forces users to either use a single panel and constantly switch between genes, or open multiple featAssayPlot panels, only to have different axes. We should really have a “Clone panel” button, to quickstart new panels interactively.
Kevin Rue-Albrecht (02:57:44): > … although there’d be a hickup: in our setup, “cloning” would require an argument, namely which of the rows in the memory DataFrame should be used to create the new panel. Givent that we have a max for each panel type, we can’t add more rows to the memory. Oh well. Probably a bad idea of mine from the start.
Aaron Lun (04:55:31): > @Kevin Rue-Albrechtbut you wouldn’t even be able to useBiocManager::installto pull down the devel version, because it would just shit itself upon detecting the wrong version of R.
Kevin Rue-Albrecht (04:55:59): > ugh. i missed that one
Kevin Rue-Albrecht (04:57:43): > .. i guess it’ll just be release version 1.2.1 that’ll get submitted to the shiny contest. We can always edit a bit the submission post to indicate that some features are only available from bioc-devel or GH. I’ve made a point of that for the multiple selection
Kevin Rue-Albrecht (04:59:55): > I mean it’s not the end of the world. This contest is just some free publicity.
Aaron Lun (05:02:43): > getting it working on ff would be a start.
Kevin Rue-Albrecht (05:06:19): > well, until I find a way to deploy a version of iSEE that includes your recent fixes, the contest app ain’t gonna work on FF
Aaron Lun (05:10:49): > Just port the commit to release.
Aaron Lun (05:11:01): > And how do you know that’s the cause anyway? I was just guessing.
Kevin Rue-Albrecht (05:11:16): > what do you think is the cause?
Aaron Lun (05:12:16): > I don’t know. But there’s no reason to think that the javascript problem is the failure mode for FF, especially if it works fine locally.
Kevin Rue-Albrecht (05:14:35): > Well, I haven’t given too much thought about how to track down the shinyapps-specific issue. Not sure if that can be done from the FF console. I was just hoping to port the fix to something that can be installed in RStudio cloud, and then deployed toshinyapps.iofor testing. One of the pains of working on cloud system being the extra steps to bring everything up to date
Kevin Rue-Albrecht (05:15:57): > but you just gave me an idea, give me some time to try
Kevin Rue-Albrecht (05:16:09): > (or feel free to tackle it your own way)
Kevin Rue-Albrecht (05:16:59): > argh.. damn..Error : object 'int_metadata' is not exported by 'namespace:SingleCellExperiment'.. is that so recent?
Kevin Rue-Albrecht (05:17:11): > (i.e., not available in release)
Aaron Lun (05:17:29): > Dunno. Maybe.
Aaron Lun (05:17:34): > Can’t really remember.
Kevin Rue-Albrecht (05:19:17): > Maybe that’s why we added the requirement for R 3.6, in fact
Aaron Lun (05:19:39): > Well, the requirements should no longer matter, because BiocManager will force you to do things the right way.
Kevin Rue-Albrecht (05:19:56): > Basically, I’ve just created a new RStudio Cloud project, that literally clones the iSEE repo (https://rstudio.cloud/spaces/11327/project/233650), I was hoping todevtools::install()without any pointer to GH or Bioc (devel or release). But apparently, I can’t install iSEE because it relies on the*metadatafunctions that are not exported by the release version of SCE
Aaron Lun (05:21:09): > Don’t worry about it, just port the js fix to a release branch and try installing that to see if it helps any.
Kevin Rue-Albrecht (05:36:04): > question: how can bioc-release iSEE by version 1.2.1 if our GH branch 3.8 is only 1.2.0? > answer: nevermind, it’s that same issue with the biocViews that I didn’t pull from upstream yet
Federico Marini (05:43:28): > yes, immunooncology is the reason
Kevin Rue-Albrecht (05:47:08): > ok, so I just need a few more minutes: I started working on a machine that only has R-devel installed. I need to do it again on one where I can test it on R-3.5, to test in the real environment
Kevin Rue-Albrecht (05:47:40): > and also work on a side branch rather than RELEASE_3_8, just for safety
Kevin Rue-Albrecht (05:54:45): > here we go:https://github.com/csoneson/iSEE/pull/274
Kevin Rue-Albrecht (05:54:53): > tested and approved locally: R-3.5,BiocManager::valid()
Kevin Rue-Albrecht (05:55:45): > just give a shout or put a:thumbsup:on the PR if you bother testing
Aaron Lun (05:59:21): > Does it help?
Kevin Rue-Albrecht (06:01:49): > That’s what I meant by “tested and approved”. But in fact, let me check that indeed the previous version before the fix (1.2.1) was not working locally (i.e., that it’s not just a shinyapps.io-specific issue)
Kevin Rue-Albrecht (06:13:48): > PR (above) is read for merge to release. Does anyone know when the daily deadline is in UK time, to push new code for the next build/release?
Aaron Lun (06:15:33): > ¯*(ツ)*/¯
Aaron Lun (06:15:42): > Did it help?
Kevin Rue-Albrecht (06:19:17): > are you teasing me? that’s twice I wrote “tested and approved”. So perhaps, to rephrase: the new version with the fix works locally. I won’t be able to tell you the effect on shiynapps until i can install it from Bioc-release
Kevin Rue-Albrecht (06:22:28): > For the record, the fact that the plots don’t appear (before the fix) seems to be a ShinyApps.io-specific issue. I’ve just locally built the version before the fix, and launched the app in FF: I can see the error message in the console, but the plotsdoshow
Aaron Lun (06:23:25): > Well, that’s what I’m trying to say. If the javascript issue wasn’t causing a problem locally, then there’s not much hope that fixing it can fix the plot problem on shinyapps. We’re just guessing.
Aaron Lun (06:23:51): > Which is fine and all, but don’t get your hopes up.
Kevin Rue-Albrecht (06:25:21): > Yup. Sad face.
Kevin Rue-Albrecht (06:26:39): > That said, any last word against merging and pushing upstream? I think it’s pretty safe
Aaron Lun (06:27:31): > Go for it.
Kevin Rue-Albrecht (06:30:05): > done
Kevin Rue-Albrecht (06:32:53): > i love how GH nicely offers the banner of merging RELEASE_3_8 into master, just because it got updated
Kevin Rue-Albrecht (06:33:16): > “yes, i do want to break my package, please open that PR”
Kevin Rue-Albrecht (07:04:39): > FYI, we might be into a similar kind of situation to this one:https://community.rstudio.com/t/shinyapps-io-dashboard-not-working-in-firefox/5829/2 - Attachment (RStudio Community): Shinyapps.io Dashboard not working in Firefox > I would suggest checking out the R logs on shinyapps.io and see if you get anything different in Chrome vs. Firefox. The other place to look would be the “network” tab in “Firefox Developer Tools.” Go to “Menu” -> “Web Developer” -> “Console” or “Network” That shows you a bit more information about what is going on behind the scenes of the waiting you are stuck in.
Kevin Rue-Albrecht (07:17:22): > hang on, this one sounds bang on, haven’t read the whole thing yet:https://groups.google.com/forum/#!topic/shiny-discuss/cGbnVQbutLI
Kevin Rue-Albrecht (07:48:55): > FWIW, i’ve posted the issue here:https://community.rstudio.com/t/issue-with-firefox-and-chrome-for-an-app-hosted-on-shinyapps-io/24760 - Attachment (RStudio Community): Issue with Firefox and Chrome for an app hosted on shinyapps.io > Hello! In relation to the app that I described in this recent thread “Killed” error during installation for many packages I have a version of the project hosted here: https://rstudio.cloud/spaces/11327/project/230765 And deployed here: https://kevinrue.shinyapps.io/isee-shiny-contest/ The issue here is that we’re somehow resurrecting an old thread described here: https://groups.google.com/forum/#!topic/shiny-discuss/cGbnVQbutLI Namely, the app hosted on shinyapps.io works fine with Sa…
Kevin Rue-Albrecht (07:49:34): > better ask directly to those have the best chance of knowing
2019-02-28
Aaron Lun (10:03:57): > LAST DAY
Federico Marini (10:04:19): > sad?
Aaron Lun (10:05:03): > ¯*(ツ)*/¯
Charlotte Soneson (10:05:38): > Is tomorrow also the first day, or do you have a buffer period?
Federico Marini (10:05:40): > when are you officially starting in SF?
Federico Marini (10:05:45): > eh eh exactly
Aaron Lun (10:17:11): > Next mon
Charlotte Soneson (10:17:50): > -day or -th?
Aaron Lun (10:18:47): > day
Aaron Lun (10:18:57): > well, either.
Federico Marini (10:19:38): > tough one
Federico Marini (10:20:09): > I can guess the new bosses wanted you also ASAP:slightly_smiling_face:
Aaron Lun (10:20:24): > well, i’m not getting paid the medium bucks for nothing.
Kevin Rue-Albrecht (13:58:31): > go have fun:slightly_smiling_face:don’t forget to send postcards and stickers
2019-03-01
Aaron Lun (04:42:11): > Business class lounge FTW
Charlotte Soneson (04:49:28): > :cocktail:
Kevin Rue-Albrecht (04:59:54): > mediumbucks eh?:stuck_out_tongue:
Aaron Lun (05:06:40): > I’ve redefined ‘big’ as 250 K +
Aaron Lun (05:06:45): > USD
Kevin Rue-Albrecht (08:20:28): > It looks like jsfix fixed the shiny contest app:https://kevinrue.shinyapps.io/isee-shiny-contest/
Kevin Rue-Albrecht (08:20:46): > Tested Chrome, moving to Firefox
Kevin Rue-Albrecht (08:20:54): > yup, all good
Charlotte Soneson (08:22:05): > Yep, works for me too. Nice!
Kevin Rue-Albrecht (08:24:13): > Alright, so happy weekend in view. I’ll have another look at the draft submission post on GDoc and I’ll submit it if everyone’s happy. Deadline is March 8, but criteria incude: > > - Apps will be judged based on technical merit and/or on artistic achievement (e.g., UI design). We recognize that some apps may excel in one of these categories and some in the other, and some in both. Evaluation will be done keeping this in mind. > > -Evaluation will also take into account the feedback/reaction of other users in the submission posts in RStudio Community.
Kevin Rue-Albrecht (08:24:31): > (I bolded the last bit)
Federico Marini (08:25:39): > Excellent Kev, thanks for following up
Federico Marini (08:26:11): > i.e. we should advertise it properly among our friends in the community
Federico Marini (08:26:17): > or even to bioc-devel?
Kevin Rue-Albrecht (08:26:31): > now that the doubt is lifted, I’m thinking of launching the default tour on startup for the contest, to avoid the need for everyone to dig through the UI, whaty’all think?
Kevin Rue-Albrecht (08:26:54): > we removed it when we thought it could be the reason of the bug
Charlotte Soneson (08:26:54): > If it works - yes, I think that would be very useful
Federico Marini (08:27:49): > pro auto-launching the tour , totally
Kevin Rue-Albrecht (08:28:55): > Perhaps over the top, but I could also throw in a disclaimer that mentions how some exciting new features are only available from the developer branch (eg multi select)
Kevin Rue-Albrecht (08:29:49): > Like when we added the extra disclaimer with the red text for the iSEE2018 tours, I think
Federico Marini (08:32:03): > For example if I recall correctly, you wrote about the voice control, but that is not in the current version,amirite?
Kevin Rue-Albrecht (08:41:09): > Yup. Devel only
Kevin Rue-Albrecht (08:41:45): > Yeah thinking about it more, it’s definitely worth a disclaimer
2019-03-02
Kevin Rue-Albrecht (04:13:26): > i’ve resize the newly introduced logo in the basic vignette, perhaps we’re good forpush upstreamof devel? (1.3.8)
Kevin Rue-Albrecht (04:14:38): > I got some stuff to do this morning, but I’m also hoping to submit the shiny contest by the end of the weekend
Kevin Rue-Albrecht (09:53:27): > I deployed an update of the contest app to shiny apps : > - disclaimer moved to last step, inviting to visit our GH for latest features instead of opening with it > - edited disclaimer to mention voice recognition and new UI > - RStudio shiny contest, not iSEE contest. My bad:hugging_face:
Kevin Rue-Albrecht (11:19:20): > I think I’ll format and submit the post to enter the contest now. It’s always possible to amend the post and/or the app if we spot anything else to fix
Kevin Rue-Albrecht (11:43:39): > Here it is:https://community.rstudio.com/t/shiny-contest-submission-isee-interactive-summarizedexperiment-explorer/25136 - Attachment (RStudio Community): Shiny Contest Submission: iSEE, interactive SummarizedExperiment Explorer > Submission Project RStudio Cloud project: https://rstudio.cloud/spaces/11327/project/230765 Live app: https://kevinrue.shinyapps.io/isee-shiny-contest/ GitHub repository: https://github.com/kevinrue/isee-shiny-contest Description iSEE (interactive SummarizedExperiment Explorer) is designed for interactive exploration of high-throughput biological data sets stored in the SummarizedExperiment S4 class. It enables simultaneous visualisation of experimental data, associated metadata and analysis …
Kevin Rue-Albrecht (11:44:04): > Let me know if I should tag you somehow (e.g. if you have an RStudio community account, for instance)
Kevin Rue-Albrecht (11:48:35): > I just spotted this other submission, from 6h ago:https://community.rstudio.com/t/shiny-contest-submission-interactive-gene-explorer/25127 - Attachment (RStudio Community): Shiny Contest Submission: Interactive Gene Explorer > This shiny application is an interactive Gene Explorer. The dataset that is used for this app is divided into 9 contrasts, each containing (the same) 100 Genes. The first tab displays the Genes per contrasts and is used to quickly view the Genes that have increased or decreased. In the second tab, individual Genes can be analyzed. Links RStudio Cloud project : https://rstudio.cloud/project/243079 Shinyapps : https://ginberg.shinyapps.io/gene_explorer/ Github : https://github.com/ginberg…
Charlotte Soneson (11:50:34): > I just created an RStudio community account (username csoneson):slightly_smiling_face:
Kevin Rue-Albrecht (11:51:22): > indulge me and try that Interactive Gene Explorer app… I have no clue how to use it xD
Kevin Rue-Albrecht (11:51:36): > I’ll tag you in the post
Charlotte Soneson (11:58:38) (in thread): > Well, seems it makes MA plots and volcano plots by taking precalculated values from a provided data frame. It’s fast, but there are only 100 genes so it doesn’t really say much about how it would scale to real data.
Aaron Lun (12:03:40): > woke up
Aaron Lun (12:04:03): > In this temp apartment
Aaron Lun (12:04:24): > 1 bedroom in downtown san fran
Aaron Lun (12:04:33): > I’ll never be able to afford this for real.
Aaron Lun (12:04:43): > God the rent must be >4000 USD a month here.
Kevin Rue-Albrecht (12:04:57): > still will be sharing toilets with students then?:stuck_out_tongue:
Aaron Lun (12:05:02): > fuck no
Kevin Rue-Albrecht (12:05:16): > you could adopt one as a pet then maybe
Aaron Lun (12:07:30): > I’ll send some pictures shortly
Aaron Lun (12:08:00): > they even gave me a free plate in a welcome basket
Aaron Lun (12:08:25): > holy crap and a pasta strainer. I just noticed thatwasthe basket.
Kevin Rue-Albrecht (12:10:41) (in thread): > And this canvasxpress thing is super confusing with a ton of contextual menus which are probably useful in other contexts, but just get in the way herehttps://blog.dominodatalab.com/large-visualizations-canvasxpress/ - Attachment (Data Science Blog by Domino): Large Visualizations in canvasXpress > Dr. Connie Brett is the owner of Aggregate Genius. Dr. Connie Brett provides custom visualization tool development and support for the Translational Bioinf
Charlotte Soneson (12:11:29) (in thread): > looks a bit like plotly to me
Kevin Rue-Albrecht (12:11:56) (in thread): > yeah, i’m probably just too lazy to climb yet another learning curve right now
Charlotte Soneson (12:12:31) (in thread): > I feel the same. There has to be a good reason.
Kevin Rue-Albrecht (12:12:34): > well i do hope the medium bucks get you something fancier than pasta:stuck_out_tongue:
Aaron Lun (13:38:39): > Sweet-ass living room. View is of downtown SF from the 14th floor. - File (JPEG): livingroom.jpg
Aaron Lun (13:39:49): > Kitchen. Those are welcome baskets, containing free stuff like a plate and a bug and some coffee and pasta and past sauce. And the pasta strainerisone of the baskets:shocked_face_with_exploding_head:
Aaron Lun (13:40:14): - File (JPEG): kitchen.jpg
Aaron Lun (13:40:39): > Bedroom. And yes, that is a walk-in wardrobe. - File (JPEG): bedroom.jpg
Kevin Rue-Albrecht (13:55:38): > Neat!
Kevin Rue-Albrecht (13:56:40): > > a plate and a bug and some coffee > pardon my ignorance, but what’s a “bug” in this case?
Kevin Rue-Albrecht (14:00:34): > Anyway, that’s a sweet space and the kitchen looks real nice. Walk-in wardrobe might convince Anna to cross the Atlantic some day if the offer arises:stuck_out_tongue:
Aaron Lun (17:24:08): > mug
Aaron Lun (17:24:38): > And - oh - you can expect this to be about 5000 USD a month, judging by nearby prices.
Stephanie Hicks (18:10:11): > 5k???
Stephanie Hicks (18:10:34): > :grimacing:
Stephanie Hicks (18:11:12): > The pictures are great though!
Kevin Rue-Albrecht (18:23:13): > Ouch. So rent is definitely a factor toward the medium bucks. Wow
2019-03-03
Federico Marini (08:51:20): > hey you even have a tv
Federico Marini (08:52:18): > you have to take a picture of you walking in the wardrobe
Federico Marini (08:52:35): > choosing among stickers of scran, scater, and iSEE
Federico Marini (08:55:41): > anyway: cool stuff my friend, enjoy the temporary paradise:smile:
Kevin Rue-Albrecht (14:54:01): > FYI, I’ve got only 2h of my 25 “active hours” available until the end of my monthly cycle ofshinyapps.io(every 12th of the month). Let’s leave them to anyone else please
Kevin Rue-Albrecht (15:38:10): > @Federico MariniI hope I’m not repeating anything mentioned earlier, but for your iSEE_instances, here’s a good one:https://twitter.com/coletrapnell/status/1098335247056269312 - Attachment (twitter): Attachment > Our atlas of gene expression during mouse organogenesis is out today (https://rdcu.be/bnIu0) ! Joint work with @JShendure, led by @junyue_cao and @malte_spielmann. Help us explore one of the largest, most complex gene expression datasets ever: https://oncoscape.v3.sttrcancer.org/atlas.gs.washington.edu.mouse.rna/landing
Kevin Rue-Albrecht (15:38:34): > In particular, check out their download page:https://oncoscape.v3.sttrcancer.org/atlas.gs.washington.edu.mouse.rna/downloads
Kevin Rue-Albrecht (15:38:40): > counts and metadata
Federico Marini (15:38:41): > definitely worth keeping an eye on
Federico Marini (15:38:49): > it is as huge as one could hope for:smile:
Kevin Rue-Albrecht (15:39:55): > I was wondering whether to try and set up an instance ourselves (I really don’t have time), or bait them into doing it for us:wink:
Kevin Rue-Albrecht (17:48:10): > I’m not clear why, but bioc-devel build fails to pass some unit tests for custom panels:http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/malbec2-checksrc.htmlI’m probably going to push 1.3.8 upstream soon, after checking that it passeschecklocally withBiocManager::valid
Aaron Lun (22:03:07): > that organogenesis data is pretty ~meh.
2019-03-04
Kevin Rue-Albrecht (02:14:27): > Unit tests appear to be broken locally too, after bringing BiocManager up to date.
Charlotte Soneson (03:12:34): > related to this?https://github.com/davismcc/scater/commit/179e9f7d78ac68154d88cf45639a2d8e3e5a1975
Charlotte Soneson (03:12:54): > seems it fails when running the PCA
Charlotte Soneson (03:17:51): > I haven’t looked too carefully, but it seemsscale_columnshas been replaced withscale_features
Kevin Rue-Albrecht (03:25:14): > Sounds like it. I can’t fix it right now though, if you wanna have a go
Charlotte Soneson (03:45:05): > That fixes one thing, but I get other errors locally as well. Trying to track them down
Charlotte Soneson (04:03:21): > Ok, fix pushed tofixpcabranch. I added acontext()to alltest_*.Rfiles, just to make it easier to keep track of where the failed tests/warnings etc occur.
Charlotte Soneson (04:50:47): > Merged
Federico Marini (04:51:27): > well spot, thanks!
Kevin Rue-Albrecht (16:49:02): > In the last couple of days, the contest app alone blew up my monthly allowance hehehe - File (PNG): Pasted image at 2019-03-04, 9:48 PM
2019-03-07
Federico Marini (04:51:01): > Hi everyone, one possible small yet significant improvement in exploration
Federico Marini (04:51:06): > After seeing this:https://twitter.com/tjmahr/status/1100871317056827392 - Attachment (twitter): Attachment > # TIL about geom_dl() #rstats > ggplot(iris) + > aes(x = Sepal.Length, y = Sepal.Width, shape = Species, color = Species) + > geom_point() + > directlabels::geom_dl( > aes(label = Species), method = “smart.grid”) https://pbs.twimg.com/media/D0cUoBQUcAAAQAi.png
Federico Marini (04:51:30): > I often have headache or can’t discern cluster numbers from the tSNE plot
Federico Marini (04:51:52): > what if we just display the label of the cluster with this?
Federico Marini (04:52:23): > can work only for factors - and has a checkbox associated to toggle
Kevin Rue-Albrecht (07:24:46): > Not that it’s particularly complicated (compute the mean x/y coordinate and ggplot another layer), but for reference I’ve also done/demonstrated that here:https://kevinrue.github.io/hancock/articles/hancock.html#interactively
Federico Marini (07:26:44): > yes, pretty much
Federico Marini (07:27:22): > geom_dl would auto-handle the placement, which can be tricky in overcrowded tSNE plots
2019-03-08
Aaron Lun (01:00:23): > scater’s plotReducedDim does this as well.
Aaron Lun (01:01:29): > though I wonder whether I could just piggy-back off this labelling thing.
2019-03-11
Kevin Rue-Albrecht (06:15:43): > @Aaron Lun@Federico Marini@Charlotte SonesonI just wanted to give everyone a gentle reminder that the bioc-devel branch of iSEE is still at 1.3.7 (https://github.com/csoneson/iSEE/commit/f478dac5d523599bd43017c2d4e90f1bbffbc0b4). > I’m not working on anything related to iSEE right now, so you’ll just have to ping me if you think our GitHubmasteris ready to be pushed upstream. > Have a good week!
Aaron Lun (11:31:27): > I thought you guys already did it.
Kevin Rue-Albrecht (13:26:51): > Last time I asked about pushing upstream we kept spotting minor fixes to throw in the release, so I decided to wait until things settle. Right now my only objective is simply to remember pushing it before the April release. I don’t mind waiting if we want to add more to 1.3.8. Alternatively, we can always push now and make a 1.3.9 if needed. Just let me know, I can be the fingers if anyone wants to be the brain.
Aaron Lun (22:08:39): > JUST DO IT
2019-03-12
Kevin Rue-Albrecht (06:57:39): > Done. I also check’ed locally one more time after updating to Bioc’valid, so we shouldn’t expect any bad surprise from the bioc build.
2019-03-14
Charlotte Soneson (04:01:12): > Ok, I just pitched our iSEE shiny competition submission to Hadley:slightly_smiling_face:he’s not involved in the competition, but still:shrug:
Federico Marini (04:52:44): > We should have the results out soon, right?
Federico Marini (04:53:36): > it would be mega to be somewhere in the rankings:exploding_head:
Charlotte Soneson (04:54:39): > Apparently now you can vote on the submissions using the heart in the submission post (I guess). Not sure how that will be used in the judging:slightly_smiling_face:
Federico Marini (05:02:44): > I did drop mine already
Charlotte Soneson (05:58:54): > Lots of interest in iSEE from (pharma) companies here, I must say.
Federico Marini (06:05:58): > damn you MIT License:stuck_out_tongue:
Federico Marini (06:06:07): > but damn cool, hey
Charlotte Soneson (07:30:32): > Also some more questions about exporting selections.
Federico Marini (09:38:10): > -> for that: could we just have an extra column where all is stored and potentially exported at once with a single dedicated button?
2019-03-18
Federico Marini (05:48:06): > did any of us ever work with methylation dataandwould have an idea of how that would look like in iSEE?
Kevin Rue-Albrecht (05:51:21): > I did (https://www.nature.com/articles/s41598-018-37618-z) and I still have PTSD about it
Federico Marini (05:55:37): > GOT IT:smile:
Kevin Rue-Albrecht (05:55:47): > I guess for illustrative purposes one could apply dimensionality reduction on a subset of loci with sufficient read coverage using the “fraction of methylated reads” (imprecise description here, but you get the idea)
Kevin Rue-Albrecht (05:56:04): > got what?
Federico Marini (05:57:19): > to kind of stay away of it if not strictly needed?
Federico Marini (05:57:54): > would the feature be a CpG island, or any region you could define a priori e.g. promoters?
Kevin Rue-Albrecht (05:59:31): > If you haven’t checked outbsseqpackage yet, you’ll see that [their function can process] a feature can be a “CG” dinucleotide (stranded or after collapsing the dinucleotide palindrom across both strands -> double your coverage in the process), or a genomic region (e.g. CpG island, promoter, whatever)
Kevin Rue-Albrecht (06:00:11): > getMeth()
Federico Marini (06:00:35): > and that returns a 0-1 value (%step), I guess
Federico Marini (06:00:44): > hi, med, low, no methylation
Kevin Rue-Albrecht (06:01:10): > Yup, ratio of reads indicating a methylation event over total number of reads
Kevin Rue-Albrecht (06:01:38): > My sanity check plot was this one:https://www.nature.com/articles/s41598-018-37618-z/figures/3
Federico Marini (06:03:06): > Cool, a no-change scenario
Kevin Rue-Albrecht (06:03:41): > You have no idea how much “fun” that was to get passed reviewer #2
2019-03-19
Federico Marini (10:49:49): > Quick “buggy-like” report
Federico Marini (10:50:32): > - in red dim plot, select visual params > - facet > - facet by col, or row > - deselect facet > -> and the facetting stays
Federico Marini (10:51:24): > probably because upon de-selecting the one checkbox, the “children” do not get reset to non-selected
Kevin Rue-Albrecht (10:55:29): > Hang on.. does that mean that once facetting is “on”, it can’t be turned off?
Federico Marini (10:55:56): > it can. it needs to be turned off also on the facet by row/col level
Kevin Rue-Albrecht (10:56:34): > Oh right. Still buggy, but at least there’s a workaround in the meantime.
2019-03-21
Federico Marini (12:54:25): - File (JPEG): 20190321_171636.jpg
Federico Marini (12:54:26): > Achievement unlocked for today:slightly_smiling_face:
Federico Marini (12:54:56): > (if she looks new to you, that’s Heike Hofmann - academic mother of Hadley & Yihui Xie:slightly_smiling_face:)
2019-03-22
Federico Marini (10:21:26): > https://community.rstudio.com/t/shiny-contest-update/26774/2 - Attachment (RStudio Community): Shiny Contest Update > Back in January, we committed to announcing a winner of the 2019 Shiny Contest on March 22. We were happy to see so many submissions with such quality. This was significantly more than we expected! And so, are plannin…
2019-03-23
Aaron Lun (18:59:53): > lol
2019-03-29
Kevin Rue-Albrecht (05:24:51): > For a split second, I read “Log in with iSEE” - File (PNG): Pasted image at 2019-03-29, 9:24 AM
Charlotte Soneson (05:25:47): > :joy:that would be taking things to the next level!
Charlotte Soneson (05:26:12): > write the paper on Overleaf directly from the results in the iSEE instance
Kevin Rue-Albrecht (05:27:35): > We need a new button in the UI to insert <Reduced dimension plot 1> panel ^^
Federico Marini (05:41:22): > New projectdraft google doc coming?:stuck_out_tongue:
Federico Marini (05:44:07): > between a Marie Kondo meme and Drake, I also put an ambitious quote edit from the hitchhiker guide to galaxy in the last presentation
Federico Marini (05:44:55): > with theiSEE first
Federico Marini (05:45:33): > (funny enough, one did not get exactly where the citation was, and came to me after the talk and said “hey cool, you even quoted the guardians of the galaxy”)
Federico Marini (05:45:53): > (to which I had to contain myself andnotreply GROOT to him)
2019-03-30
Aaron Lun (01:57:46): > lol
Aaron Lun (02:53:44): > Is whatever problem@Kevin Rue-Albrechtreported fixed?
Aaron Lun (02:53:53): > I can’t do any iSEE on my computer right now. Only 2GB of RAM.
Kevin Rue-Albrecht (08:37:39): > Damn: what did I report? I’ve been so out of the loop recently:sweat_smile:I really need to file those things as proper GH issues
2019-04-02
Federico Marini (07:51:09): > Had we had this back in December 2017 ->https://arxiv.org/abs/1903.12180v1 - Attachment (arXiv.org): ACRONYM: Acronym CReatiON for You and Me > Each year, countless hours of productive research time is spent brainstorming creative acronyms for surveys, simulations, codes, and conferences. We present ACRONYM, a command-line program…
Federico Marini (07:51:57): - File (Plain Text): Untitled
Federico Marini (07:52:09): > I like INTRUDER
Charlotte Soneson (07:59:11): > :slightly_smiling_face:
2019-04-03
Kevin Rue-Albrecht (04:08:06): > FYI: iSEE workshop accepted:party_parrot:Without going into details, after a bit of hesitation (wedding etc.), I’ve decided to go to Bioc 2019. > That said, I would appreciate a lot if@Federico Marini@Aaron Luncould also join in the effort to help@Charlotte Sonesonand myself prepare the workshop materials. > As I discussed with Charlotte, all the necessary materials already exist in some shape or form, it’s just about stitching it in the format required for BiocWorkshops > I’ll forward the email about workshop instructions to everyone shortly.
Federico Marini (04:08:43): > Sure. We can all benefit from this actually
Federico Marini (04:08:56): > Re: wedding. So is now the wedding in NYC?:smile:
Federico Marini (04:09:38): > But first things first: congratulations:hugging_face:
Federico Marini (04:09:53): > Is it coupled to a travel fellowship as well?
Kevin Rue-Albrecht (04:10:46): > Wedding: Still Naples (I value my life!) > Travel scholarship: Yep. I would not consider going otherwise.
Federico Marini (04:14:08): > > Wedding: Still Naples (I value my life!) > Well said:smile:
Federico Marini (04:14:56): > Operatively, after checking the email: it follows the nice f1000res manuscript put together by sean davis about the experience of last year
Kevin Rue-Albrecht (04:20:59): > Cool. Well, right now this morning I really need to put in some work to make him at least a bit happy, so I’ll let you talk about it with Charlotte if you like. Just tag me if you need me for anything, although I think I forwarded you all the information I received from the organizing committee so far.
Federico Marini (04:24:01): > I think the best way to orient would be to check out last year’s workshops. Which I did already last year, so I need to re-check:slightly_smiling_face:
Federico Marini (04:24:18): > Probably we could get closest to the cytoscape-ish application
Federico Marini (04:24:41): > but for us the challenge is the static nature of the material
Stephanie Hicks (21:31:01): > congrats on the workshop@Kevin Rue-Albrecht@Charlotte Soneson!!
2019-04-04
Charlotte Soneson (01:06:29): > Thanks@Stephanie Hicks! Hope to see you in New York:smiley:
Kevin Rue-Albrecht (03:09:15): > Thanks! I am so looking forward to seeing everyone!
Aaron Lun (03:18:13): > re reviewing: I’ll try to look at it, but I can’t make any promises. You’ll notice the reduced rate of my commits on GH; this is literally the dregs of my programming willpower when I get home from work.
Kevin Rue-Albrecht (03:30:51): > I’ve been there recently. Obviously nowhere near as many packages as you typically work on but being split on too many things and feeling drained. > Anyway, I’m pretty confident that as soon as one of us lays down the first chunk of code, the rest of us will snowball as usual into something pretty cool pretty fast
Kevin Rue-Albrecht (04:05:43): > Btw, I was thinking about using thehttps://bioconductor.org/packages/release/data/experiment/html/TENxPBMCData.htmlpackage for source data. > From there we could follow a workflow similar tohttps://github.com/kevinrue/hancock2018/blob/master/0-Seurat_PBMC3k_tutorial_TENxPBMCData.Rmd(replacing Seurat by good ol’scran``DropletUtilsetc ) to get some clustering, markers, etc.
Federico Marini (04:08:55): > Sounds sound. Also given the fact that the seurat vignette is pretty well known, and that people have been dissecting it quite a while, with many marker-identifying methods and stuff like that going on
Federico Marini (04:09:22): > If you find something that could fit to that: it would be cool to have also something based on trajectories to check
Federico Marini (04:14:04): > I can try and ask a local person - if I recall correctly, he already had such a set
Federico Marini (04:14:20): > with differentiation trajectories and a nice bifurcation
Charlotte Soneson (04:15:44): > Yes, I think that sounds good. So we can have a main workflow that uses scRNA-seq data, and then perhaps show some other applications in the end (like bulk RNA-seq and cytometry).
Federico Marini (04:16:22): > … or that, yes.
Charlotte Soneson (11:30:43): > So if I get it right we eventually have to push an .Rmd tohttps://github.com/Bioconductor/BiocWorkshops.@Kevin Rue-Albrecht- shall we make a new .Rmd file in your repository to keep it all there (I guess we don’t change the current README, or)?
Kevin Rue-Albrecht (12:41:50): > Right about the Rmd. I just invited you all as contributors to my fork
Kevin Rue-Albrecht (12:44:36): > ignore the iSEE branch and just start fresh
Charlotte Soneson (12:45:18): > Okidok
Kevin Rue-Albrecht (12:58:52): > That branch was just a draft of workshop outline, but we should use the one from the submissionhttps://github.com/kevinrue/Bioc2019-iSEE-workshop
Charlotte Soneson (15:37:10): > Ok, I made a new branchisee2019inhttps://github.com/kevinrue/BiocWorkshopsand started a .Rmd file with some suggestions on a backbone (XXX_RueAlbrecht_iSEE.Rmd) (copying the content of theBioc2019-iSEE-workshopREADME).
Kevin Rue-Albrecht (15:46:00): > Awesome. Thanks!
Kevin Rue-Albrecht (15:49:15): > Nice job on the outline. That’s actually super helpful to picture the workflow and maybe even distribute sections amongst ourselves
Charlotte Soneson (15:59:17): > Not really sure how to knit it though - running thebookdowncommand tells me that I’m missing'BioC2018.rds':thinking_face:will try again tomorrow.
2019-04-05
Kevin Rue-Albrecht (08:34:49): > I remember trying a while back and discussing technical issues with@Levi Waldron, but I don’t remember this one. I actually remember having to install some packages directly from GH myself:https://github.com/kevinrue/BiocWorkshops/blob/master/101_MacDonald_Annotation.Rmd#L97needinghttps://github.com/jmacdon/Bioc2018Anno
Kevin Rue-Albrecht (08:36:37): > I highly doubt that the issue is related to the new iSEE notebook, but perhaps you’ll want to start by compiling themasterbranch of the original repo (https://github.com/Bioconductor/BiocWorkshops)
Charlotte Soneson (08:37:11): > Right, thanks. I’ll play a bit more with this soon.
Kevin Rue-Albrecht (08:37:40): > Not that it particularly helps, but here is an acknowledgement that “Bioc2018.rds” is a real thing:wink:https://github.com/Bioconductor/BiocWorkshops/commit/3db1b991ff02c730bf27af02659831f207f97d13
Charlotte Soneson (08:38:18): > Haha, yes, but clearly not intended to be committed to GitHub!
Kevin Rue-Albrecht (09:11:58): > Can’t remember how I got it, but …:stuck_out_tongue: > > $ pwd > /Users/kralbrecht/git/BiocWorkshops > $ find . -name BioC2018.rds > ./BioC2018.rds >
Charlotte Soneson (09:12:19): > Thanks! That’s good to know
Charlotte Soneson (09:12:31): > Maybe I just need to build the index. I haven’t tried very hard to be honest…
Kevin Rue-Albrecht (09:13:28): > ohhhh that’s right, I just read that a few minutes ago, I definitely did that last time
Kevin Rue-Albrecht (09:17:21): > That said, I can’t explain which dark magic createdBioc2018.rdsThe closest I’ve seen is a file inpacker/bioc_2018.json
Kevin Rue-Albrecht (09:19:18): > Applying > > Rscript -e "bookdown::render_book( \ > 'index.Rmd', 'bookdown::gitbook')" > > I’ve not at: > > processing file: 102_Lawrence_GenomicRanges.Rmd > ... > ordinary text without R code > > |................................................... | 78% > label: unnamed-chunk-8 >
Charlotte Soneson (10:07:47): > Yep, same for me, been standing there for a while.
Kevin Rue-Albrecht (10:13:06): > I finally crashed a bit further > > Error in library(GenomicDataCommons) : > there is no package called 'GenomicDataCommons' > > If you’re lucky you have it installed, otherwise I’d recommend opening another R shell and installing it while the current long step is running:wink:
Charlotte Soneson (10:14:37): > Thank you:pray:indeed I didn’t:smile:
Charlotte Soneson (10:16:06): > Checking all the other packages in DESCRIPTION…
Kevin Rue-Albrecht (10:16:34): - File (Shell): From BiocWorkflows DESCRIPTION
Kevin Rue-Albrecht (10:17:09): > I’ve just done a sed-style trick to write the command above:wink:
Kevin Rue-Albrecht (10:17:43): > Argh, there are some versions in there… hang on ^^
Kevin Rue-Albrecht (10:18:45): > updated
Charlotte Soneson (10:19:54): > :+1:
Federico Marini (10:20:31): > been there, done that:slightly_smiling_face:Remembering the days when the source of the workshop got published, but not yet the pdf, and I wanted to do some DIY
Kevin Rue-Albrecht (10:48:27): > Oh, and there are obviously also > > BiocManager::install(c("rstudio/bookdown", > "seandavi/SRAdbV2", > "jmacdon/Bioc2018Anno", > "seandavi/BiocPkgTools", > "npcooley/FindHomology", > "drisso/fletcher2017data", > "bhklab/mCI", > "bhklab/PharmacoGxML")) >
Charlotte Soneson (10:49:48): > :joy:still hoping to get there on time before the build crashes…
Kevin Rue-Albrecht (10:51:41): > oh no wait, that’sdevtools::install_github!
Federico Marini (10:51:55): > does work anyway
Federico Marini (10:52:09): > install falls back to that, AFAIK
Kevin Rue-Albrecht (10:52:28): > Oh ok, wasn’t sure, it’s crashed anyway because the library database was corrupt.
Kevin Rue-Albrecht (10:52:49): > I was running the two commands to install packages concurrently hehehe
Federico Marini (10:54:09) (in thread): > options(Ncpus=2)or more is always in my profile file
Federico Marini (10:54:28) (in thread): > but yes, sometimes I get impatient as well
Kevin Rue-Albrecht (10:58:31): > PharmacoGxML requiresBiocManager::install("glmnet")
Charlotte Soneson (11:00:01): > It’s great to be a little bit behind - I don’t have to figure out anything myself:wink:
Kevin Rue-Albrecht (11:02:09): > Can’t believe I’m writing that but… the installation of remotes just completed!
Kevin Rue-Albrecht (11:02:23): > I suppose I got’em all
Charlotte Soneson (11:02:52): > Wohoo!
Kevin Rue-Albrecht (11:03:07): > > 102_Lawrence_GenomicRanges.knit.md > |....................................... | 60% > label: unnamed-chunk-10 (with options) > List of 1 > $ cache: logi FALSE >
Federico Marini (11:03:21): > Looks like a race here:smile:
Charlotte Soneson (11:03:59): > Haha - I’m racing against myself - will I manage to install all packages before they are needed?
Charlotte Soneson (11:04:18): > And will things pick up where I left when I have to close my laptop and go home:smile:
Kevin Rue-Albrecht (11:04:19): > … Probably not the smartest thing to run on my laptop then, if it’s a race:wink:
Federico Marini (11:06:31): > Come on Charlotte, you can keep the lid open and still let it run:smile:
Federico Marini (11:06:52): > Plus, you maximize the exposure of your stickers
Charlotte Soneson (11:07:17): > I don’t have them anymore!! I changed computers when I changed jobs…the stickers are on the other laptop, on the table at home…
Charlotte Soneson (11:08:09): > But they did indeed start at least one or two conversations on the train:upside_down_face:
Kevin Rue-Albrecht (11:13:02): > Ugh. Beware of that “unnamed-chunk-10” above …
Federico Marini (11:14:59): > Need a new one for iSEE or do you still have some left?
Charlotte Soneson (11:15:32): > ah, I still have some
Charlotte Soneson (11:18:36): > Ok, all packages installed - now I’ll just wait…
Kevin Rue-Albrecht (11:23:30): > I haven’t timed it, but that chunk is infinite. Let me know if you get passed it faster, because I’m starting to think all that package installation messed with the compilation ^^
Charlotte Soneson (11:25:10): > Ok. I’m still at > > |................................................... | 78% > label: unnamed-chunk-8 >
Kevin Rue-Albrecht (11:25:44): > We’ll need to have a word with Michael:stuck_out_tongue:
Kevin Rue-Albrecht (11:43:16): > my bad , it’s actually Levi’s workshop, I missed a section header:sweat_smile:
Charlotte Soneson (11:44:12): > Ah! I’m not sure what’s going on here, I’m stuck now since almost 2 hours…maybe I just need to restart:woman-shrugging:
Kevin Rue-Albrecht (11:47:28): > I’ve done that and got stuck at the same spot
Charlotte Soneson (11:47:43): > Hmm, might have been our firewall that doesn’t likeAnnotationHub…:woman-facepalming:I’ll try from home instead
Kevin Rue-Albrecht (11:47:58): > If I count right, the 10th unnamed chunk is > > sra_get_swagger_json_url() >
Kevin Rue-Albrecht (11:48:10): > I’m running the notebook interactively now
Kevin Rue-Albrecht (11:50:29): > No, hang on.. interactively I get into chunk 41, which has > > while(s$fetched < 500) { > res = s$yield() > # do something interesting with `res` here if you like > j = j + 1 > message(sprintf('total of %d fetched records, loop iteration # %d', s$fetched, j)) > } >
Kevin Rue-Albrecht (11:50:44): > And right now it’s sayingtotal of 0 fetched records, loop iteration # 1167
Kevin Rue-Albrecht (11:51:09): > it should have stopped at 500 I suppose
Charlotte Soneson (11:51:27): > Sounds like it
Kevin Rue-Albrecht (11:51:37): > s$fetchedprints0
Kevin Rue-Albrecht (11:51:51): > looks like the counter isn’t useful at all, hence the “infinite”
Kevin Rue-Albrecht (12:16:57): > The README does say “You do not need to build the complete book yourself, it is sufficient to ensure that your own workshop compiles successfully.” I guess we can do that while the repo / older workshops are being brought to 2019
Charlotte Soneson (12:18:51): > Yes - but when I tried to do that it told me I was missing that rds file. If you are able to build our workshop, maybe you can send me the file…:nerd_face:
Kevin Rue-Albrecht (12:19:53): > ahhhhh I didn’t try that yet
Kevin Rue-Albrecht (12:20:24): > but didn’t the RDS file show up at the root of the repo for you?
Kevin Rue-Albrecht (12:20:41): > I genuinely cannot remember how I got it
Charlotte Soneson (12:22:15): > Ah, you’re right! Didn’t even look - it seems to have been generated despite the whole book not finishing compiling. I will try to build only the iSEE one again.
Kevin Rue-Albrecht (12:33:42): > The last chunk of the README worked for me! > > mkdir tmp > cd tmp/ > ln -s ../XXX_RueAlbrecht_iSEE.Rmd > Rscript -e "bookdown::render_book( \ > 'XXX_RueAlbrecht_iSEE.Rmd', 'bookdown::gitbook', preview=TRUE \ > )" >
Kevin Rue-Albrecht (12:34:05): - File (PNG): Pasted image at 2019-04-05, 5:34 PM
Charlotte Soneson (12:34:49): > Yup, me too. Ok, that was easier (and more complicated at the same time) than expected:slightly_smiling_face:Thanks!
Kevin Rue-Albrecht (12:42:47): > I just had a chat with Levi. Bottom line > > Building the complete book is a pain > So, I’m a bit supporter of the strategy “BYOB” (build you own book, what were you thinking?!) with the trick above
Kevin Rue-Albrecht (12:43:26): > That and sitting tight for instructions about the bigger picture (or in this case, book), if any
2019-04-06
Charlotte Soneson (01:13:23): > Did you guys see?!? - File (JPEG): Image from iOS
Aaron Lun (01:58:01): > Yeah, I just found out from one of my Genentech colleagues who asked me to present it on Mon.
Aaron Lun (01:58:47): > The most important prize is that we get a half-hour one on one to press for changes to the brush and lasso.
Aaron Lun (01:59:22): > Also see if we could improve the speed of the plot generation, but that’s really ggplot’s problem.
Charlotte Soneson (02:46:19): > According to Hadley’s talk last month they are working on this, aiming for a 15x ggplot2 speed-up.
Kevin Rue-Albrecht (05:10:52): > I was talking with a colleague and wondered whether we could take it all the way to « gganimate »:innocent:
Kevin Rue-Albrecht (05:11:58): > Although that said I would have to make my first gganimate plot before:joy:
Kevin Rue-Albrecht (05:12:35): > (And clearly that would have to be an endpoint panel)
Kevin Rue-Albrecht (05:13:24): > Anyway: Awesooooooome!
Kevin Rue-Albrecht (05:30:25): > If they contact me directly (e.g. email), I’ll make sure to include you in the discussion as soon as possible. > Re: prize > - Basic plan for a year: as I remember we all get it > - stickers: I doubt they’ll send 4 of each, but I’ll keep you posted (literally, if you give me a mailing address). I can send a photo of all the stickers when they arrive and let you have a pick, if there are some you “must have”:slightly_smiling_face:- $200 should easily cover 1 T-shirt and 1 mug each, then maybe I can see if a book fits in budget, or grab extra T-shirt/mug > - Special & persistent recognition by RStudio:@Aaron Lunmake sure you have an RStudio community profile (after Bioconductor, you’ll be able to compete with the next league on community sustainers) and I’ll make sure we all get the winner badge on our profile page > - Half-an-hour one-on-one: I genuinely hope they’ll accept as many of us as can make it (i.e. up to 4). I’m not sure what I’d talk to them by myself. Here is a GDoc to identify and list topics of conversation:https://docs.google.com/document/d/196tFRn4PpARh1leTWfGYnuG4v3r8TdkG1gRgZowYMYE/edit?usp=sharing
Kevin Rue-Albrecht (06:45:31): > I just realized I got a “Could you be so kind as to: …” message from “EconomiCurtis” on RStudio community. Rather than spamming the channel, I’ll start a GDoc tracking down the process, and ping you here about the important stuff ok?
Charlotte Soneson (06:48:00): > Sounds good!
Kevin Rue-Albrecht (07:03:58): > Please check the GDoc, section “Help needed: Choosing the prize preferences”
Kevin Rue-Albrecht (07:22:42): > From a mix of: > -https://www.rstudio.com/resources/training/books/-https://www.amazon.co.uk/Markdown-Definitive-Guide-Chapman-Hall/dp/1138359335/ref=asc_df_1138359335/?tag=googshopuk-21&linkCode=df0&hvadid=310810203983&hvpos=1o3&hvnetw=g&hvrand=18257018787173676001&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=1006976&hvtargid=pla-503600170968&psc=1&th=1&psc=1it looks like books are about ~$20 - Attachment (RStudio): Books - RStudio > The RStudio team has authored several of the most popular books on data science with R. - Attachment (amazon.co.uk): R Markdown: The Definitive Guide (Chapman & Hall/CRC: The R Series) > R Markdown: The Definitive Guide (Chapman & Hall/CRC: The R Series)
Charlotte Soneson (07:23:34): > Can we get them signed?:wink::nerd_face::party_parrot:
Kevin Rue-Albrecht (07:26:21): > ohhhh do you want to Tweet Hadley about the idea? I’m serious ^^
Kevin Rue-Albrecht (07:28:07): > Btw, I’m too scared to click the “Next” button on that form without having properly filled the “Prize preferences” section. In case I can’t edit it later. > Will need@Federico Marini@Aaron Lunto give me their feedback before I continue through the process
Charlotte Soneson (07:29:06): > Haha, I don’t know. Maybe we can bring it up with “EconomiCurtis” in case:wink:
Kevin Rue-Albrecht (07:31:02): > true
Federico Marini (07:45:37): > Reading up late enough but hey…
Federico Marini (07:45:41): > IS THIS AWESOME:slightly_smiling_face:
Federico Marini (07:50:53): > Regarding how to spend the money: I have already some of the latest books as well from my institute
Federico Marini (07:51:07): > A tshirt and a mug would be just fine
Federico Marini (07:51:27): > We could maybe push for the stickers for everyone. In the end it is an extra advert
Federico Marini (07:55:56): > tbh: the Virtual Lab for Teaching Physiology is also supercool
Kevin Rue-Albrecht (07:55:57): > Can you clarify your last message?
Federico Marini (07:55:58): > but hey
Kevin Rue-Albrecht (07:56:02): > (the stickers)
Federico Marini (07:56:05): > The stickers, yes
Kevin Rue-Albrecht (07:56:17): > do you mean asking for 4x each?
Federico Marini (07:56:20): > I think it is potentially easy to get the full pack of ’em for all
Federico Marini (07:56:43): > I really think it does not make that much of a difference. Plus as I said, it is advertising some of their products:slightly_smiling_face:
Kevin Rue-Albrecht (07:57:38): > Cool. Will do
Charlotte Soneson (07:58:35): > I also don’t think that would be a problem, especially given that it was stated as “all hex stickers we can find”:slightly_smiling_face:
Federico Marini (08:05:18): > exactly
Federico Marini (08:05:27): > Oh god guys I want to hug you all
Federico Marini (08:05:37): > but that would be painful for my back a little:smile:
Charlotte Soneson (08:08:14): > Still not painfree?:flushed:
Federico Marini (08:08:54): > Well, if I limit my movements
Federico Marini (08:09:02): > there’s still some degree of stiffness
Charlotte Soneson (08:09:26): > :-1:
Federico Marini (08:13:13): > Yeah well. I need to give some time
Federico Marini (08:37:23): > If I think that I almost missed the plane to Cambridge 2017 because of the heavy snow that hit germany the day before, then the train was canceled, and I got to the last hidden gate in Frankfurt. Sliding doors ya know:smile:
Federico Marini (09:28:23): > Shall we do a iSEE sticker memorial edition?:smile:
Kevin Rue-Albrecht (09:58:32): > Fun idea! I can’t contribute time designing it, but I’m happy to provide feedback!
Kevin Rue-Albrecht (09:59:28): > On a related note, our next PR should probably put « prize-winning » somewhere in the README file:innocent:
Federico Marini (10:39:06): > Even if we do not design it new, we could just have people contacting us to get a special one
Federico Marini (10:39:25): > I’m not saying signed but at least say for these that did not meet us yet
Federico Marini (10:39:51): > > On a related note, our next PR should probably put « prize-winning » somewhere in the README file:innocent: > humblebragging FTW
Federico Marini (10:40:36): > We could add shiny-contest to the repo topics
Federico Marini (10:40:46): > and put it somewhere in the README as well
Federico Marini (10:40:47): > :slightly_smiling_face:
Kevin Rue-Albrecht (11:57:03): > @Federico Marini@Charlotte Soneson@Aaron LunI think each of us needs to fill this form for themselves (if applicable):https://www.rstudio.com/about/advocate/ - Attachment (RStudio): advocate - RStudio > RStudio Advocate Program Terms Last Revised on: March 2015 RStudio’s Advocate Program (the “Advocate Program”) is designed to encourage select users of RStudio’s software products to promote the use of “R” and RStudio software products in useful and innovative ways. By participating in the Advocate Program, each Participant agrees to these Advocate Program Terms
Kevin Rue-Albrecht (11:57:25): > > Users selected by RStudio (“Participants”) will receive a coupon code (a “Coupon”) and shall be entitled to the benefits under the Advocate Program plan the Participant has been selected or (the “Advocate Program Plan”) for one year from the date you use your Coupon (the “Initial Term”). > > In exchange, the Participant agrees to create and maintain one or more assets (for example, Shiny applications, R Markdown documents, or other similar items) mutually agreed to by RStudio and the Participant (collectively “Assets”) and hereby grant RStudio a worldwide, non-exclusive, royalty-free, fully paid up, transferable right and license for the Term, at RStudio’s option, to access, use, copy, adapt, distribute, promote, and publicly display and perform such Assets
Kevin Rue-Albrecht (13:35:19): > Screenshot of the recent app usage:sweat_smile: - File (PNG): Pasted image at 2019-04-06, 6:35 PM
Charlotte Soneson (13:36:27): > :joy:but 24.58 hours total…:scream:
Kevin Rue-Albrecht (13:38:01): > is that bad?
Charlotte Soneson (13:38:19): > I thought the limit was 25h per month?
Kevin Rue-Albrecht (13:38:59): > ah.. well they clearly must have disabled that for the Shiny contest, because I had 30+ h of runtime in March
Charlotte Soneson (13:39:09): > ah, good:slightly_smiling_face:
Kevin Rue-Albrecht (13:39:23): > Plus, part of the prize is to give us the Basic plan, which is 500h per month
Charlotte Soneson (13:39:40): > Indeed:slightly_smiling_face:
Federico Marini (13:50:21): > FYI: I still was unsuccessful in going by “mr. technically impressive” with the lady here at home
Federico Marini (13:50:32): > maybe one of you has higher success?
Kevin Rue-Albrecht (13:53:30): > hahahah well Anna just clicked in the fact that we won. Fede will probably understand that my French voice/tone doesn’t come across as impactful as the typical Italian tone:stuck_out_tongue:
Charlotte Soneson (13:58:11): > Btw, I can confirm that I have been added to the RStudio PM
Federico Marini (13:59:01): > @Aaron LunYou should try that out as a pickup line tonight
Federico Marini (14:43:48) (in thread): > same here
Aaron Lun (15:13:40): > Okay, what - exactly - do I have to do?
Kevin Rue-Albrecht (15:14:21): > check out the Google Docs, there a tiny list there
Aaron Lun (15:14:31): > okay, going for lunch, will be back to do this.
Kevin Rue-Albrecht (15:15:17): > also, create a RStudio community account, and give me the @handle so that you can be invited to the private thread that the RStudio team has created with me. They’ve added Charlotte and Fede on request today
Aaron Lun (18:25:16): > One of us should put it on the NEWS of the support site.
Kevin Rue-Albrecht (18:26:02): > Well I’m off to sleep now, so perhaps tomorrow.
Aaron Lun (18:27:50): > I can do it.Well I
Aaron Lun (18:28:15): > fuck thisaI asdasdfasddas
Aaron Lun (18:28:17): >
Kevin Rue-Albrecht (18:28:30): > so far, i’ve got 2 things for you@Aaron Lun: > - giving me a T-shirt size and model for you here:https://docs.google.com/document/d/1EtibBy_8kZSPH6CrycA7cEjQV6UEG0ecrJuWac31gXI/edit#heading=h.cxrkxr8ohphg- giving me you RStudio Community (https://community.rstudio.com) handle to add you to the private conversation with the RStudio team - Attachment (community.rstudio.com): RStudio Community > A community for all things R and RStudio - File (Google Docs): iSEE-shiny-contest-prize-process
Aaron Lun (18:28:36): > Goddam lilsehfufsli
Aaron Lun (18:28:57): > i hate this coputer so mcuh
Kevin Rue-Albrecht (18:29:20): > better than having a stroke, my other hypothesis based on your last few messages
Aaron Lun (18:30:04): > I’ll deal with this later today. Gotta write some unit tests for IndexedRelations.
Kevin Rue-Albrecht (18:30:16): > Sounds like fun.
2019-04-07
Aaron Lun (03:26:15): > wizard_of_oz is my Rstudio handle.
Kevin Rue-Albrecht (10:08:19): > @Federico MariniGood that you cleaned up the README file for the contest:sweat_smile: - File (PNG): Pasted image at 2019-04-07, 3:08 PM
Kevin Rue-Albrecht (10:11:26): > @Charlotte Sonesonas i mentionned, it’s probably still a bit early to work simultaneously on GH for the workshop. How about a Google Doc to layout the workflow plan in a more interactive way? (I feel like I’m making GDoc’s for everything these days)
Charlotte Soneson (10:19:01): > Sure, that’s fine with me
Kevin Rue-Albrecht (10:30:36): > You should have got an invitation by email to the GDoc I’ve created.
Charlotte Soneson (10:33:03): > Yep
Charlotte Soneson (16:35:05): > Have you guys triedgource(https://gource.io)? It’s weirdly mesmerising:joy:(saw it here first:https://www.youtube.com/watch?v=C42F5Y8kCU0&feature=youtu.be) - Attachment (YouTube): The development of dynverse
Kevin Rue-Albrecht (17:24:33): > I’ve submitted the “Prize” form to RStudio, let’s see what happens next.
Kevin Rue-Albrecht (17:27:08): > Btw, big thanks to@Federico Marinifor cleaning up the README during the contest. It certainly gave us a better visibility, and I am so glad this is already taken care of at this stage. That said, feel free to edit it if you think it could still be made better, as RStudio mentioned that they may link to it for several years:https://github.com/kevinrue/isee-shiny-contest
Federico Marini (18:09:15): > re: gource, we should give a go with iSEE:slightly_smiling_face:
Federico Marini (18:09:25): > I recall the good old starlogs
Federico Marini (18:10:25): > @Kevin Rue-Albrecht: thank you for picking up in the fight against the memory limits when I was up full with other tasks:slightly_smiling_face:
Federico Marini (18:11:09): > I still can’t really believe it happened. I mean, one thing is liking what one develops, then there’s others appreciating it, but this is a new scale
2019-04-08
Kevin Rue-Albrecht (03:41:34): > Turns out@Federico Mariniit’s at least the second time that I follow up on a fun idea that you brought up, so thanks for that ^^ (the other being voice recognition, which might still have gone unnoticed from the contest as it’s only on devel)
Federico Marini (03:42:10): > Eheh this one was a very low hanging fruit
Kevin Rue-Albrecht (03:47:47): > Well… now we know how much memory the app uses when running persistently ^^ - File (PNG): Pasted image at 2019-04-08, 8:47 AM
Federico Marini (03:48:05): > See how much we could learn:stuck_out_tongue:
Kevin Rue-Albrecht (04:37:47) (in thread): > FYI given the « one on one » name, I’ve mentioned to them that we’re 4 collaborators and so far the most likely scenario is a « 4 in 1 » due to time constraints on their side
Kevin Rue-Albrecht (04:38:29) (in thread): > I highly doubt they would go for 4 separate 1 on 1
Federico Marini (07:18:51): > Jeez if only gource had a soundtrack as well
Kevin Rue-Albrecht (07:42:40): > Yay I can get more likes per day!:stuck_out_tongue:(RStudio community) - File (PNG): Pasted image at 2019-04-08, 12:42 PM
Federico Marini (08:47:40): > wut
Federico Marini (08:47:47): > can’t we by default?
Federico Marini (08:47:48): > :stuck_out_tongue:
Kevin Rue-Albrecht (09:17:47): > I’m surprised too. I didn’t imagine a limit on the number of likes per day xD
Kevin Rue-Albrecht (09:18:00): > Probably a protection against bots
Federico Marini (09:20:29): > That must be it
Kevin Rue-Albrecht (09:23:17): > Unrelated: aside from more urgent “to do” items like the workshop, I can’t help but wonder how to put the year of ShinyApps Basic Plan to good use (e.g. network of ligand-receptor interactions, … )
Kevin Rue-Albrecht (09:25:13): > Aaahrgh stupid me, I know: we could set up your iSEE_instances as live apps!@Federico Marini
Kevin Rue-Albrecht (09:26:33): > Well anyway, there’s time to think
Federico Marini (09:27:52): > That’d be a fitting use case
Federico Marini (09:28:23): > Or, workshop-oriented, deploy an instance of what you’d create live with the users
Kevin Rue-Albrecht (09:28:30): > I’m thinking especially about the additional runtime memory allowance
Federico Marini (09:28:42): > How much would that be?
Kevin Rue-Albrecht (09:29:07): > Up to 8GB of RAM
Kevin Rue-Albrecht (09:29:29): > https://www.shinyapps.io> Pricing > Basic Plan > Performance Boost - Attachment (shinyapps.io): shinyapps.io > shinyapps.io
Federico Marini (09:30:45): > We should put a modal popup with the streaming of gource running on the iSEE repo
Federico Marini (09:30:58): > “Where it all began”
Kevin Rue-Albrecht (09:31:21): > Don’t tempt me:stuck_out_tongue:This one is yours if you want to go ahead with it
Kevin Rue-Albrecht (09:32:41): > Which reminds me, we need to do thatpkgdownat some point. I don’t know where you left it at, but having done it forunisetsandhancock, it turned out to be pretty easy. Just need to make sure that the website is deployed on a side branch.
Kevin Rue-Albrecht (09:34:28): > Neat: the “member” status finally allowed me to invite Aaron to the RStudio thread. I couldn’t before
Federico Marini (09:35:50): > Yes, I still have it in my mind
Federico Marini (09:36:03): > We did find a solution which could avoid cluttering the master right?
Federico Marini (09:36:32): > which still involved Charlotte having to do some tweak or so
Kevin Rue-Albrecht (09:45:11): > Indeed, I’ve successfully avoided cluttering the master branch on both projects. I’ll have to check again how I did it, but now that I think of it, I’m pretty sure I followed a recipe that your found
Kevin Rue-Albrecht (09:49:45): > https://github.com/kevinrue/unisetslooks like it needs: > - an empty_pkgdown.ymlfile at the root (of the master branch) > - the following section in.travis.yml > > before_deploy: Rscript -e 'remotes::install_cran("pkgdown")' > deploy: > provider: script > script: Rscript -e 'pkgdown::deploy_site_github()' > skip_cleanup: true > on: > branch: master > > - the following line in.gitignore(although, that is probably even superfluous if the website is not pushed tomaster) > > docs/ > > - In the GH repo settings (@Charlotte Soneson): > > Your GitHub Pages site is currently being built from the gh-pages branch.
Kevin Rue-Albrecht (15:30:34): > Ow. Seems like they’ve only given the year ofshinyapps.ioto the “lead” submitter of each app. I can kind of understand the limitation, as this prize was given all the way down to honorable mentions: - File (PNG): Pasted image at 2019-04-08, 8:30 PM
Kevin Rue-Albrecht (17:13:40): > :sweat_smile:I had to put my card details along with the coupon for the free year, glad to see the discount on the billing info ! - File (PNG): Pasted image at 2019-04-08, 10:13 PM
2019-04-09
Aaron Lun (00:45:08): > Do I have to do anything?
Aaron Lun (00:45:38): > Also there’s a pile of bugs and tweaks that I spammed onto the repo. I fixed most of them, there’s a couple outstanding that we should deal with.
Kevin Rue-Albrecht (01:32:38): > Nothing to do for the prizes right now@Aaron LunAn update I got overnight on a separate thread that Curtis set up to message all winners at once: > > When should I expect swag? - I just did a quick inventory of our shirts, books, stickets etc, and we realize we need to re-order a lot of this. So we’re going to send this to y’all in mid-to-late May.
Federico Marini (12:22:23): > (to keep in mind: as of now TravisCI for R is broken because ofremotes)
Kevin Rue-Albrecht (13:19:41): > Yup. I should have said, I saw that when I checked out Aaron’s recent commits. Travis fails but I don’t see anything we can do about it for now
Kevin Rue-Albrecht (14:08:16): > lol 93h - File (PNG): Pasted image at 2019-04-09, 7:08 PM
Kevin Rue-Albrecht (14:33:18): > Please try and update the NEWS before you merge the PR
Kevin Rue-Albrecht (15:20:15): > It looks likehttps://github.com/csoneson/iSEE/commit/02226f0f59d77f4aa43b43ac996707c9eae0b4d6got us past theremotesissue. > > install_* functions now support the R_REMOTES_UPGRADE environment variable, to set the default for the upgrade argument. See README for details (@kevinushey, #240). > https://github.com/r-lib/remotes/blob/master/NEWS.md
Federico Marini (15:20:41): > Is it required to be set this way?
Federico Marini (15:21:07): > Or was it just a safe way to force-reset the cache?
Kevin Rue-Albrecht (15:23:58): > Well I see some activity on theremotesGitHub, but in the meantime yes it’s a safe way to set the argument toremotes::install_bioc(upgrade = )
Kevin Rue-Albrecht (15:25:30): > I guess I’ll need to push that to release too, although it might only be for our own benefit on Travis. Depending on the bio build machine
Kevin Rue-Albrecht (15:26:39): > Anyway. Done. I don’t like leaving Travis failing and unhappy.
Kevin Rue-Albrecht (15:31:24): > Quick clarification: not force-reset, just force-upgrade (I hope).
Kevin Rue-Albrecht (15:34:06): > If it turns out that the optionsdoesforce the cache everytime, we’ll just remove the line again whenremotesis stable again
Kevin Rue-Albrecht (16:41:51): > So.. problem solved forprerelease, butRELEASE_3_8failed somewhere in the unit tests after ~40min with a weird string ofcat()statements that I’ve never seen before. I’ve just relaunched the build to see if it’s a timeout issue
Kevin Rue-Albrecht (17:27:52): > Hm. Same weird error:https://travis-ci.org/csoneson/iSEE/builds/517948018?utm_source=github_status&utm_medium=notification
Kevin Rue-Albrecht (17:28:58): > Though it might be due to a couple of unit tests: > > OK: 1054 SKIPPED: 3 FAILED: 2 > 1. Error: .make_customDataPlot responds to a transmitting column brush receiver (@test_custom.R#113) > 2. Error: .make_customDataPlot responds to a transmitting column brush receiver (@test_custom.R#146) >
Kevin Rue-Albrecht (17:29:14): > …. do we really have 1054 unit tests?! … sorry 1059 !?!
Kevin Rue-Albrecht (17:40:29): > problem solved, it’s a unit test that Charlotte solved onmasterlast month, related toscater::runPCA, which we haven’t ported torelease
Kevin Rue-Albrecht (18:41:03): > I’ve pushed ****RELEASE_3_8******** upstream**
2019-04-10
Aaron Lun (01:42:24): > OOnly bugfixes from this point on.
Aaron Lun (01:42:32): > The other shit will just have to wait.
Kevin Rue-Albrecht (03:16:26): > That said, I’ve just experimented with the panel width/height controls, and I think it falls within the bugfixes, considering the stroke the app is having when updating panel width
Kevin Rue-Albrecht (03:16:53): > Please checkout branchpanel_organizationand if you’re happy with my fix, continue development there
Kevin Rue-Albrecht (03:21:58): > I’m not bothering to disable enable the action button when settings match the current ones. Clicking the button when the settings match the current ones does not do anything.
Charlotte Soneson (06:20:04): > @Kevin Rue-Albrecht- just playing with the panel width/height controls. This is what I was doing: > (1) Change the width of all panels (nothing happened since I didn’t press “Apply settings”) > (2) Change the order of two panels. This immediately moved the panels, and the width modifications from step (1) were lost. > This seems unintentional.
Kevin Rue-Albrecht (06:20:55): > Hm.. Riiight, well I didn’t consider this sequence of actions
Kevin Rue-Albrecht (06:22:07): > Can we just train users use the inputs sensibly? Or this a sequence of actions you’d expect to do yourself?
Charlotte Soneson (06:22:59): > Well - I didn’t intentionally try to break it…
Charlotte Soneson (06:23:10): > This is what I would have done
Charlotte Soneson (06:23:39): > I guess my expectation was that “Apply settings” would apply all the changes that you had done (i.e., that changing the order of the panels would also not take effect immediately)
Kevin Rue-Albrecht (06:23:39): > Ok then to answer your actual question: the inputs widgets are redrawn and populated with their value inrObjects, which is not updated unless the button is clicked, hence the loss of information
Kevin Rue-Albrecht (06:24:58): > In that case, it’s all just about moving the code currently triggered by theorder_panelobserver to theupdate_uiobserver
Kevin Rue-Albrecht (06:29:37): > It was early morning when I made the decision to only regulate the height and width (see commenthttps://github.com/csoneson/iSEE/issues/283). I thought it was nice to immediately update the order of the width/height widgets to match the selectize, but now I do agree with you that everything should be under the control of the action button. We can leave it to the user to: > 1. reorder panels in the selectize (at this point, the order of panels in the selectize and width/height inputs will not match) > 2. apply the changes to reorder both panels and height/width inputs to match the selectize > 3. change width/height ad libitum > 4. apply the second round of changes
Kevin Rue-Albrecht (06:33:20): > That said, looking at the code now, I realize that the selectize can also add and remove panels, which also means that for a while, there may be panels: > - “just added”: present in the selectize but without height/width controls > - “just remove”: absent in the selectize but with height/width controls > Again, at that point I’d argue that it’s up to the users to: > 1. add/remove panels > 2. apply the changes > 3. change height and width > 4. apply the changes > Thoughts?
Charlotte Soneson (06:35:41): > Yeah, I guess that might be the most reasonable solution.
Kevin Rue-Albrecht (06:39:32): > I’m pushing now. Let me know what you think, but I won’t be able to do more before this evening
Kevin Rue-Albrecht (06:41:54): > That said, I’ve spotted another “bug” in the process. The selectize is initially listing the panel full names (e.g. “reduced dimension plot 1”), but the additional choices of panels to add are the short identifier (e.g. “redDimPlot2”). Can you see that too? I’ll have to check whether that was the case before, or if i introduced a bug with my last couple of commits.
Charlotte Soneson (06:42:25): > Yes, that was the case already before actually.
Kevin Rue-Albrecht (06:43:06): > Yup. I just ran the code from theprereleasebranch and saw that too.
Kevin Rue-Albrecht (06:43:25): > Well, as I said, I can only look at it tonight if no one gets to it first
Charlotte Soneson (06:57:23): > Yes, this panel organization looks good to me.
Federico Marini (07:08:20): > :thumbsup:
Federico Marini (09:13:28): > Quickly checking in with you:
Federico Marini (09:14:24): > the app is crashing when I am adding new panels, starting fromexample(iSEE,ask=F)
Federico Marini (09:14:47): > > iSEE(sce) > 'Custom data plot 1' in 'initialPanels' is not available (maximum ID is 0) > > Listening on[http://127.0.0.1:4863](http://127.0.0.1:4863)Warning: Error in <-: replacement has length zero > [No stack trace available] >
Federico Marini (09:16:39): > Removing panels is fine, but also switching them crashes it
Kevin Rue-Albrecht (09:26:57): > Uh. Can you test if that’s related to my recent commits? (checkout an earlier commit, or theprereleasebranch)
Federico Marini (09:32:36): > Inprereleaseit does not kill the app when I just move around the items
Kevin Rue-Albrecht (09:33:29): > eh… you were saying > > adding new panels
Federico Marini (09:38:34): > Adding new panels OR reordering did kill the app
Federico Marini (09:38:52): > Sorry, that came a couple of lines later
Federico Marini (09:39:11): > Adding works fine. Clunky slow, but works.
Kevin Rue-Albrecht (09:57:53): > I know what the issue is. I think
Kevin Rue-Albrecht (10:07:17): > Hm… Not sure. I won’t fix it that fast so I’ll have to get back to other business right now, and either deal with it tonight or see if it’s obvious to Aaron. I think it’s to do with thelocal({...})which was previously outside an observer, and now is inside one. Reactive values are a pain to track, and I wonder whether it’s about putting anisolate({...})somewhere
Kevin Rue-Albrecht (10:14:40): > When I try to addredDimPlot2, it crashes because it cannot find the panel width for that panel, which makes sense because this panel isn’t initialized yet. > > Listening on[http://127.0.0.1:1234](http://127.0.0.1:1234)[1] "redDimPlot1_" > [1] "redDimPlot1_PanelWidth" > [1] "redDimPlot2_" > [1] "redDimPlot2_PanelWidth" > integer(0) > Warning: Error in <-: replacement has length zero > 78: eval [/Users/kralbrecht/git/iSEE/R/iSEE-main.R#566] > 77: eval > 72: observeEventHandler [/Users/kralbrecht/git/iSEE/R/iSEE-main.R#552] > 1: shiny::runApp > > > > Adding works fine. Clunky slow, but works. > @Federico MariniI think you meant “removing” works fine. Which it does for me
Kevin Rue-Albrecht (10:17:02): > So it all makes sense now: because adding/removing/moving panels and resizing are all triggered by the same button, when new panels are “to be added”, they don’t exist yet, and therefore they don’t have height/width inputs in the modal, so we need to make sure that the UI update only adds them, and doesn’t look for their height/width input
Federico Marini (10:18:13): > I meant adding, but in theprerelease, i.e. without the button enhancement
Kevin Rue-Albrecht (10:19:09): > ah right
Kevin Rue-Albrecht (10:19:40): > well, I’m just testing now, but I think I fixed it by just moving some code around > > Update panel width/heightbeforeadding new active panels, as those don’t have height/width inputs
Kevin Rue-Albrecht (10:22:13): > pushing now, check it out when you can
Federico Marini (10:23:00): > I can now
Federico Marini (10:23:04): > lemme check it out
Federico Marini (10:24:07): > I threw in my 2 cents on your backbone organization for the workshop
Federico Marini (10:24:17): > the rest is already very well structured
Federico Marini (10:24:50): > In the end, most of the workshop will be rather live actions more than written material
Federico Marini (10:25:16): > What we could do is anyway give a bulleted point of the list of aspects we will touch upon
Kevin Rue-Albrecht (10:26:39): > True, but the written part is what stays and even propagates, so it should provide some example data, raw and processed, and then how to configure apps to get the most out of it and share it efficiently
Federico Marini (10:27:44): > Works for me. I still get an error - silently, in the console
Federico Marini (10:27:58): > Warning: Error in pngfun: invalid quartz() device size
Federico Marini (10:29:29): > So probably there is still one instant where that kicks in
Kevin Rue-Albrecht (10:30:54): > I saw that too, but only the first time I added a panel. What happens if you add more than one panel in the same session?
Federico Marini (10:31:09): > This occurs in RStudio, as well as in the console
Federico Marini (10:31:20): > once for every added panel, de facto
Kevin Rue-Albrecht (10:31:25): > uh
Federico Marini (10:31:45): > (well, plots)
Federico Marini (10:31:56): > (tables do not raise this)
Kevin Rue-Albrecht (11:17:58): > Do you have time to add fewprintto see whether it’s an issue about the initialization of width and height?
Kevin Rue-Albrecht (11:37:08): > Here’s a pretty straightforward MWE that has the same issue:https://github.com/rstudio/shiny/issues/2136So perhaps it’s not even anything I did wrong, although I don’t know why we would have dodged the issue until now
Federico Marini (11:59:24): > oh, pickng up too late maybe
Federico Marini (11:59:53): > I think it is something we did trigger new. Like a plot being rendered when it is not there yet the place for that
Federico Marini (12:00:24) (in thread): > Sorry, could not check it out right away, I am wrapping up a couple of consulting projects:disappointed:
Kevin Rue-Albrecht (13:12:29): > Must be. In our case it’s not random like they describe. It specifically showed up with my commit
Kevin Rue-Albrecht (13:16:43): > I’ll look into it later this evening. In the meantime is it fair to say that this warning is still worth the overall behaviour of the modal? I.e. one action button to rule them all?
Federico Marini (14:45:42): > I am sure it is a feature which is worth having. Point is, it is a “cold blooded error” instead of just a warning
Federico Marini (14:47:25): > to me it sounds it is trying to render an object which is not created fully yet
Federico Marini (14:50:13): > can this point to a solution?
Federico Marini (14:50:14): > https://stackoverflow.com/questions/41622795/shiny-only-calculating-when-the-user-is-looking-at-the-output - Attachment (Stack Overflow): Shiny only calculating when the user is looking at the output > I’m having a problem with Shiny and I don’t know if it’s a bug or I’m missing something. I tried to find a solution but I can’t seem to find references to this problem online. I have a function th…
Kevin Rue-Albrecht (15:50:02): > Not sure… I’m just stepping back in the commit history, and - while I don’t the answer yet - a new piece of information is thatWarning: Error in pngfun: invalid quartz() device sizealready happens at commitff020b1b: > - the selectize was not under the control of the button > - the height / width were
Kevin Rue-Albrecht (15:51:39): > … which basically means that it was already the case onprerelease(before I touched anything) … checking now
Kevin Rue-Albrecht (15:54:33): > Confirmed. It’s already onprerelease.
Kevin Rue-Albrecht (15:59:06): > Well, I’m just putting it out there: y’know we’ve got a call “Half-an-hour one-on-one with a representative from the RStudio Shiny team for Q&A and feedback”… We’ve got some feedback…:stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (16:01:04): > I’ve rolled back all the way tomasterand I still see the warning. I’m genuinely starting to think it’s on the Shiny side
Kevin Rue-Albrecht (16:05:10): > Oh wait.28 packages out-of-dateincludingshiny
Kevin Rue-Albrecht (16:08:37): > On a separate note, it’d be awesome if we could prepare a new section in the NEWS file like they do in the Shiny package after a version bump:https://github.com/rstudio/shiny/commit/f6e8e645f25a6de0e2c739f3d207c312886348bd
Kevin Rue-Albrecht (16:41:25): > From asking a fellow user@Rob Amezquita, the warning also happens on for version 1.3.8 (the current “devel” version on bioc). Either we haven’t tested the addition of panels in a long time and it could be any update we introduced in that time, or something got updated in one of our dependencies. > In the latter case, we either sit and wait for other bug reports in the community, or we chase it for however long it takes. Personally I’m done for the day.
Kevin Rue-Albrecht (16:42:59): > Just to conclude:BiocManager::valid()=TRUE, shiny version is 1.3.0 and I still get the warning.
Federico Marini (16:44:23): > Damn.
Federico Marini (16:44:47): > As a positive thing still, the idea with the button “would just work”
Federico Marini (16:45:23): > Other thing we might want to do before the new release: there’s the option to display our orcid ids in the package page
Federico Marini (16:45:49): > It is just acommentin the person info (DESCRIPTION)
Kevin Rue-Albrecht (16:45:54): > ahhhh I want that!
Kevin Rue-Albrecht (16:46:25): > yup I’ve done it here:https://github.com/kevinrue/unisets/blob/master/DESCRIPTION
Federico Marini (16:47:17): > We could piggyback that in the prerelease branch
Kevin Rue-Albrecht (16:48:45): > so just to say what need not saying: we’re good to merge my side-side branch back toprerelease, given that I haven’t broken anything more than it already was?
Kevin Rue-Albrecht (16:49:11): > if you give me your ORCID I kill two of them with 1 commit
Kevin Rue-Albrecht (16:51:55): > … too late now:stuck_out_tongue:
Federico Marini (16:56:54): > Uh. And another nice item.
Kevin Rue-Albrecht (16:57:19): > this ORCID feature is especially nice with pkgdown
Federico Marini (16:57:25): > We could just add > > output: > BiocStyle::html_document: > toc_float: true > code_download: true >
Kevin Rue-Albrecht (16:57:47): > oh yeah - well this one’s up to you
Federico Marini (16:58:15): > with the extracode downloadthere’s the Rmd to be taken directly from the vignette. We do not have that code-intensive vignettes, tbh
Kevin Rue-Albrecht (17:01:30): > Ok - I couldn’t resist. I’ve also just tried thereleaseversion of iSEE and I get the same warning message. - File (PNG): Pasted image at 2019-04-10, 10:01 PM
Kevin Rue-Albrecht (17:02:48): > I don’t think I can (or want) to track it further than that. Now I know that I’ll pester the poor RStudio Shiny representative about lassos andthat
Kevin Rue-Albrecht (17:03:31): > (for lack anything better to say… I don’t suppose they’re hiring, are they?:money_mouth_face:)
Federico Marini (17:05:30): > I think they could consider us for whatevs. Otherwise I would not see the point of asking whether we are available for consultancy or so
Federico Marini (17:05:45): > (I think I saw that in the form, right?)
Kevin Rue-Albrecht (17:05:55): > yup. training or consulting
Kevin Rue-Albrecht (17:06:51): > well.. considering how I’ve already done the best part of my contributions to iSEE outside office hours… I still have another 8h left for a 3rd job I guess:expressionless:
Federico Marini (17:10:18): > :smile:
Federico Marini (17:22:58): > FWIW: I installed shiny 1.2.0 and tried the branches: > - panel_organization: error pops up when creating > - master: same > - release (iSEE 1.2.3): no error when I create the new panel
Kevin Rue-Albrecht (17:23:26): > “error”.. you mean warning?
Federico Marini (17:24:33): > error masked to warning
Federico Marini (17:24:45): > that pngfun thing
Kevin Rue-Albrecht (17:24:51): > Oh sorry yes
Federico Marini (17:25:43): > So: it’s a nasty one. But we can talk to the deus ex machina soon:slightly_smiling_face:
Kevin Rue-Albrecht (17:27:38): > ok, so if I get it right, we didn’t spot the issue before because “old” 1.2.0 shiny and “release” iSEE were happy together?:musical_note:
Federico Marini (17:28:39): > kinda
Federico Marini (17:29:12): > at least, after my 2-cents-investigation I could spot this
Federico Marini (17:33:03): > thing is, the pngfun function in shiny seems untouched since some time…
Federico Marini (17:33:09): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (17:45:44): > well i got there too, but you’ll see thatpngfunif like ourFUN0. It’s not a function itself, it’s a variable that switch between different functions depending on whichcapabilities()are available
Kevin Rue-Albrecht (17:46:21): > > shiny:::startPNG > function (filename, width, height, res, ...) > { > if (capabilities("aqua")) { > pngfun <- grDevices::png > } > else if ((getOption("shiny.usecairo") %OR% TRUE) && nchar(system.file(package = "Cairo"))) { > pngfun <- Cairo::CairoPNG > } > else { > pngfun <- grDevices::png > } > pngfun(filename = filename, width = width, height = height, > res = res, ...) > op <- graphics::par(mar = rep(0, 4)) > tryCatch(graphics::plot.new(), finally = graphics::par(op)) > grDevices::dev.cur() > } >
Kevin Rue-Albrecht (17:46:46): > that said,grDeviceshasn’t changed since 2016 on GH and Cairo since 2015
2019-04-11
Kevin Rue-Albrecht (04:34:21): > https://kevinrue.shinyapps.io/plot_coverage/
Federico Marini (04:37:06): > Loads but when I click it disconnects from server?
Kevin Rue-Albrecht (04:37:56): > uh worked for me
Kevin Rue-Albrecht (04:38:00): > tried reloading?
Federico Marini (04:38:42): > yup
Federico Marini (04:38:58): > disconnects probably just after a few moments
Kevin Rue-Albrecht (04:39:37): > funny, because i could follow the tour the first time i launched the app, now i’m like you
Kevin Rue-Albrecht (04:39:46): > maybe something about giving the app more resources
Federico Marini (04:40:41): > probably
Federico Marini (04:40:56): > if you fiddle with that: do you want to give a specific title upon launching?
Kevin Rue-Albrecht (05:44:07): > Sure, but I’ll do that with calm later
Kevin Rue-Albrecht (05:44:31): > it works again. I think the app only handle 1 concurrent user - File (PNG): Pasted image at 2019-04-11, 10:44 AM
Federico Marini (06:56:55): > Upon click it does crash again - unless you are also trying out things?
Kevin Rue-Albrecht (07:20:29): > Nope. I’ve been in talks for the last 2h
Kevin Rue-Albrecht (07:22:07): > This is really not a top priority right now, so I won’t play with it much the next few days. It’s just the result of trying to deploy apps last night, in parallel to our debug session
Federico Marini (08:26:13): > ok:slightly_smiling_face:
Federico Marini (08:26:54): > (it makes sense still to have such working demos: it is a perfect companion link to each case in the iSEE_custom repo)
Kevin Rue-Albrecht (08:48:52): > Yup…. I just need to find the time within the next year, as I don’t particularly want to spend $440 to keep the Basic plan running after that
Federico Marini (08:49:57): > Well thing is, we can always host the apps on the server I setup
Federico Marini (08:50:09): > it is good you have “your own cloud” there
Kevin Rue-Albrecht (08:50:35): > It’s definitely good for testing
Kevin Rue-Albrecht (08:51:44): > Speaking of testing, I think Aaron wasn’t particularly interested in testing mypanel_orgbranch yesterday, and he kept working on theprerelease. We should mergepanel_orgintoprereleaseASAP, before it gets ugly
Kevin Rue-Albrecht (15:10:05): > ISEE 1.2.4 (release) is fixed
Kevin Rue-Albrecht (15:10:08): > http://bioconductor.org/checkResults/release/bioc-LATEST/iSEE/
Kevin Rue-Albrecht (17:15:32): > @Aaron Lunwanna takeprereleasefor another spin after I merged the panel organization update?
Kevin Rue-Albrecht (17:17:11): > Let’s remember to bump up the version, date and NEWS before merging to master. I’m happy to push upstream when i receive the order, because I don’t have a good sense right now if anything else needs fixing. As always, if we merge “too soon”, it doesn’t cost anything to open another branch and PR again
2019-04-12
Aaron Lun (00:46:56): > I am so tired.
Aaron Lun (00:47:01): > So, so tired.
Aaron Lun (00:49:43): > Anyway, I forgot to actually make the review, so I’ve done that.
Aaron Lun (00:51:27): > All of the problems about updating the width/height and changing the plot order can be resolved by having a variable in thepObjectsenvironment that acts as a side-memory just for the modal.
Aaron Lun (00:53:48): > It gets updated with the latest parameters whent eh modal opens; it gets filled in with the user modifications from fiddling with things; and it gets applied back to overwriteinitialPanelswhen you press the apply button. Incidentally this would also provide a source of information that can be used to update the width/height UI when you add/remove panels.
Kevin Rue-Albrecht (02:39:33): > Thanks for the review. Go get some sleep. Perhaps some annual/sick leave too. I think it’s fair to say that you work a lot.
Aaron Lun (03:05:49): > Just got my new laptop. Not getting any sleep tonite.
Kevin Rue-Albrecht (03:07:16): > already installed R and Bioc?
Aaron Lun (03:11:52): > yep
Aaron Lun (03:11:59): > first thing
Kevin Rue-Albrecht (03:24:53): > Well, I’ve got a new thing that depends on iSEE:https://github.com/kevinrue/magick-profile
Kevin Rue-Albrecht (03:25:15): > Not sure how John did his own
Aaron Lun (03:27:45): > okay, going to sleep now.
Aaron Lun (03:27:51): > Someone stick my orcid on there to join the party.
Federico Marini (05:44:25): > done
Federico Marini (07:37:07): > Ops, sorry, wrong branch - I missed the merging of panel_orga
Federico Marini (07:37:20): > re-merging forprereleasenow
Kevin Rue-Albrecht (07:44:32): > Yeah sorry I didn’t memntion, but I merged to grab Aaron’s attention yesterday
Federico Marini (07:48:38): > :sunglasses:
Kevin Rue-Albrecht (18:26:23): > > All of the problems about updating the width/height and changing the plot order can be resolved by having a variable in thepObjectsenvironment that acts as a side-memory just for the modal. > > It gets updated with the latest parameters whent eh modal opens; it gets filled in with the user modifications from fiddling with things; and it gets applied back to overwriteinitialPanelswhen you press the apply button. Incidentally this would also provide a source of information that can be used to update the width/height UI when you add/remove panels. > I’m looking at the code right now, and to me it doesn’t look like it needspObjectsto behave in this way > > it gets filled in with the user modifications from fiddling with things > input$...get updated but don’t apply until the button is clicked and “flushes” the changes
Kevin Rue-Albrecht (18:27:41): > … or do we want the modal to remember the changes that were not yet applied?
Aaron Lun (18:27:46): > That’s right.
Aaron Lun (18:27:59): > No, hold on.
Aaron Lun (18:28:30): > I had been thinking about usinginput, but it wouldn’t work for some reason… why was that.
Aaron Lun (18:28:59): > Hm. Can’t remember. Maybe it does work.
Kevin Rue-Albrecht (18:29:31): > As in, if the user makes changes to the inputs and closes the modal without applying them… I don’t think we want to save that, do we really?
Aaron Lun (18:29:35): > No,
Aaron Lun (18:29:38): > no you don’t.
Kevin Rue-Albrecht (18:29:44): > Ah. We agree then
Aaron Lun (18:29:55): > Can’t remember why usinginput$wouldn’t work.
Aaron Lun (18:30:20): > Depends on what exactly happens during UI re-rendering, with respect to updatinginput.
Kevin Rue-Albrecht (18:31:04): > Uh.. why would UI re-rendering affectinput? It’s the other way around usually
Aaron Lun (18:31:25): > Because you re-initialize the widgets.
Aaron Lun (18:31:42): > It should be okay if the widgets are re-initialized at the values they were previously on…
Kevin Rue-Albrecht (18:31:45): > As long as we reinitialize them with their last know value
Aaron Lun (18:31:56): > Well, that’s the thing.
Aaron Lun (18:32:12): > What is the last known value - inrObjects$initialPanels, or whatever they last had set?
Aaron Lun (18:32:34): > The latter is important if you’re fiddling with the order and triggering a re-rendering but do not want to lose the new height/width settings.
Kevin Rue-Albrecht (18:33:26): > I think > > output$panelParams <- renderUI({ > .panel_organization(rObjects$active_panels) > }) > > takes cares of what you’re saying
Kevin Rue-Albrecht (18:34:10): > Namely,.panel_organizationtakes care of regenerating the modal with the values updated inrObjects
Kevin Rue-Albrecht (18:34:28): > which (rObjects), is updated when the button is clicked to apply the changes
Kevin Rue-Albrecht (18:35:33): > Anyway, having played with the app, it quacks like a duck etc.
Aaron Lun (18:35:40): > Stop, stop.
Aaron Lun (18:36:04): > I’m talking about the UI rerendering of the width/height parameter boxeswithinthe modal once you change the selectize.
Aaron Lun (18:36:08): > But before you apply anything.
Aaron Lun (18:36:57): > active_panelshasn’t been updated at this point, so the rerendering will reset the width/heigth choices to whatever was there previously.
Aaron Lun (18:37:19): > Users who set the width/height first and then modify the selectize before pressing the apply would see their width/height choices disappear.
Kevin Rue-Albrecht (18:37:32): > Pause there.
Kevin Rue-Albrecht (18:40:29): > I think the answer here is that noweverythingin the modal is under the control of the button. So users can change width/height (without applying yet) and modify the selectize (doesn’t do anything either). Then they click the button and all the width/height/add/remove/reorder happens simultaneously in a single observeEvent. So nothing is lost. > Then obviously, if they added panels, the associated width/height inputs will only be visible after this first round of modal + UI update
Aaron Lun (18:40:50): > Okay, I think I know what is not being communicated here.
Aaron Lun (18:41:01): > I want the parameter UI to change upon selectize.
Aaron Lun (18:41:19): > Especially if the user adds a new parameter box, they should be able to edit the width/heigth of that box straight away.
Kevin Rue-Albrecht (18:41:35): > Ah. I did that first, and Charlotte had a point that it wasn’t optimal
Aaron Lun (18:41:40): > Why not?
Kevin Rue-Albrecht (18:41:48): > Checking now…
Kevin Rue-Albrecht (18:43:00): - Attachment: Attachment > I guess my expectation was that “Apply settings” would apply all the changes that you had done (i.e., that changing the order of the panels would also not take effect immediately)
Kevin Rue-Albrecht (18:43:30): > Check also the messages around this one up and down
Aaron Lun (18:44:40): > Yeah, that’s what my suggestion tries to fix.
Aaron Lun (18:45:16): > If you have a working reactive separate fromactive_panels, everything should just work.
Kevin Rue-Albrecht (18:47:09): > Well, for now I went with the pragmatic solution that gave a sensible compromise rapidly. I can’t promise time, but if we agree on a clear app behaviour I can try to spare an hour somewhere. I really gotta go to sleep now though
Aaron Lun (18:47:53): > I will do this once R reinstalls.
Kevin Rue-Albrecht (18:50:35): > Ok. thanks. > I understand and agree that the selectize immediately updating the modal UI immediately makes sense (note that it also has to update the UI panels… otherwise we’d have to implement a mechanism to update the UI panels when the modal closes). > My motivation for putting everything under the control of the button was to reduces the number of UI updates, especially when adding multiple panels in a go > It gets a bit annoying to add a panel, wait for UI update, add a panel, …
Aaron Lun (22:03:56): > god this new laptop is so fast.
2019-04-13
Aaron Lun (01:32:48): > See updates@Kevin Rue-Albrecht@Charlotte Sonesonand merge if happy.
Aaron Lun (02:05:10): > I’m properly moving house tomorrow so just do it if it looks good.
Aaron Lun (04:13:32): > OMG this laptop is so slick
Aaron Lun (04:13:41): > IT IS THE MOST BEAUTIFUL THING I HAVE EVER OWNED.
Kevin Rue-Albrecht (05:00:22): > xD care to share some details/specs about it?
Kevin Rue-Albrecht (05:37:11): > Wow the new behavior is so reactive!
Kevin Rue-Albrecht (05:38:49): > Especially because the UI panels are not affected until the button is clicked. And I can see now how you meant that the parameters are set to the current state every time the modal is opened, i.e. “unsaved changes” (changes done without clicking the button) are forgotten, which is fine
Charlotte Soneson (08:08:45): > Yes, I agree that the new behaviour works really nicely!
Kevin Rue-Albrecht (10:12:00): > @Federico Mariniopinion? > I’ll start scrolling through the recent commit to update the NEWS in the next couple of hours, then I’ll bump the version and push I think
Federico Marini (10:12:29): > just got back from a run
Federico Marini (10:12:42): > I’ll have a shower first and a look just later
Kevin Rue-Albrecht (10:13:07): > :+1:
Federico Marini (10:21:21): > all good in the hood from my side:slightly_smiling_face:
Federico Marini (10:21:55): > For the fans of DESeq2, I wrote up a small gist for plugging in the input+results into iSEE:https://gist.github.com/federicomarini/4a543eebc7e7091d9169111f76d59de1
Kevin Rue-Albrecht (10:21:59): > I didn’t tried my hardest to break it, but from what I can see it holds fine to add/move/remove
Federico Marini (10:22:15): > same here:slightly_smiling_face:
Kevin Rue-Albrecht (10:23:24): > Neat “plugin”
Charlotte Soneson (10:24:32): > @Federico MariniNice DESeq2 plugin:slightly_smiling_face:is it intentional to get rid of the originalrowDataof thedds?
Kevin Rue-Albrecht (10:24:47): > ah right, you should probablycbind
Federico Marini (10:24:50): > Not really
Federico Marini (10:24:57): > good catch
Kevin Rue-Albrecht (10:25:06): > or better, add it as a nested DF e.g. “DESeq2” column in rowData
Federico Marini (10:25:07): > It is updatable of course
Charlotte Soneson (10:25:39): > Yes, that’s what I typically do too - add the whole df as a column in therowData, with the contrast as the name.
Federico Marini (10:25:53): > I was talking with Mike yesterday and he asked me whether we wanted a plug for iSEE
Federico Marini (10:26:11): > in the DESeq2 vignette
Federico Marini (10:26:38): > with the “why not” thought in mind, that was the quickest I came up with:slightly_smiling_face:
Federico Marini (10:30:36) (in thread): > is there an accessor for the contrast name?
Federico Marini (10:31:27): > If it is a nested DF, can iSEE still pick the names?
Charlotte Soneson (10:32:03): > Yes, they will be expanded
Kevin Rue-Albrecht (10:32:06): > It unwraps them yes
Charlotte Soneson (10:34:27) (in thread): > Good question:slightly_smiling_face:
Charlotte Soneson (10:36:57) (in thread): > You can get the text version:https://github.com/mikelove/DESeq2/blob/a263198a47bc677da443da92da0becee9df366fa/R/methods.R#L707-L711
Charlotte Soneson (10:37:44) (in thread): - File (PNG): Screen Shot 2019-04-13 at 16.37.28.png
Federico Marini (10:43:45): > Cool:slightly_smiling_face:Sorry for the naive question
Federico Marini (10:44:25) (in thread): > that’d be something to store in a slot, d’you agree?
Kevin Rue-Albrecht (10:45:01): > the syntax to create a new nested column can be a bit weird, I think it’s either[[or wrapping the new column inI()(capital i)
Federico Marini (10:54:24) (in thread): > Ok I got this
Federico Marini (10:54:26) (in thread): > sub(".**p-value: (.**)","\\1",mcols(res, use.names=TRUE)["pvalue","description"])
Federico Marini (10:55:05): > I can update it later
Federico Marini (10:55:44): > your call would be to keep the dds rowdata, after the nested res inserted
Kevin Rue-Albrecht (11:02:29): > Yup. I would say that the object (dds,sce, ..) always takes priority, and that results and other metadata are added as nested columns. Obviously, I’m not considering complex scenarios, but that should be a reasonable strategy for small plugins like this
Federico Marini (11:05:13): > I was thinking for the simplicity of seeing the results-related stuff in the top positions
Kevin Rue-Albrecht (11:07:03): > I can see the motivation in that. Well, that’s up for personal taste I suppose. Maybe your plugin can take options likenewMetadataFirst=TRUE
Kevin Rue-Albrecht (11:07:20): > Btw: it’s calming down a bit - File (PNG): Pasted image at 2019-04-13, 4:07 PM
Charlotte Soneson (11:20:22): > I don’t think you need anything fancy to add the nested DF - this is what I did in a course this week: - File (PNG): Screen Shot 2019-04-13 at 17.19.33.png
Charlotte Soneson (11:20:58): > And actually, there shouldn’t even be a need to convert toSummarizedExperiment, since theDESeqDataSetalready extendsSummarizedExperiment
Charlotte Soneson (11:22:31): > Sorry, I see now that you make aSingleCellExperiment. However, there are noreducedDims in there, so maybe the easiest would be to just return aSummarizedExperiment- then the user could convert toSingleCellExperimentand add dim.reds themselves (to avoid depending on another package) (?)
Charlotte Soneson (11:24:15) (in thread): > Looks good (I think:slightly_smiling_face:- my regex knowledge typically requires an example to see what’s happening:stuck_out_tongue:)
Kevin Rue-Albrecht (11:43:38): > If you didn’t see yet, I’v pushed the NEWS update, date and version bump (1.3.9). I think I’ll push upstream before 5pm EST so that it makes the nightly build and we get an idea of how it behaves on bioc-build
Kevin Rue-Albrecht (11:43:58): > The sooner we know, the sooner we can fix bad surprises
Charlotte Soneson (11:44:51): > Sounds reasonable
Kevin Rue-Albrecht (11:49:38): > merged
Kevin Rue-Albrecht (11:50:09): > btw,@Charlotte Sonesonwhat’s the github-pages settings for iSEE? nothing?
Charlotte Soneson (11:51:47): - File (PNG): Screen Shot 2019-04-13 at 17.51.29.png
Kevin Rue-Albrecht (11:53:04): > can you set it togh-pages branch?
Charlotte Soneson (11:53:38): > Nope, onlymasterormaster/docs - File (PNG): Screen Shot 2019-04-13 at 17.53.25.png
Kevin Rue-Albrecht (11:54:59): > mhh i kind of remember this mismatch with the docs, but somehow forunisetsI got this - File (PNG): Pasted image at 2019-04-13, 4:54 PM
Kevin Rue-Albrecht (11:55:33): > Ah.. you know what, maybe it enables the choice only once the branch actually exists
Kevin Rue-Albrecht (11:56:42): > (as right now you’re choices all refer tomaster)
Charlotte Soneson (11:57:53): > Possibly
Kevin Rue-Albrecht (11:59:08): > I’m setting up the files right now and pushing in a few minutes
Kevin Rue-Albrecht (12:06:58): > a’right, moment of truth.. or at least in whatever time it takes Travis to run thepkgdownbranch.
Charlotte Soneson (12:08:07): - File (PNG): Screen Shot 2019-04-13 at 18.07.50.png
Charlotte Soneson (12:08:15): > No need to change anything, it was automatically set
Kevin Rue-Albrecht (12:09:05): > ow, that’s why I don’t remember “what” I did xD
Kevin Rue-Albrecht (12:41:50): > <!channel>Alright,masterpassed after merging. I’m pushing 1.3.9 upstream now
Kevin Rue-Albrecht (12:45:17): > Done.
Kevin Rue-Albrecht (13:01:22): > Duh. It seems that branches need to be explicitly allowed to deploy the GH website. Fixing now.
Kevin Rue-Albrecht (13:02:14): > I don’t understand what makes some build fail to fetch the package cache. The last couple of them ran for ~50min because of that:sleepy:
Federico Marini (13:32:18) (in thread): > Well the param is not in the gist, but the idea would also b eto have something like > > wrapup_for_iSEE <- function(dds, res, anno_df, > add_reddim = FALSE) >
Federico Marini (13:32:56) (in thread): > but i SEE all reasonable points here
Federico Marini (13:33:22) (in thread): > plus. You basically just remind me I can go without extra dep for this as the souvenir button afterideal
Charlotte Soneson (13:34:09) (in thread): > :+1:
Federico Marini (13:34:34): > re: news. Do we want to put the shiny contest win as well?
Kevin Rue-Albrecht (13:35:20): > ahh not a bad idea, though I was thinking more about the README or the vignettes than the NEWS tbh
Kevin Rue-Albrecht (14:06:57): > Tadaaaa :https://csoneson.github.io/iSEE/ - Attachment (csoneson.github.io): Interactive SummarizedExperiment Explorer > Provides functions for creating an interactive Shiny-based graphical user interface for exploring data stored in SummarizedExperiment objects, including row- and column-level metadata. Particular attention is given to single-cell data in a SingleCellExperiment object with visualization of dimensionality reduction results.
Kevin Rue-Albrecht (14:08:17): > PS: we don’t want to rename our carefully alphabetically ordered vignettes, but if we wantpkgdownto automatically create a “Getting started” tab in the navigation bar, then we should create a vignette namediSEE.Rmd
Kevin Rue-Albrecht (14:29:07): > Usually, I would put a link to the pkgdown website at the top of the GH repo page, but I think it’s better to keep the official Bioc landing page up there. So until we find an adequate place to advertise it, I guess we’ll share it “word of mouth”
Charlotte Soneson (14:29:19): > :+1:
Charlotte Soneson (14:29:59): > So this will be automatically deployed every time we push, or how will it work (sorry for my ignorance:flushed:)?
Kevin Rue-Albrecht (14:34:19): > Yep, every time we push (tomaster, which is protected, so actually every time we merge), Travis will build and deploy the website (i.e., push all the “website files” togh-pages). > That said, I’ve left another possibility open in.travis.yml, namely any commit pushed to a branch namedpkgdownwould also have the same effect. Use with caution as there is only one website (gh-pages), so the “main” website would be replaced.
Kevin Rue-Albrecht (14:36:13): > I’ve got to say that I’m pretty amazed myself about the integration between platforms these days
Kevin Rue-Albrecht (14:49:12): > Only issue now is that the logo in the README doesn’t show up on the pkgdown website, because theinst/folder doesn’t exist there
Kevin Rue-Albrecht (14:52:22): > The only solution stable for both the README and the pkgdown website would be to link to the full online URL:https://github.com/r-lib/pkgdown/issues/133#issuecomment-249990513I generally prefer using 1) relative paths , 2) local files, but it’s just a README which are generally text files anyway, so I don’t think many users will compile the README.md file locally without an internet connection… which would be the only way for them to have an issue ^^
Aaron Lun (18:09:07): > okay, second round of moving stuff.
Aaron Lun (18:09:24): > moved all my clothes last round - time for electronics.
2019-04-14
Kevin Rue-Albrecht (14:38:38): > FYI: 1.3.9 passed the build on Bioc, it’s just awaiting being pushed / propagated to the landing page
Kevin Rue-Albrecht (15:18:59): > Does anyone mind if I unwrap the lines in the workshop Rmd, 1 phrase per line in the Rmd file, like we’ve done for the package vignettes?
Charlotte Soneson (15:19:26): > Sure, that’s fine.
Federico Marini (15:20:31): > :thumbsup:
Federico Marini (15:20:52): > Sorry I could not throw in more than a couple of ideas as of now
Kevin Rue-Albrecht (15:47:23): > I think we’ve got a good base. I was just looking at trimming my earlier contribution where I launched an app as soon as thepbmc3kobject is created. As pointed out by a comment, it’s too early and confusing. Especially, because that app would not have a redDimPlot panel to showcase. > I’m trying to rephrase that part to mention that it’s possible to launch an instance, but that for the purpose of the workshop, we’ll first add process the data a bit more.
Aaron Lun (16:12:20): > Moved another table
Aaron Lun (16:12:25): > Can’t raise my arms right now.
Kevin Rue-Albrecht (16:59:47): > Good that iSEE has voice recognition now then
Aaron Lun (17:29:09): > lol
Aaron Lun (17:38:59): > omg R loads so fast on this laptop
Aaron Lun (17:39:12): > Bioc packages attach in a flash.
Kevin Rue-Albrecht (17:42:07): > What kind of laptop is it?
Aaron Lun (17:42:24): > Dell precision 5530
Aaron Lun (17:42:32): > 8th gen I7
Aaron Lun (17:42:32): > 16 GB RAM
Aaron Lun (17:42:33): > 500 GB SSD
Aaron Lun (17:42:43): > cost >2000 USD
Aaron Lun (17:43:22): > I picked it because it came pre-loaded with ubuntu so I didn’t have to wrestle with the re-installation of the correct drivers, etc.
Aaron Lun (17:43:57): > I was going to get the xps 15 but the web was full of difficulties with installing ubuntu on it.
Aaron Lun (17:44:26): > Ironically, the ubuntu laptops cost more than the u$oft laptops, because the former only got sold with high-specs.
Aaron Lun (17:44:42): > ACtually, I was going to really splurge
Aaron Lun (17:44:42): > i9
Aaron Lun (17:44:47): > 32 Gb ram
Aaron Lun (17:44:50): > 1 TB ssd
Aaron Lun (17:44:57): > cost >$3000
Aaron Lun (17:45:03): > I could run cellranger on this thing
Aaron Lun (17:45:11): > but then I realized I didn’t actually need to run cellranger at home.
Aaron Lun (17:45:16): > so I went second tier.
Kevin Rue-Albrecht (17:46:04): > hahaha
Kevin Rue-Albrecht (17:46:26): > well i kind of miss the specs that come with non-Apple products > 2k $
Aaron Lun (17:46:57): > If I went for a desktop, I could have gotten a monster for 2k
Aaron Lun (17:47:06): > 32 GB RAM easy.
Kevin Rue-Albrecht (17:47:53): > alright, gotta go to sleep now
Kevin Rue-Albrecht (17:48:02): > damn time difference
Kevin Rue-Albrecht (17:50:03): > I’ve pushed a minor edits and cleanup athttps://github.com/kevinrue/BiocWorkshops/tree/isee2019
Kevin Rue-Albrecht (17:50:24): > it’s grown fast already !
Aaron Lun (18:06:47): > that’s what I say when I watch grad students.
2019-04-16
Charlotte Soneson (11:50:29): > I made some updates to the Bioc workshop (basically, subsections of section 1.2.4) to illustrate different ways of interacting with an app. It’s not very polished, but if someone would like to have a look and see whether you agree to the general plan/way of presenting things that would be good. Not sure whether we should have screenshots for every step, depends a bit on how people may use it eventually.
Kevin Rue-Albrecht (14:15:59): > I’ll compile and check this evening. Hopefully I’ll contribute something useful sometime soon:slightly_smiling_face:
Kevin Rue-Albrecht (14:16:15): > For what it’s worth, I’ve added a few slides to the pptx in our shared Dropbox
Kevin Rue-Albrecht (14:16:32): > Courtesy of@Rob Amezquita’s OSCA
2019-04-17
Federico Marini (09:53:36): > People cannot unseeiSEEthis summer
Federico Marini (09:54:34): > 3 weeks in a row there’s something:smile:
Charlotte Soneson (09:55:57): > :+1:I think I’ll also have some time to show it in the summer school in Belgium in July:stuck_out_tongue_winking_eye:
Federico Marini (09:56:51): > I was already taking that into account:smile:
Federico Marini (09:57:05): > Mayyybe we do have to order some new round of stickers?
Federico Marini (09:57:29): > stickermule is also coming to useR, maybe we get a special pack:party_parrot:
Charlotte Soneson (09:57:29): > Ah, I see:wink:
Kevin Rue-Albrecht (15:46:21): > Thanks a ton again for the work on the workshop@Charlotte Soneson. > Going through it and making small edits is reviving my impostor syndrom x1000
Kevin Rue-Albrecht (15:46:23): > :stuck_out_tongue:
Kevin Rue-Albrecht (15:47:24): > And there I was.. worried about filling 2h… we should have applied for a day-long workshop
Charlotte Soneson (16:09:29): > Haha, yeah, I don’t think we’ll have any problems filling 2 hours…probably we’ll need to go through once we have all the material and cut here and there.
2019-04-18
Aaron Lun (02:20:59): > I realized why my rent was so cheap despite the amazing views.
Aaron Lun (02:21:23): > As I was walking to the bus in the morning, there was a guy taking a dump in a paper bag.
Aaron Lun (02:21:55): > And when I was walking back to my apartment, there were police investigating a robbery in the subway below my apartment.
Aaron Lun (02:21:58): > Yeah, it’s that kind of hood.
Charlotte Soneson (02:22:41): > Sounds…pleasant. Are you still in downtown SF, or closer to work?
Aaron Lun (02:23:33): > downtown
Aaron Lun (02:23:55): > but the views are truly amazing.
Aaron Lun (02:24:06): > It’s fine as long as you don’t go outside.
Charlotte Soneson (02:24:50): > Well, I guess that’s one solution
Aaron Lun (02:25:21): > Just wear PPE at all times
Aaron Lun (02:25:24): > closed footwear, etc.
Aaron Lun (02:25:35): > There’s not as many needles as I expected, but still.
Kevin Rue-Albrecht (04:39:32): > Ok.. I know it’s not a competition, but perhaps UK aren’t all that bad after all:sweat_smile:
Aaron Lun (23:09:25): > God being an adult sucks
Aaron Lun (23:09:32): > I spent 1 hour at home dealing with administrative crap.
Aaron Lun (23:09:38): > Bills etc.
Aaron Lun (23:32:39): > Thank god I found some food in the office so I didn’t have to cook dinner.
2019-04-19
Federico Marini (07:36:06): > Which of the different shades of “found”?:smile:
Kevin Rue-Albrecht (17:22:44): > Once I was given a desk where i found a travel mug that contained what probably used to be coffee with a record layer of mould floating on top:nauseated_face:
Aaron Lun (17:25:24): > Yeah, I’ve seen that too.
2019-04-20
Aaron Lun (01:35:32): > Kevin your gravatar disappeared. Can’t see your pretty face anymore.
2019-04-22
Aaron Lun (22:19:20): > shinygot updated. IsiSEEstill okay?
Aaron Lun (22:21:05): > Update totestthatas well, which is possibly more concerning.
2019-04-23
Charlotte Soneson (04:51:12): > I will restart the travis build to check.
Charlotte Soneson (07:33:09): > Seems ok > > shiny (1.3.1 -> 1.3.2 ) [CRAN] > testthat (2.0.1 -> 2.1.0 ) [CRAN] >
Charlotte Soneson (07:33:18): - File (PNG): Screen Shot 2019-04-23 at 13.31.52.png
Kevin Rue-Albrecht (09:09:37) (in thread): > Really? I can it fine myself.
Aaron Lun (11:25:50) (in thread): > ah, it came back.
2019-04-24
Aaron Lun (00:01:47): > okey dokey
2019-04-25
Kevin Rue-Albrecht (06:44:38): > Reading some Brainstorming material:http://www.brainstorming.co.uk/tutorials/challengefactstutorial.html > > “What seemed impossible one week can become plausible the next, reality within months and an accepted way of life in a year or two.” > Reminds me of iSEE.. we’ve got another year or so:stuck_out_tongue: - Attachment (brainstorming.co.uk): Challenge facts techniques tutorial for better brainstorming > Challenge facts techniques tutorial training for better brainstorming for problem solving
Federico Marini (11:52:41): > is the website intentionally like megacities?
Federico Marini (11:52:44): > or whatever it was called
Federico Marini (11:53:31): > I see the gif with Bill Gates morphing into the devil is missing:smile:
Federico Marini (11:53:48): > wait
Federico Marini (11:53:49): > sorry
Federico Marini (11:53:51): > geocities
2019-04-26
Kevin Rue-Albrecht (04:22:22): > No idea:sweat_smile:
2019-04-29
Federico Marini (13:28:58): > I like it calm before release:slightly_smiling_face:
Kevin Rue-Albrecht (14:18:22): > I like the download page, slowly but surely climbing
2019-04-30
Kevin Rue-Albrecht (13:48:16): > I’ve added a description of adatatableglitch in the 1-to-1 RStudio Google Docs (https://docs.google.com/document/d/196tFRn4PpARh1leTWfGYnuG4v3r8TdkG1gRgZowYMYE/edit) - File (Google Docs): iSEE-shiny-contest-one-on-one
Kevin Rue-Albrecht (14:00:59): > @Charlotte Sonesonmini update to the workshop - don’t forget topull
Kevin Rue-Albrecht (15:21:24): > Also, I’ve put a series of to do’s in the workshop Google Docs, as we can’t add issues on the my BiocWorkshop fork (https://docs.google.com/document/d/10r0hhrQE89McjEVmq2aaOKBQ260PKJSzxik1jrL2da4/edit#) - File (Google Docs): isee-bioc2019-workshop
2019-05-01
Kevin Rue-Albrecht (15:53:45): > I’m trying to add a bit of workshop now, and I ran into what looks like a bug: > - in areduced dimension plot xxpanel > - setColor bytoSample name- set the second dropdown to acolumn stat table xx- > - Open the panel chart > All the panels are there, but there is no link from the table to the redDimPlot. > Can anyone else see that?
2019-05-02
Aaron Lun (02:01:19): > I am so tired.
Aaron Lun (02:01:24): > Almost fell asleep on the bus.
Kevin Rue-Albrecht (05:02:27): > The opposite here. Snoozed the alarm and overslept. Calculated risk as there I had no meeting this morning:grimacing:
Kevin Rue-Albrecht (18:01:48): > Tadaaa. I’m finally happy (for now). I’ve added a code chunk for each of the existing workshop sections, to illustrate each point with a minimal app instance.
Aaron Lun (23:48:18): > Put a bug report up for the behaviour you observed.
Aaron Lun (23:48:34): > Also, look at sctransform’s “perfect” variance stabilization on a simple simulation.
Aaron Lun (23:48:45): - File (PNG): blah.png
Aaron Lun (23:48:52): > I mean… WTF>
2019-05-03
Charlotte Soneson (02:14:39): > Awesome - I’ll take a look at this and add some more stuff.https://community-bioc.slack.com/archives/C8BJLSP8T/p1556834508017800 - Attachment: Attachment > Tadaaa. I’m finally happy (for now). I’ve added a code chunk for each of the existing workshop sections, to illustrate each point with a minimal app instance.
Kevin Rue-Albrecht (03:29:29): > One thing I struggle to decide is whether each code chunk should create a new app “de novo”. > In a few cases, they don’t: they continue editing an app set up in the previous chunk(s). > Let me know if it’s too confusing/risky
Charlotte Soneson (03:30:45): > I think overall that’s fine. It’s always possible to go back to the previous chunk to recreate the app there.
Federico Marini (08:19:13): > maybe we could go like:app <- iSEE(sce)withrunApp()every time to keep things “stored”?
Federico Marini (08:19:46): > and give meaningful names to the app, sayapp_allpanels_linked
Kevin Rue-Albrecht (08:55:51): > Could be. I never paid attention to the size of the shiny app objects stored byapp <- iSEE(sce), but they should be negligible as they don’t store the data (I believe)
Federico Marini (09:05:42): > > > app <- iSEE(sce) > 'Custom data plot 1' in 'initialPanels' is not available (maximum ID is 0) > > pryr::object_size(app) > 708 MB >
Federico Marini (09:05:53): > on the 3k set
Federico Marini (09:06:12): > I don’t know if then it exactly pile up for the others:disappointed:
Federico Marini (09:08:56): > Kev, one thing I saw in use which could come in handy
Federico Marini (09:09:14): > apart from the action encoding, maybe a small image?
Federico Marini (09:10:03): > like Questions/solutions in msmb
Federico Marini (09:10:05): > https://www-huber.embl.de/users/msmith/msmbstyle/questions-and-solutions.html - Attachment (www-huber.embl.de): MSMB HTML Book Style > MSMB HTML Book Style
Federico Marini (09:11:09): > We would just need nicely-shaped css classes
Federico Marini (09:13:57): > (or here, i think these are in by default in bookdown)
Federico Marini (09:13:58): > https://www.tidytextmining.com/ngrams.html - Attachment (tidytextmining.com): 4 Relationships between words: n-grams and correlations | Text Mining with R > A guide to text analysis within the tidy data framework, using the tidytext package and other tidy tools
Federico Marini (09:14:11): > (just before 4.1.1)
Kevin Rue-Albrecht (09:14:23): > Sure. We could see how an inline fontawesome “triangle” icon looks like at the start of the line
Federico Marini (09:14:52): > Probably slightly bigger could be even a nicer visual cue
Federico Marini (09:15:10): > or also helped out by the quote-like environment
Federico Marini (09:15:14): > that could also be enough
Federico Marini (09:15:25): > with a different background, it is quite hard to miss
Kevin Rue-Albrecht (09:17:38): > Well. If you have time, perhaps throw some screenshots in the channel? > To save time, I’d write a few different strategy under each other and compile the whole lot once. (as opposed to writing one, compiling it, . > > <fontawesome approach> Action ... > > <other approach> Same action ... >
Federico Marini (09:18:17): > will do
Kevin Rue-Albrecht (09:18:28): > I’ll spend some time on it this evening. I’m cleaning up a pipeline and preparing slides for my lab meeting next week now:sleepy:
Federico Marini (09:19:49): > I’ll run down through the whole text soon enough
Federico Marini (09:20:00): > and try out the action highlighting
Kevin Rue-Albrecht (09:22:24): > Good idea anyway. I think it will look better that way
Federico Marini (09:23:08): > I have a couple of advertising-related ideas on top
Federico Marini (09:23:15): > let me throw them in here
Federico Marini (09:23:41): > 1.iSEE.io- as a mirror to the main demo app? > 2. some css-based app pimping - fonts, stuff like that (i saw this one recentlyhttps://polcastellano.shinyapps.io/POMA/) > 3. can google analytics help in understanding where users might get stuck?
Kevin Rue-Albrecht (09:24:28): > not sure what you mean here
Federico Marini (09:25:24): > an extra domain for iSEE
Federico Marini (09:25:28): > “just that”
Federico Marini (09:27:46): > 4. a commemorative sticker- glossy “shiny” style, to be given out e.g. to early adopters and/or worshippers
Kevin Rue-Albrecht (09:27:59): > Hm. If you wanna get a prototype of theiSEE.iostarted somewhere, I might get a better idea of what you’d hope to see there. I’ve been happy with the Bioc landing page and vignette, but I suppose that it wouldn’t hurt to have a space to describe more freely other aspects of the package / app
Kevin Rue-Albrecht (09:28:49): > Also, we do have the GitHub pages website now. I like it for the cool look, but to be honest it doesn’t bring that much more than the Bioc vignette and reference manual.
Kevin Rue-Albrecht (09:29:06): > Still, we have it and we haven’t advertised it very widely yet
Kevin Rue-Albrecht (09:30:19): > Oh. Also, looking through the branches of our GH repo, I notice that your “runtravisrun” branch still isn’t merged tomaster. I think it’s safe enough to merge, if you like. Otherwise we should probably delete it to avoid stale branches
Federico Marini (09:31:25): > The gain is marginal, but it is a harmless extra - I’d say we call the shot after release is done?
Kevin Rue-Albrecht (09:33:02): > Oh right - I thought the release was already done
Federico Marini (09:34:21): > they slightly postponed I guess
Federico Marini (09:34:43): > Which reminds me of…
Federico Marini (09:34:58): > we need a tweet to advertise the release?
Federico Marini (09:35:34): > If we want to follow up on the GoT style, let me know if I have to activate my stupid part of the brain:smile:
Federico Marini (09:36:02): > The Danaerys meme image could be effectively used:stuck_out_tongue:
Kevin Rue-Albrecht (09:53:43): > My stupid part of the brain is wayyyy too easy to activate… you just made me think I could add “Dracarys” as an Easter egg vocal command…. could change all the color maps to a yellow-red gradient:wink:
Federico Marini (09:53:59): > Oh
Federico Marini (09:54:19): > Wish I could say “do it”
Federico Marini (09:54:25): > and that just would happen
Federico Marini (09:54:53): > What do we say to the god of non reproducible analysis?
Kevin Rue-Albrecht (09:55:05): > Or “What do we say to the God of copy-pasting code to generate a thousand variants of the same plot? - Not today”
Kevin Rue-Albrecht (09:55:09): > aaaaargh
Kevin Rue-Albrecht (09:55:12): > you beat me to it
Federico Marini (09:56:08): > Great (?) minds think alike?
Kevin Rue-Albrecht (09:57:32): > “iSEE - the Valyrian steel of reproducible analysis” ?
Kevin Rue-Albrecht (09:58:09): > Or perhaps “Use Valyrian steel. Use iSEE”:sweat_smile:
Charlotte Soneson (09:59:11): - File (JPEG): download.jpeg
Kevin Rue-Albrecht (10:02:01): > “Don’t have a three-eyed raven? iSEE is here for you!”
Federico Marini (10:03:04): > Finally we lured Charlotte out:smile:
Federico Marini (10:03:29): > Brown points, green points, BLUE POINTS
Kevin Rue-Albrecht (10:03:36): > lol
Federico Marini (10:04:12): > I assume you’ll demo the voice live as well?
Kevin Rue-Albrecht (10:04:58): > If one’s brought back to life 19 times, and dies on the 20th… is that a significant death?:face_with_raised_eyebrow:
Federico Marini (10:05:52): > Empirically speaking, yes
Federico Marini (10:06:18): > But with so many dothrakis, you should apply FDR corrections
Kevin Rue-Albrecht (10:06:25): > > I assume you’ll demo the voice live as well? > Possibly. I guess we’ll have to wait and see if it works on the environment that we’ll have at the conference
Kevin Rue-Albrecht (10:08:48): > > But with so many dothrakis, you should apply FDR corrections > I have to say, the Winterfell group did a pretty good job of aligning those batches of soldiers for a moment. What that CCA of MNN ?
Federico Marini (10:11:49): > Tactically, it sucked to send them against a platoon of non-scarable undead
Federico Marini (10:12:29): - File (PNG): Pasted image at 2019-05-03, 4:12 PM
Federico Marini (10:12:37): > I like a lot the third one
Federico Marini (10:13:03): > Could be also nicely mirrored by what happens
Federico Marini (10:13:40): > and it is just a > > > <i class="fa fa-wrench"></i> ***Action:*** Using your open `BiocStyle::Biocpkg("iSEE")` instance, open the `Organization` menu and click on the `Organize panels` button. > Try to add and remove panels, and resize the existing ones. > Remember to click on `Apply settings` to apply the changes you have made to the interface. >
Federico Marini (10:13:50): > no fiddling of css
Kevin Rue-Albrecht (10:16:57): > I think I like the last one best
Kevin Rue-Albrecht (10:18:36): > Reasoning: > - “action” is there to point out there is something to do. > - it’s good to have a logo as a visual cue > - the indent helps bring out better where the action paragraph starts and ends
Federico Marini (10:19:17): > Yep, pretty much
Federico Marini (10:19:23): > If it helps more,<i class="fa fa-wrench fa-3x"></i>
Federico Marini (10:19:41): > or 2x, FWIW
Federico Marini (10:19:48): > makes it slightly bigger
Federico Marini (10:20:24): > If we pursue this way, we can have a demonstrative action block up at the beginning
Federico Marini (10:20:42): > so that users learn right away that that is the cue to look for
Kevin Rue-Albrecht (10:21:58): > Sure. Go for it
Federico Marini (10:23:15): > :thumbsup:
Federico Marini (10:55:55): > edited and pushed
Aaron Lun (15:54:15): > bugfixed.
Kevin Rue-Albrecht (15:59:18): > Seems like we’ll have to wait for 3.10 to be fully deployed for Travis to pass
Kevin Rue-Albrecht (15:59:29): > > Error: (converted from warning) unable to access index for repository[https://bioconductor.org/packages/3.10/data/annotation/src/contrib](https://bioconductor.org/packages/3.10/data/annotation/src/contrib): > cannot open URL '[https://bioconductor.org/packages/3.10/data/annotation/src/contrib/PACKAGES](https://bioconductor.org/packages/3.10/data/annotation/src/contrib/PACKAGES)' >
2019-05-04
Kevin Rue-Albrecht (15:52:47): > Done my bit of workshop for the day. Two more sections. It’s taking more time that I thought, between the “introduction - interactive action - mini app preconfiguration” pattern:sweat_smile:
Aaron Lun (17:28:55): > I submitted another Bioc workflow today.
Aaron Lun (17:29:21): > Almost finished phase II of the Genomics interactions refactoring.
Kevin Rue-Albrecht (18:48:42): > Following up on an email I received from Levi tonight (I will forward to you now), I’m migrating the current draft of the workshop here:https://github.com/kevinrue/Bioc2019-iSEE-workshop
Kevin Rue-Albrecht (18:49:37): > New instructions (whispers have been around for a while now) are that workshops should be presented as packages
Aaron Lun (18:50:21): > huh.
Kevin Rue-Albrecht (18:54:51): > I pushed. It built for me (devtools::build_vignettes())
Aaron Lun (18:55:49): > okay. Well, let me know if you need urgent and special help.
Aaron Lun (18:56:00): > Think of me like your spec ops team.
Aaron Lun (18:56:08): > Because otherwise I’ve got this other crap to deal with.
Kevin Rue-Albrecht (18:58:00): > I’m about to go to sleep, but help is welcome: see the email about strict deadlines.
Kevin Rue-Albrecht (19:00:25): > In the ‘helpful’ section: > - checking the content so far, to spot any obvious gap and fill in sections still blank if you feel like it > - checking the email i just forwarded you, if you can spot any guidelines we’re not respecting
Aaron Lun (20:00:53): > going for a walk. If I keep on writing code in my room I’ll explode.
2019-05-05
Kevin Rue-Albrecht (04:25:22): > So …kevinrue/Bioc2019-iSEE-workshopworks like this for me: > > devtools::install(build_vignettes = TRUE) > browseVignettes("iSEEWorkshop") >
Federico Marini (08:49:09): > Pinging again about the post-release message
Federico Marini (08:49:22): > Anyone with a strong meme idea?:stuck_out_tongue:
Kevin Rue-Albrecht (08:57:38): > “iSEE 10 times over”
Federico Marini (09:02:47): > me do not get it
Federico Marini (09:03:05): > but it can be because we went to a Sektfrüstück in the wine fields:smile:
Kevin Rue-Albrecht (09:04:49): > https://wdawkt.wordpress.com/2019/04/22/im-no-king-but-if-i-were-id-knight-you-ten-times-over/ - Attachment (We Drink and We Know Things): “I’m no king, but if I were. I’d knight you ten times over.” > Season 8, Episode 2: A Knight of the Seven Kingdoms > > 1. Redemption seems to be a major theme in this week’s episode. How do the characters seeking redemption succeed or fail when faced with the consequences of their actions? > E: Redemption is a clear story arc, and we see that front and center with Jamie. Jamie, in front of an impromptu trial, did all that he could to show those in the room that he is a different man and that he is a sincere ally in the fight for the living. However, I think it is too early to determine who will succeed or fail in their quest to be redeemed. Everyone appears to have their hearts and minds in the right place, but what comes after this? Who knows? > S: Many characters that have now convened at Winterfell are seeking redemption in a significant way. I’ll start with Jaime since he arrives at Winterfell and is immediately confronted with the sins of his past– Bran– waiting for him in the courtyard. This week’s episode doesn’t skip a beat, and we find Jaime, situated in an impromptu “trial” of sorts, face-to-face with Daenerys, the aftermath of his moniker, Kingslayer. I do like that Jaime does not offer an apology to Daenerys for murdering her father, and I thought that her anger towards him was a bit over-the-top, given what she now knows about her father. I understand that she wants to flex her authority, but now it’s not the time, they need as many people as possible fighting for the Northern cause. > > Jaime’s encounter in the godswood with Bran was a poignant moment in the episode. Jaime’s apology to Bran and Bran acknowledging that at the time of the push, Jaime was not sorry for what he did shows the extent to which Jaime has changed, and by extension, how it facilitated Bran’s transformation. For all of the lack of humanity Bran projects, this small exchange between him and Jaime (and Bran’s decision to not reveal Jaime as the perpetrator of his fall) is an excellent indicator that there are still some in the Winterfell camp who understand the chasm of doom that is rapidly approaching, and that the suppression of certain knowledge is key to focusing energies on what needs to be done. > > Theon has returned to fight for Winterfell, and more importantly, by Sansa’s side and volunteers to protect Bran during the battle. I hope his bid for redemption finds a substantial end, given all that he has suffered. I agree with you, Earl in that people’s hearts and minds, mostly, are in the right place, it all comes down to what happens in next week’s episode and those remaining after that. > > 2. The calm before the storm gave viewers many beautiful moments between beloved characters. Is that any indication as to the fates of some of these characters in the coming episodes? > E: I think Brienne and Theon are as good as dead in the next episode. Brienne’s interactions with Jamie and Theon’s interactions with Sansa both felt like finality for me. Theon especially. It’s noble that he wanted to go to Winterfell to fight for the Starks, his wards and the only family he grew to know and to fight for Sansa who he both protected and rescued. Him dying at Winterfell feels appropriate. > Brienne dying now also feels appropriate. Jamie knighting her is a final act. He showed both his love and respect for her. What more does their story need? > > S: I think we all know quite well that whenever we get a taste of something good from Game of Thrones, there’s always something horrific lingering just outside the frame to wreck it all. These beautiful moments between Gendry and Arya (!!), Missandei and Grey Worm and even the motley crew of Tyrion, Jaime, Tormund, Davos, Podd and Brienne does not bode well for what’s to come. > > > The knighting of Brienne by Jaime reminded me of a similar one in Season 6 between Tyrion and Daenerys when she names him Hand of the Queen. For so long in this series, both Brienne and Tyrion were seeking approval, either from the wrong people or from those not in permanent positions of authority, never been validated for their contributions to the world they live. Tyrion finally receives the highest form of validation when Daenerys names him Hand in his own right, and not as a stand-in for someone better (although this week’s conversation between Jorah and Dany might say otherwise). Likewise for Brienne, who from the moment we met her, wanted nothing more than to be a knight in the King’s Guard. It was fitting for Jaime to be the one to knight her; to him, she upholds the principles of knighthood in a way that he never could. > This is precisely why I think Brienne might not make it out of the battle next week, and this pains me. > > 3. Jaime and Brienne’s relationship was made plain in this episode. What does that mean for these two characters in terms of their paths moving forward? > E: I touched on this in the previous question but I expect Brienne to die, and I expect Jamie to live only because his story can only end with one more confrontation with Cersei. Jamie dying at Winterfell does not seem appropriate in my mind. Jamie needs to face Cersei one last time. > S: Jaime and Brienne’s affection for one another has grown out of mutual respect. With that said, I think Brienne’s most significant effect on Jaime was to lead him on a path of transformation and that role is coming to an end. Jaime’s loyalty to Cersei no longer holds sway over him, and so he has come to Winterfell, with Brienne’s words in his mind, “this goes beyond houses and honor and oaths..” to fight on the side of the living. This change of heart would not be possible without Brienne. Knighting her was a way to seal her legacy and to do it before the battle at Winterfell almost confirms her demise. I’m not confident that Jaime will make it out alive either, I’d be surprised if he does. > > > 4. What do you make of the conversation between Daenerys and Sansa? > E: Daenerys tried to be human at that moment, but of course it all failed miserably. Sansa is the smartest player of the game right now and the fact that she still pressed for the North, even in the face of a girl talk session with Daenerys, was the right move. It’s great that Jon brought Daenerys north, but what about the North? Good move, Sansa. Good move. > S: It seems as though Jorah would’ve made a better Hand as it is his counsel that nudges Daenerys towards clearing the air with Sansa. There was this fleeting moment where I thought Daenerys would offer Sansa Hand of the Queen, but alas…. > Daenerys’ reaction to Sansa’s question about the fate of the North after the dust settles was quite worrying. If we think about how Daenerys treated Yara’s request to retain the Salt Crown and the Iron Islands as compared to her inability to even entertain Sansa’s request to keep the North under Stark rule, it is a bit hypocritical. Tyrion was the one who expressed some trepidation when the Greyjoys asked for independence, but it was Daenerys who remarked that the kingdoms were all free to ask, and she would decide what kingdoms keep what crowns. Here, Sansa is asking, but Daenerys refuses to answer. > > 5. What do you make of the Winterfell military strategy? > E: What strategy? There was no strategy, and that felt so weird. You know the White Walkers are coming, and you have known it for a while so there should have been a strategy. It all was patched together at the table, and a lot of it doesn’t make sense. All that we know is Bran will try to attract the Night King, and there is plenty of dragon glass. > S: Haha, what strategy indeed! After the carnage at Last Hearth, I would imagine that keeping as many bodies inside the walls of Winterfell would be paramount. Now that Rhaegal has a rider, can’t Jon and Daenerys decimate as much of the Army of the Dead as they can from the air,…
Federico Marini (09:05:03): > Aaaah
Federico Marini (09:05:04): > damnit
Federico Marini (09:05:49): > I had a couple of memes for presentation purposes but we can repurpose them for announcements as well
Federico Marini (09:06:06): - File (JPEG): meme_boyfriend.jpg
Federico Marini (09:06:09): - File (JPEG): meme_drake_isee.jpg
Federico Marini (09:06:12): - File (JPEG): meme_mim.jpg
Charlotte Soneson (09:07:13): > I feel like I can’t really contribute much to the creativity here - my best suggestion would be “iSEE 1.4.0 is out, here are the new features”:face_with_rolling_eyes:and right now it seems my R/Bioconductor installation has gone completely crazy so my priorities is to be able to run it at all…
Federico Marini (09:08:29): > :stuck_out_tongue:
Federico Marini (09:08:59): > That would also be fine
Federico Marini (09:10:45): > We had voice + multiselection + organization of panel, no more than these right?
Charlotte Soneson (09:33:17): > Think that’s it
Federico Marini (09:34:22): > Ok
Federico Marini (09:34:30): > Drafting it now
Charlotte Soneson (09:34:34): > Quick question: DoesExperimentHubwork as expected for you guys with the new devel? The following gives me a hub object with only 171 entries, not the >2,000 that it should have. > > > library(ExperimentHub) > > eh <- ExperimentHub() > snapshotDate(): 2019-04-29 > > eh >
Federico Marini (09:36:54): > Did you update to the new format?
Federico Marini (09:37:00): > Or is it unrelated?
Charlotte Soneson (09:37:22): > Ehm, what new format do you mean?
Federico Marini (09:37:40): > I recall Lori posting something on the new underlying structure
Federico Marini (09:37:49): > but as of now: also 171 records for me
Charlotte Soneson (09:39:02): > Hm, ok. There seems to be something wrong withExperimentHubon the new develhttp://bioconductor.org/checkResults/devel/bioc-LATEST/ExperimentHub/
Federico Marini (09:40:12): > they also fail to fetch the one in the vignette
Charlotte Soneson (09:40:41): > Which may be the reason why this doesn’t work: > > > library(TENxPBMCData) > > sce <- TENxPBMCData(dataset = "pbmc3k") > snapshotDate(): 2019-04-29 > Error in .local(x, i, j, ...) : 'i' must be length 1 >
Federico Marini (09:41:04): > same here
Charlotte Soneson (09:41:54): > Ok, at least it’s not me going crazy…@Kevin Rue-Albrecht, I guess you didn’t build the workshop with the new devel?
Kevin Rue-Albrecht (09:49:18): > I’ve just runBiocManager::install(version = "devel")now. Let me try
Kevin Rue-Albrecht (09:51:49): > Oh right. I’m getting the error too now
Kevin Rue-Albrecht (10:02:39): > Not sure how we’re supposed to adapt to this now:confused:
Charlotte Soneson (10:03:10): > Well, release works for me so I’ll just go with that for now I guess
Kevin Rue-Albrecht (10:04:10): > So how did you do that?BiocManager::install(version = "release")or something?
Charlotte Soneson (10:04:25): > At least right now they should be the same…justBiocManager::install(version = "3.9")
Federico Marini (10:04:30): > Specifying 3.9 should do the job
Federico Marini (10:04:36): > Charlotte was quicker
Charlotte Soneson (10:04:39): > In a separatelib.locso I can swap back and forth
Kevin Rue-Albrecht (10:16:17): > Definitely more efficient than up/down-grading ~60 packages back and forth:sweat_smile:
Kevin Rue-Albrecht (10:29:20): > How do you set up thatlib.locagain? I’m confused. One thing is to give it toinstalled.packages(lib.loc=...)as “character vector describing the location of R library trees to search through” > The other thing is to load the package from the right folder in the session.
Charlotte Soneson (10:31:36): > I did > > .libPaths("<my_3.9_lib_loc>") > BiocManager::install(version = "3.9", lib.loc = .libPaths()[1]) >
Kevin Rue-Albrecht (10:43:56): > Right. And then the original (i.e. regular) libPaths is for 3.10 I assume
Charlotte Soneson (10:45:05): > If that’s howBiocManagerwas configured for that one, yeah, I think it should work like that.
Charlotte Soneson (10:45:22): > Haven’t tried yet going back, but it should work
Kevin Rue-Albrecht (11:13:31): > I’ve got plan B too, while I’m on my lab meeting: I’ve installed R-3.7 and I’m installing iSEE there too
Kevin Rue-Albrecht (11:14:35): > Anyway. I don’t hear complaints about the current state ofkevinrue/Bioc2019-iSEE-workshop. Either that’s positive feedback on my recent edits, or no one has seen them yet:wink:
Charlotte Soneson (11:14:51): > I think it looks good. Working on filling some more sections right now
Kevin Rue-Albrecht (12:11:38): > tempted… - File (PNG): Pasted image at 2019-05-05, 5:11 PM
Kevin Rue-Albrecht (12:11:57): > @Federico Marini
Aaron Lun (12:35:40): > the most interesting man in the world was the best one.
Aaron Lun (12:35:57): > Though I would be like “I don’t always explore SummarizedExperiment instances”
Aaron Lun (12:36:02): > “But when I do, iSEE”.
Aaron Lun (12:36:13): > “Stay thirsty, my friends.”
Federico Marini (13:18:54): > If your minds conceive more, keep on posting
Federico Marini (13:19:21): > We can have a pool from which to pick according to the audience:grimacing:
Federico Marini (13:21:50): > Zum Beispiel: if we find out what to put in here, I’ll be very happy ->https://imgflip.com/memegenerator/American-Chopper-Argument - Attachment (imgflip.com): American Chopper Argument Meme Generator - Imgflip > Insanely fast, mobile-friendly meme generator. Make American Chopper Argument memes or upload your own images to make custom memes
Aaron Lun (13:47:18): - File (PNG): Pasted image at 2019-05-05, 10:47 AM
Kevin Rue-Albrecht (13:48:24): - File (PNG): Pasted image at 2019-05-05, 6:48 PM
Kevin Rue-Albrecht (13:48:59): > How’s that? There’s only #4 that i’d like to improve a bit
Kevin Rue-Albrecht (13:52:01): > Actually, here’s a slightly better version I think - File (PNG): Pasted image at 2019-05-05, 6:51 PM
Aaron Lun (13:52:15): > can’t read it, too small font.
Kevin Rue-Albrecht (13:52:37): > > I can’t make any more violin plot, reduced dimension, clustering, heat map, …
Kevin Rue-Albrecht (14:20:37): > FWIW, having installed R-3.7 from source today, a lot of packages got installed without problem, but I’m getting segfaults when installing things likestringr
Kevin Rue-Albrecht (14:31:34): > Also, we’re gonna have@Sean Davisto thank for fixes, I just noticed he forked the workshop repo already ^2(https://github.com/seandavi/Bioc2019-iSEE-workshop/commits/master)
Kevin Rue-Albrecht (14:35:48): > ah. spotted him 10s before he opened the PR:joy:
Federico Marini (14:40:44) (in thread): > This one’s perfect to respond to the flurry of expression browsers showing up
Federico Marini (14:42:10): > We do not need that 3.7 for this round of devel?
2019-05-06
Kevin Rue-Albrecht (07:31:04): > Oh right!
Kevin Rue-Albrecht (07:32:45): > Btw, Travis is set up for the workshop. It fails for the same ExperimentHub issue:https://travis-ci.org/kevinrue/Bioc2019-iSEE-workshop
Kevin Rue-Albrecht (07:33:54): > At least we can keep developing using the release version, and Travis will let us know when the devel version works again, I guess
Aaron Lun (14:32:43): > Probably want to eitherquickClusterprior tocomputeSumFactorsor uselibrarySizeFactors.
Aaron Lun (14:33:02): > If you’re going to do it most accurately, do it right; or just do a quick approximation in the first place.
Aaron Lun (14:33:13): > And add a batman slapping robin thing.
Aaron Lun (14:33:24): > Change the first to “let’s write our own shiny app”
Kevin Rue-Albrecht (14:37:15): > Batman thing in the workshop material ?:sweat_smile:
Aaron Lun (14:37:58): > I dunno. Just have it in a slide somewhere I guess.
Kevin Rue-Albrecht (16:20:50) (in thread): > Any suggestion to tone down the reply maybe? I find it a bit… brutal ^^
Aaron Lun (16:21:47) (in thread): > “just use iSEE”
Kevin Rue-Albrecht (16:22:26) (in thread): > Done:ok_hand:
2019-05-08
Kevin Rue-Albrecht (19:19:16): > https://kevinrue.github.io/Bioc2019-iSEE-workshop/index.html:satisfied: - Attachment (kevinrue.github.io): A workshop on the iSEE package > This workshop covers basic usage of iSEE for interactive visualization of SummarizedExperiment objects.
Aaron Lun (19:20:04): > :+1:
2019-05-12
Kevin Rue-Albrecht (11:41:02): > Preferred syntax? > > # Option 1 > redDimArgs[1, PointSize] <- 1 > redDimArgs[2, PointSize] <- 2 > # Option 2 > redDimArgs[["PointSize"]] <- c(1, 2) >
Kevin Rue-Albrecht (11:41:35): > Note that option 2 implies that there are only two rows
Charlotte Soneson (11:42:47): > Maybe 1 is clearer (I guess it’s for the workshop)
Kevin Rue-Albrecht (11:45:15): > thanks! exactly (i don’t mind confusing myself, but i’d rather be clear to others):slightly_smiling_face:
Kevin Rue-Albrecht (11:52:54): > Note to self: writing that code chunk for the zoom feature made me realize another simple case for a custom panel: one where the custom panel simply zooms into the area brushed in the transmitting panel. > At the moment, the closest use case we have already computes a new PCA on the transmitted data points. > Instead, all we’d have to do is to grab the range of coordinates covered by the selected data points, and draw the new custom panel zooming into those coordinates, without any recomputing. Kind of a dynamic “loupe” feature
Charlotte Soneson (11:56:56): > Feels harder to motivate though, since you could get that from the original panel
Kevin Rue-Albrecht (11:59:16): > uh. the voice of reason:sweat_smile:
Kevin Rue-Albrecht (12:02:03): > I’ve just pushed a code chunk to demonstrate a preconfigured zoom, alongside an original “100% zoom” version with a brush to indicate the zoomed area
Kevin Rue-Albrecht (12:02:58): > it gives the same impression of the thing I just described, but without an active link between the two panels
Charlotte Soneson (12:03:10): > :+1:
Kevin Rue-Albrecht (12:07:01): > Unrelated. We’ll probably have a good reason to celebrate our 100th star on the GH repo during bioc2019 if we make a good impression. We’re at 90 now:stuck_out_tongue_winking_eye:
Charlotte Soneson (12:07:26): > :party_parrot:
Kevin Rue-Albrecht (12:13:44): > on a less happy note, I’ve stumbled onto a “blame-the-user” type of bug. I’ve just opened an issue for it. Easy fix, but I want to finish the last workshop section first.
Kevin Rue-Albrecht (12:15:14): > Good to see that it’s getting harder finding bugs though:slightly_smiling_face:
Kevin Rue-Albrecht (12:24:23): > Also, I suppose there’s nothing we can do about the glitch with DT tables when clicking on the filter box for columns that are originally out of screen (scroll to the right)
Kevin Rue-Albrecht (12:41:00): > Ugh. I’m not sure how many users out there want to preconfigure filters on tables, but damn it’s a bit of a pain
Kevin Rue-Albrecht (12:48:39): > > SearchColumns: > A list containing character vectors of length equal to the number of columns in colData(se), specifying the initial value of the search field for each column. All entries default to an empty string. > > I get that this is how the information is actually stored during the app runtime, but it’s actually painful to figure out which field amongst 48 is the one that I want to preconfigure the filter for. I’ll open an issue to discuss the possibility for users to give as an alternative the following > > a list of named character vectors specifying the name of columns in colData(se) and the initial value of the search field for each column > > or better actually, it’s probably not a good idea to allow/support multiple formats of input, but instead, we could have a new function that could be usedbeforelaunching the app, and that would convert the new format I’ve just described into the correct one
Kevin Rue-Albrecht (13:05:40): > Argh.. I’ve pushed the code chunk but forgot to do the screenshots to illustrate them:sob:
Kevin Rue-Albrecht (13:11:26): > Screenshots uploaded. I think I’m done with the workshop for today. It’s all there until I change my mind again
Kevin Rue-Albrecht (13:26:20): > [iSEE-pkg update] I’ve pushed the extra sanity check forinitialPanelsI’ve mentioned above and roxygenized after@Charlotte Soneson’s recent docfix
Federico Marini (14:55:04): > On the side of fun: did you try out whether “Dracarys” would work as an easter dragon egg?
Kevin Rue-Albrecht (15:00:31): > I have not. But if it doesn’t work straight from voice recognition, it can be part of those words that I match with a number of mismatches allowed in whatever annyang hears
Federico Marini (15:03:26): > :slightly_smiling_face:
Federico Marini (15:04:00): > If that could trigger some js-black magic that “burns” the screen
Federico Marini (15:04:08): > oh my god:smile:
Kevin Rue-Albrecht (15:09:30): > Seems like it’s getting the spelling alright > (Sorry for the overkill full screenshot) - File (PNG): Screenshot 2019-05-12 20.08.43.png
Kevin Rue-Albrecht (15:14:29): > that said I don’t know what to Google to find a way to, say, play an animation/video overlaid onto the webpage, e.g. a dragon flying across the screen, coming in from one side and leaving by the other
Aaron Lun (15:18:15): > If we have any BiocGenerics::t, rowMeans, rowSums, colMeans, colSums, someone should switch these to Matrix::.
Aaron Lun (15:18:25): > See Bioconductor/BiocGenerics#2.
Federico Marini (15:32:57) (in thread): > I’ll check what enters the circle of feasibility:slightly_smiling_face:
Kevin Rue-Albrecht (16:34:59): > All I can see is: > > $ grep BiocGenerics R/* > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/defaults.R:#' @importFrom BiocGenerics colnames > R/dynamicUI.R:#' @importFrom BiocGenerics rownames > R/iSEE-extras.R:#' @importFrom BiocGenerics sizeFactors > R/iSEE-pkg.R:#' @importMethodsFrom BiocGenerics ncol nrow > R/plotting.R:#' @importFrom BiocGenerics rownames > R/plotting.R:#' @importFrom BiocGenerics colnames > > So I guess we’ll just have to wait thatscatergets updated to see if there’s anything still broken on our side after that
Aaron Lun (17:41:42): > Good. I thought@Federico Marinisaw somerowMeansgoing on in the vignette.
Kevin Rue-Albrecht (17:42:26): > ow. didn’t check the vignette:sweat_smile:
Kevin Rue-Albrecht (17:42:49): > On a different note, check out that branch Check out this branch, launch an app (any app) and let me know what you thinkDo NOT merge:sweat_smile:
Kevin Rue-Albrecht (17:45:12): > We have > > data.frame( > Mean = rowMeans(assayMatrix), > Var = rowVars(assayMatrix), > Sum = rowSums(assayMatrix), > n_detected = rowSums(assayMatrix > min_exprs), > row.names = ri > ) > > in the vignette. > So I assume that those will need to be “switched toMatrix::” ?
Aaron Lun (17:46:59): > Probably not, actually.
Aaron Lun (17:47:14): > Well, it couldn’t hurt, but it’s not necessary for the vignette.
Aaron Lun (17:47:25): > It’s only necessary if someone copies and pastes the vignette for use with a sparse matrix.
Aaron Lun (17:47:39): > I guess it’s worth doing and addingMatrixto theSuggests:if it’s not already there.
Aaron Lun (19:06:03): > hey, how do I get ggplot to show me two plots side by side?
Aaron Lun (21:58:10): > nvm.
2019-05-13
Federico Marini (03:42:15) (in thread): > It was in there, in one of the advanced ones
Federico Marini (03:43:33): > We have therowMeanshere:
Federico Marini (03:43:38): > > Federicos-MBP:iSEE fede$ grep -rin 'rowMeans' vignettes/* > vignettes/basic.Rmd:124:rowData(sce)$mean_log <- rowMeans(logcounts(sce)) > vignettes/custom.Rmd:140: Mean = rowMeans(assayMatrix), > vignettes/custom.Rmd:182:rowData(sce)$mean_log <- rowMeans(logcounts(sce)) > vignettes/custom.Rmd:249: active <- rowMeans(assayMatrix[,ci$active,drop=FALSE]) > vignettes/custom.Rmd:253: saved <- rowMeans(assayMatrix[,ci$saved[[i]],drop=FALSE]) > vignettes/custom.Rmd:274: top <- head(order(rowMeans(abs(everything)), decreasing=TRUE), 50) > vignettes/custom.Rmd:331: in.subset <- rowMeans(logcounts(sce)[,columns]) > vignettes/custom.Rmd:332: out.subset <- rowMeans(logcounts(sce)[,setdiff(colnames(sce), columns)]) >
Federico Marini (03:44:01): > with themean_logcalls being the ones I was tackling, if I recall correctly
Aaron Lun (12:12:40): > I think if you throw in alibrary(Matrix)at the start it’ll automatically useMatrix::rowMeans, that might be the least intrusive change.
Federico Marini (12:18:33): > WithMatrixgoing to the Suggests: in the DESCRIPTION, I guess?
Aaron Lun (12:22:51): > Presumably, yes. And add a comment next tolibrary(Matrix)about supporting S4 matrices.
2019-05-14
Charlotte Soneson (11:30:58): > Here’s one thing that just showed up - we had a SingleCellExperiment object that was obtained bycbind-ing multiple objects from different samples, and due to the large number of data points, we downsampled. Now, since the downsampling chooses the “last” point in each bin (via!duplicated), we ended up with almost all subsampled points being from the same sample, and when we facetted the t-SNE by sample, the panels naturally looked very different, which doesn’t seem desirable. In the end, we shuffled the columns of our SCE object and things looked fine, but I’m not sure we can expect users to do that. Thoughts?
Kevin Rue-Albrecht (12:00:13): > The intuitive solution would be to shuffle the data points before running the downsampling > I was worried that we would have to inefficiently create a shuffled copy of thesce, but looking at where we do the downsamplinghttps://github.com/csoneson/iSEE/blob/4598b8c45e6684e23a3731188a0c1155a0d1bd85/R/plotting.R#L623it seems like we do it on theplot.dataused for plotting, which means that we don’t need to touch the originalsce. > How about we just add the following > > plot.data <- plot.data[sample(nrow(plot.data))] > > before the downsampling command (linked above)?
Charlotte Soneson (12:03:07): > I guess the problem with that is that it wouldn’t be reproducible by the code reported in the code tracker.
Aaron Lun (12:03:20): > set the seed, as we do for vipor calls.
Charlotte Soneson (12:04:36): > Right - I had forgotten that we actually did that.
Kevin Rue-Albrecht (12:10:47): > before randomising, check out the issue I just reported:https://github.com/csoneson/iSEE/issues/295
Aaron Lun (12:16:53): > Well, I have a talk to prep, so do your best.
Charlotte Soneson (12:29:51): > So theColorByis set before the downsampling - maybe some points should be “protected” from being excluded?
Charlotte Soneson (12:52:03): > I started pushing to adownsample_randomlybranch - have to go now, but I’ll think about this a bit more later.
2019-05-16
Charlotte Soneson (12:53:46): > I pushed a proposed solution to Kevin’s issue above to thedownsample_randomlybranch
Aaron Lun (15:56:41): > DRY
Kevin Rue-Albrecht (16:23:38): > > DRY > Is that a case of “don’t repeat yourself”? Care to elaborate?:grimacing:
Aaron Lun (16:27:11): > yes.
Aaron Lun (16:27:20): > see how you have two invocations to subsetby genes.
Aaron Lun (16:27:27): > no need for that.
Kevin Rue-Albrecht (16:45:36): > took me a second, but I see what you mean:https://github.com/csoneson/iSEE/blob/downsample_randomly/R/plotting.R#L629and the next line > it’s basically the same command,plusa little extra > now that Charlotte’s done 99% of the job, i can refactor that to avoid the ‘redundancy’ indeed
Kevin Rue-Albrecht (16:48:17): > To be honest, the Do-RY version didn’t shock me, as it’s probably easier to read. > That said, the rest of the code-generating part of iSEE is a whole network of optimised/refactored DRY stuff, so we might as well continue the philosophy that took us this far
Kevin Rue-Albrecht (16:50:32): > What do people think of adding a comment line in, like: > > downsample_cmds <- c( > "plot.data.pre <- plot.data;", > ---> "# Randomize data points to avoid a data set bias", <--- > "set.seed(100);", > "plot.data <- plot.data[sample(nrow(plot.data)), , drop=FALSE];", >
Aaron Lun (16:52:18): > It won’t shock you until you find a bugfix and only fix half of it.
Kevin Rue-Albrecht (16:55:10): > … a few weeks without heavy iSEE development and I already let me coding standards slip:wink:
Kevin Rue-Albrecht (17:08:19): > I’ve pushed a commit. Let me know whether I made it prettier or not:stuck_out_tongue:
2019-05-17
Aaron Lun (02:01:50): > well. a little. line 634 is still a monstrosity, and I don’t understand why you need a gsub.
Kevin Rue-Albrecht (05:48:35): > Thoughts about a potential panel type to plot a reduced dimension against other sample-level covariates? e.g. colData
Charlotte Soneson (05:51:48): > Sounds like it could be useful. I wonder if it could go in one of the current panels, i.e. instead of plotting a colData column on the x-axis, one could plot the redDim coordinates.
Kevin Rue-Albrecht (06:00:08): > Indeed. A new dropdown menu sounds more reasonable than a new panel type in this case
Kevin Rue-Albrecht (10:43:41): > FYI, i’ve seen Travis failing for unit tests. I’ll fix that this evening. I also need to reply to the RStudio who got in touch again to set up iSEE for the Shiny Showcase gallery. I’ve been updating the RStudio Cloud project over the day (they still need to iron out sessions crashing when installing packages)
Aaron Lun (11:14:13): > :+1:
Aaron Lun (11:14:28): > Will deal with the branch over the WE.
Kevin Rue-Albrecht (11:17:56): > Ok. > Out of curiosity, did anyone tried tried thekra-bugbranch yet?
Federico Marini (11:18:48): > I wanted to do that at the workshop but the connection was miserable enough to abort the git pull attempts
Federico Marini (11:19:08): > I’ll be home tonight, I promise to peek in:wink:
Kevin Rue-Albrecht (11:21:25): > FYI: It’s optionalized, so you’ll have to enable the bugs (never thought I’d say that:sweat_smile:)iSEE(..., bugs=TRUE)for a taste.
Kevin Rue-Albrecht (13:35:08): > I missed throwback Thursday by a day, but I just stumbled onto this old screenshot. I forgot we had those buttons to add new panels at some point ^^ - File (PNG): Screenshot 2018-01-31 16.21.06.png
2019-05-18
Aaron Lun (00:41:01): > y’knkow we have parameter types for lines 625 and 626.
Aaron Lun (00:41:12): > Encode the panel name and compare torow_point_plot_typesorcol_point_plot_types.
Aaron Lun (00:41:47): > Also don’t know whatprotect_coloy_bymeans.
Charlotte Soneson (03:58:04): > Fair point, thanks. Changed to > > is_color_by_sample_name <- param_choices[[.colorByField]] == .colorBySampNameTitle && .split_encoded(rownames(param_choices))$Type %in% col_point_plot_types > is_color_by_feature_name <- param_choices[[.colorByField]] == .colorByFeatNameTitle && .split_encoded(rownames(param_choices))$Type %in% row_point_plot_types >
Kevin Rue-Albrecht (12:26:23): > I’m curious why the workshop is failing Travis after@Federico Marini’s last update, it looks unrelated. > Travis now fails with an error that I typically see when trying to cache large objects in R markdown vignettes:https://travis-ci.org/kevinrue/Bioc2019-iSEE-workshop/builds/533253017That and the fact that it only crashes when deploying the pkgdown website. > Really not a key concern, but annoying that the error shows up now for no apparent reason.
Kevin Rue-Albrecht (12:55:25): > Yep. Seems like pkgdown doesn’t like cached large objects. It deploys again withcache=FALSE
Aaron Lun (13:09:37): > why don you just split_encoded once and store it in a variable?
Kevin Rue-Albrecht (13:42:54): > pushed
Kevin Rue-Albrecht (13:43:28): > Aaron “iSEE (your code)” Lun:wink:
Federico Marini (17:44:47): > :smile:good catch@Kevin Rue-Albrecht
2019-05-19
Aaron Lun (01:16:43): > your goddamn right I can see it.
Kevin Rue-Albrecht (08:51:56): > damn that’s pretty nice:https://gallery.shinyapps.io/ggtree/
Aaron Lun (23:42:17): > seems like another ggplot obscenity to me.
2019-05-20
Kevin Rue-Albrecht (04:32:50): > I got word from RStudio about the shiny contest prizes > > RStudio has annual workweeks where everyone gets together in person. Next week we’re all in Columbus, and so we planned to sort out all your prize goodies to be sent from there.
Kevin Rue-Albrecht (04:33:46): > I watched the first episode of the “designated survivor” recently and now can’t help wondering whether RStudio has one too:joy:
Federico Marini (04:38:41): > I have heard the concept of the bus factor
Federico Marini (04:38:48): > and had to think of Hadley for this
Federico Marini (04:39:11): > funny enough: after 3 weeks I gave this thought, they announced they hired Thomas Pedersen:slightly_smiling_face:
Kevin Rue-Albrecht (04:44:07): > Nice to think that RStudio’s contributing toward my wedding list, tho:stuck_out_tongue_winking_eye:
Federico Marini (04:44:31): > In terms of stickers to give away at the venue?
Kevin Rue-Albrecht (04:45:16): > Nah, I meant the mugs and t-shirts we’ve asked for
2019-05-23
Aaron Lun (19:19:13): > @Kevin Rue-Albrechtdid you end up getting a time to speak with the rstudio peeeps
Aaron Lun (19:19:58): > best also mention a better way to distinguish between a brush selection and a click. currently every attempt to brush also triggers a click, which is annoying.
Kevin Rue-Albrecht (19:34:10): > Haven’t heard back about the 30 min chat yet. I innocently chased them down about the goodies, and Curtis said they were planning to ship them today from Columbus.
Kevin Rue-Albrecht (19:36:11): > I’ll ping him another message specifically about the chat, to get some news, and in the process mention that I’m (supposed to be) out of reach from today to the 5 June (getting married in a week)
Aaron Lun (19:36:30): > That can’t be right.
Aaron Lun (19:36:34): > I haven’t received my invitation yet!
Aaron Lun (19:36:39): > Must have gotten lost in the mail.
Kevin Rue-Albrecht (19:38:03): > À friend cancelled because work sent him on a mission (army) today. His wife his still coming. I can ask if she’s interested to swap her +1
Kevin Rue-Albrecht (19:39:13): > Also. There’s a google docs linked somewhere where we’ve started a list of points that can be brought up during the RStudio call, if you wanna add yours
Aaron Lun (19:39:37): > lol
Aaron Lun (19:41:51): > forgotten where it was.
Kevin Rue-Albrecht (19:43:38): > that’s the one:https://docs.google.com/document/d/196tFRn4PpARh1leTWfGYnuG4v3r8TdkG1gRgZowYMYE/edit?ouid=101242403717063959562&usp=docs_home&ths=true - File (Google Docs): iSEE-shiny-contest-one-on-one
Kevin Rue-Albrecht (19:45:51): > On a separate note, during the flight to Italy today, I’ve written the code to add “reduced dimension” as a choice for the x-axis of sample based plots (colDataPlot, featAssayPlot). Branchkr-reddimUse the example apps athttps://gist.github.com/kevinrue/7d060d60eee1c7795b052eb48731c966to have a feel for it. Go straight to the last app.
Aaron Lun (19:47:19): > If you’re going to do that, you’ll want to have a rethink about the entire panel structure.
Kevin Rue-Albrecht (19:47:20): > I just really need to stop now, and won’t touch anything until 5 June. So if anyone wants to pick it up and polish it (update the docs, refactor, etc.) they’re more than welcome
Kevin Rue-Albrecht (19:49:21): > all i cared today is to get the prototype to work (the UI works, the plots update, the code tracker too), but there are probably some pieces I haven’t got to
Kevin Rue-Albrecht (19:52:45): > On that note, good night!
Aaron Lun (19:53:55): > STRAPLESS DRESS FTW
2019-05-24
Federico Marini (07:57:50): > Aaron for useR we should nerdify even more the presentation
Federico Marini (07:58:05): > How are you in 90s pop culture?
Federico Marini (07:58:35): > I already have britney spears for “knit me baby one more time” for reproducibility
Federico Marini (07:59:03): > And the Backstreet Boys for “I want [the plot] that way” for customizing
Federico Marini (07:59:27): > (the spice girls had way tooooo many nipples in their videos, I re-discovered recently:smile:)
2019-05-25
Aaron Lun (14:04:08): > How was I supposed to know?
Aaron Lun (14:04:14): > That the plot wasn’t right (yeah)
Aaron Lun (14:04:51): > I shouldn’t have let you go
Aaron Lun (14:05:09): > Now the colors are out of sight (yeah)
Aaron Lun (14:05:11): > Show me
Aaron Lun (14:05:14): > how you want it to be
Aaron Lun (14:05:19): > tell me baby
Aaron Lun (14:05:27): > cause I need to know, now, oh because
Kevin Rue-Albrecht (15:24:24): > My personal motivation for Shiny apps was more about collaborators who kept coming back for plots like: > > I gotta feeling that this plot’s gonna be a good plot > > That this plot’s gonna be a good plot > > That this plot’s gonna be a good, good plot > > > Let’s plot it, let’s plot it, let’s plot it, let’s plot it > > And plot it and plot it, let’s plot it up > > And plot it and plot it and plot it, plot it, plot it > > Let’s plot it, let’s plot it, let’s plot it
Aaron Lun (16:46:58): > Why are you still here? Aren’t you getting married?
Aaron Lun (17:01:38): > Maybe you could use that as your first dance song.
Kevin Rue-Albrecht (17:04:43): > > Why are you still here? Aren’t you getting married? > A combination of coordinating my family by 3+ modes of online communication and a bad habit of leaving Slack on:wink:
Kevin Rue-Albrecht (17:05:40): > > Maybe you could use that as your first dance song. > I’m sure I can do nerdier
Kevin Rue-Albrecht (17:15:01): > A personal favorite, even though I never managed to interpret all the lyrics:https://www.youtube.com/watch?v=UTby_e4-Rhg - Attachment (YouTube): Finite Simple Group (of Order Two)
Aaron Lun (17:23:15): > lol
Aaron Lun (17:23:48): > the lyrics are in the description, but if you mean “understand” the lyrics, then I think you’ll need a math PhD.
Kevin Rue-Albrecht (17:24:11): > I meant the second. So, you reassure me.
Aaron Lun (18:56:01): > Cleaned my toilet for the first time.
Aaron Lun (18:56:05): > Man, that wasn’t nice.
Aaron Lun (18:56:12): > Now my toilet water is blue.
2019-05-31
Federico Marini (03:03:28): > https://easystats.github.io/blog/posts/see_presentation/ - Attachment (easystats): The ‘see’ package: beautiful figures for easystats > The see package We have recently decided to collaborate around the new easystats project, a set of packages designed to make your life easier. This project encompasses several packages, devoted for instance to model access or Bayesian analysis, indices of model performance or visualisation. Without further ado, please let us introduce the latest addition to the easyverse; the see package! Essentially, see is the visualisation companion to other functions and packages in easystats.
2019-06-03
Kevin Rue-Albrecht (04:23:20): > Yop! > Just resurfacing now after the wedding frenzy (my family stayed for tourism a few extra days). > I’ll be catching up with emails and GH issues and stuff shortly, but just ping me if anything important came up while I was offline. > I’m flying back only the 5th so I’ll still be on and off the next couple of days.
Charlotte Soneson (04:24:29): > Hey, congratulations!:slightly_smiling_face:
Aaron Lun (13:52:37): > BOO
Aaron Lun (13:52:40): > bachelors FTW
Federico Marini (15:18:54): > I’m still one
Federico Marini (15:19:13): > Although I make up a fuzzy cluster
Aaron Lun (15:23:18): > you have kids, fed. THat’s even worse./
2019-06-04
Kevin Rue-Albrecht (04:26:41): > > what k-mean clustering has joined together, let no man put asunder - File (PNG): Unknown-4.png
Federico Marini (05:16:05): > I knew this would have come
Federico Marini (05:16:06): > https://www.biorxiv.org/content/biorxiv/early/2019/06/03/650176.full.pdf
Kevin Rue-Albrecht (06:28:58): > I just sent that to Curtis - File (PNG): image.png
Kevin Rue-Albrecht (13:10:10): > Note to self > Add a FAQ point about the error$ operator is invalid for atomic vectorsin the vignette about custom panels when users forget to setcustomSendAll = TRUE
Kevin Rue-Albrecht (13:11:06): > I got bit by that one again when adding the multi-DE example from the vignette into the workshop
Aaron Lun (13:52:39): > this 3d VR crap is the stupidest thing I’ve seen.
Federico Marini (18:04:31): > Agreed
Federico Marini (18:04:56): > As if I would really understand more of a manhattan plot if I navigate this in VR
2019-06-05
Kevin Rue-Albrecht (05:49:49): > New toy:https://twitter.com/10xGenomics/status/1136004311950004226 - Attachment (twitter): Attachment > We are excited to announce the release of our large-scale #immunology dataset. The dataset, using over 150k CD8+ T cells from 4 human donors, analyzed 44 specific antigens. We hope it enables a new way of exploring adaptive immunity. https://bit.ly/2Z56kJt https://pbs.twimg.com/media/D8Pl5oQXYAIGHvC.jpg
Kevin Rue-Albrecht (16:11:28): > :cry: - File (JPEG): Image from iOS
Aaron Lun (16:13:15): > WTF
Aaron Lun (16:13:31): > Really?
Aaron Lun (16:13:38): > Should have gotten them delivered to me.
Kevin Rue-Albrecht (16:14:07): > Anyway. Just got home. Unpacking
Federico Marini (16:21:58): > Is it for the RStudio goodies?
Kevin Rue-Albrecht (17:19:37): > looks like it. haven’t ordered anything else from the US recently
Kevin Rue-Albrecht (17:22:38): > Just ln case, looks like it’s not a scam:wink: - File (PNG): image.png
Kevin Rue-Albrecht (17:23:29): > I got one better “revised custom charges” - File (PNG): image.png
Kevin Rue-Albrecht (17:30:44): > I learn every day:https://www.parcelforce.com/sites/default/files/Customs%20leaflet.pdf
Federico Marini (17:37:17): > hmpfh, it sucks
Federico Marini (17:38:35): > Whatever you’ll have to pay, happy to split and fill in my share
Kevin Rue-Albrecht (18:01:47): > i still feel like i owe you a share of stickers:sweat_smile:
2019-06-07
Kevin Rue-Albrecht (03:19:48): > Not kidding. That’s the whole box content - File (JPEG): Image from iOS
Kevin Rue-Albrecht (03:26:19): > 4 mugs 2 tees
Kevin Rue-Albrecht (03:26:45): > Still the box label said content worth 200$
Federico Marini (03:40:57): > WUT
Federico Marini (03:41:43): > I can’t see the z-axis of the stickers, but they look like they had to scratch the bottom of the warehouses?:disappointed:
Charlotte Soneson (03:46:40): > Interesting also that the content is not very consistent with what we asked them for:woman-shrugging:- I mean, it’s fine, but it makes you wonder why they asked:slightly_smiling_face:
Federico Marini (03:47:01): > well said
Federico Marini (03:47:15): > gosh this is sad
Federico Marini (03:47:26): > At least the sizes are the ones we chose?
Federico Marini (03:47:43): > which leads to the next question, who fits in which
Charlotte Soneson (03:49:42) (in thread): > If the bookdown book is $120, that would be true…
Kevin Rue-Albrecht (03:53:01) (in thread): > I doubt it….:pensive:they said 40 tops
Kevin Rue-Albrecht (03:53:29) (in thread): > And I explicitly put it way down the priority list
Charlotte Soneson (03:53:40) (in thread): > Yep, I remember
Kevin Rue-Albrecht (03:54:19) (in thread): > To be fair he was honest from the start about sending according to availability. Still…
Charlotte Soneson (03:54:59) (in thread): > Sure - feels like the value should be the same though…ah well:slightly_smiling_face:
Kevin Rue-Albrecht (03:57:50): > Two S sizes. Sorry Fede and Aaron (who might be able to pay them a visit:yum:)
Kevin Rue-Albrecht (03:58:07): > Two « shiny » models btw
Federico Marini (07:01:35): > Can’t hide I am lil disappointed
Kevin Rue-Albrecht (07:31:45): > Yeah, me too. Not sure if it’s worth a message to Curtis. If I hadn’t paid the custom too, I would immediately let it go, but here… it’s a bit frustrating
Federico Marini (07:33:19): > I’m sure they can do the math:face_with_raised_eyebrow:
Federico Marini (07:33:50): > that’s far away from the 200$ (which in the end is most like a cookie jar worth)
Federico Marini (07:34:11): > All I wanted was a tshirt and shitloads of stickers TBH
Kevin Rue-Albrecht (07:35:49): > Looks like they invested more in ceramic than cotton
Federico Marini (07:36:23): > Not even if it was blown glass would cost that much, damn
Kevin Rue-Albrecht (09:14:27): > FWIW I did send a message about it to Curtis. I’m not expecting them to do anything about it, but whoever was in charge of filling the parcels, at least he knows
Aaron Lun (11:16:44): > Hm.
Kevin Rue-Albrecht (11:21:00): > > Hm. > That sums it up.
Aaron Lun (11:25:14): > I’ll cover my bit of the 47 by buying you lunch the next time I see you.
Kevin Rue-Albrecht (13:23:37): > “All the stickers we can find” - File (JPEG): Image from iOS
Aaron Lun (13:26:05): > lol. Gee, that’s not much for 47 GBP.
Federico Marini (17:17:39): > 2 stickers per type?
Federico Marini (17:17:43): > jeeeez
Federico Marini (17:18:03): > find -r, guys…
Kevin Rue-Albrecht (17:18:36): > I scratched at the bottom of the parcel, believe me:sweat:
Federico Marini (17:19:30): > Ouuuh. You know, that was actually all I really wanted:stuck_out_tongue:
Federico Marini (17:20:04): > I’ll be in Toulouse and will comply there face to face:smile:
Kevin Rue-Albrecht (17:22:44): > “complain” I assume?:wink:
Federico Marini (17:23:14): > yup
Federico Marini (17:23:20): > I’m tired:smile:
Federico Marini (17:27:45): > Just checked the program for useR
Federico Marini (17:27:55): - File (PNG): image.png
Federico Marini (17:28:15): > “Hey mum, look, I anticipated the topic of his keynote in 2016”:smile:
Federico Marini (17:28:44): > https://github.com/federicomarini/pcaExplorer/commit/e44696db90df51ce586dec7eff9b2d2ee95b8815
Federico Marini (17:37:14): > Hey@Aaron LunI’ll have Michael just before me!
2019-06-08
Aaron Lun (02:17:22): > lol.
Aaron Lun (02:17:26): > It’s a shame it’s not in SF.
Aaron Lun (02:17:40): > Nowadays, I don’t go places unless I fly business.
Aaron Lun (02:17:43): > And you can quote me on that.
Kevin Rue-Albrecht (12:11:07): > Just read the email from Levi about workshop guidelines and deadlines. > I think we’re pretty much set (as shown by the lack of messages in our direction in the “iSEE” GH issue) > Something that might benefit more than 1 pair of eyes: “to use citations and footnotes instead of or in addition to hyperlinks”. > I think our “links” are mostly to package landing pages, but I remember links to the iSEE vignettes that might need an update. > Thoughts?
Charlotte Soneson (14:01:34): > Ok, so I guess this means that they don’t wantonlyhyperlinks - I’ll go through it again and look for this, and see if there’s anything else that could be added/fixed.
2019-06-12
Kevin Rue-Albrecht (05:01:47): - File (PNG): image.png
Kevin Rue-Albrecht (05:02:02): - File (PNG): image.png
Kevin Rue-Albrecht (05:09:03): > Curtis has always been super helpful. I guess they’re just generally optimistic about things and catch up with reality as it happens
Kevin Rue-Albrecht (05:10:21): > To be fair, custom taxes on arrival would be among the last things I would think of when sending a gift to someone
Federico Marini (05:16:49): > Fair enough, I guess
Charlotte Soneson (05:19:36): > Yeah. It’s not really clear what he means by “I thought we were totally clear with items less than $200” though - clear for the customs? I guess they were the ones indicating the worth on the package?:thinking_face:
Kevin Rue-Albrecht (05:21:49): > Yeah. Not clear either about his meaning, I’ve sent another message with my PayPal details but also gently asking how they’ll calculate the amount to send: the missing $80 and/or the custom taxes. > Indeed, @rstudio.com (email written on the parcel label) declared the package content to be worth $200 on the label
Kevin Rue-Albrecht (05:23:13): > According to the letter, customs are allowed to open the parcel to inspect content and value, but the way it arrived, it looked pretty “original packaging” to me, so I guess they took the label at face value
Federico Marini (05:38:42): > I just want to point out that I never planned to do this for the prize
Federico Marini (05:38:43): > yet
Federico Marini (05:39:01): > “We are clear it is less than 200$”
Federico Marini (05:39:03): > vs
Federico Marini (05:39:08): - File (PNG): image.png
Federico Marini (05:39:39): > I am not a native speaker, so I’ll stop complaining here. But this is not the same thing:stuck_out_tongue:
Kevin Rue-Albrecht (05:46:55): > Well, as Charlotte, I honestly don’t know what “with items less than $200” meant in his latest message. > Fede’s screenshot of the original announcement is clear that $200 is a maximum. What isn’t clear is how much lower below $200 it could drop.
Federico Marini (05:51:11): > low:disappointed:
Federico Marini (05:51:30): > All hex sticker they can find STA CIPPA:smile:
Kevin Rue-Albrecht (05:52:20): > https://forum.wordreference.com/threads/di-sta-cippa.1407734/ > > It means cheap, shitty, second-rate. - Attachment (WordReference Forums): di sta cippa > Buongiorno, Sta mattina ho letto una frase con la parola “stacippa” e non l’ho trovata nel dizionario. Ecco la frase: “che ci vuoi fare, è il destino di noi supereroi (di stacippa)….” Sapete cosa vuol dire? Grazie mille!
Kevin Rue-Albrecht (05:52:44): > I can “impress” someone when I get home tonight:stuck_out_tongue:
Federico Marini (05:52:58): > ehehe
Federico Marini (05:53:26): > Charlotte has some reference at home as well
Federico Marini (05:53:41): > He could I guess even make it more colourful
Charlotte Soneson (05:53:49): > Most likely
Federico Marini (05:54:33): > I can’t avoid thinking we really are in the use case of “Rstudio went all the way into the Shiny contest, and all I got is 2 S-sized (hopefully not lousy) tshirts”
Federico Marini (09:30:42): > FellowiSEErs, I feel the temptation to try and submit a science slam on iSEE is growing strong
Federico Marini (09:31:32): > The GMDS Conference will have such a spot, and I was thinking to put together a mini abstract
Federico Marini (09:32:51): > Probably I feel overconfident after the many stupid gifs we came up with:slightly_smiling_face:
Charlotte Soneson (09:34:00): > Not really my strong side, but if you feel up for it, go for it:slightly_smiling_face:there’s nothing to lose!
Federico Marini (09:34:14): > yup
Federico Marini (09:36:26): > 1. “iSEEing is believing” + something about viz, or > 2. Interactivity and reproducible science: making data analysis better while getting~lazy~efficient?
Federico Marini (09:36:53): > (or of course, better ideas are welcome)
Charlotte Soneson (13:18:36): > I did indeed receive an extensive exposition of this exquisite expression:stuck_out_tongue:https://community-bioc.slack.com/archives/C8BJLSP8T/p1560333221049500 - Attachment: Attachment > He could I guess even make it more colourful
Kevin Rue-Albrecht (14:34:14): > Woooot Travis is happy again. I’m kicking in all the latest failed builds:grin:
Charlotte Soneson (14:34:30): > yaaaayyy:tada::party_parrot:
2019-06-13
Kevin Rue-Albrecht (14:51:28): > Curtis repaid me the custom taxes by Paypal. Forget anything about contributing to that:slightly_smiling_face:
Charlotte Soneson (14:52:01): > Ok, good:slightly_smiling_face:
Federico Marini (14:53:51): > Good:slightly_smiling_face:
Federico Marini (14:54:23): > I did assemble the abstract for the science slam in the end
Federico Marini (14:55:03): > A phone call with my former boss gave me the final convincement points
Charlotte Soneson (14:55:20): > :+1:
Federico Marini (15:02:41): > I told him I was 95/100 convinced (sorry Charlotte for canceling out the couple of points gained this morning), and started describing the intention. Halfway through the first sentence he said “you should just do it”:smile:
Charlotte Soneson (15:05:23): > :smile::+1:
Federico Marini (15:06:35): > Now, I don’t know whether he was in a meeting, but that was a quick way to convince
Federico Marini (15:19:49): > Shameless public display of happiness:https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2879-1 - Attachment (BMC Bioinformatics): pcaExplorer: an R/Bioconductor package for interacting with RNA-seq principal components > Principal component analysis (PCA) is frequently used in genomics applications for quality assessment and exploratory analysis in high-dimensional data, such as RNA sequencing (RNA-seq) gene expression assays. Despite the availability of many software packages developed for this purpose, an interactive and comprehensive interface for performing these operations is lacking. We developed the pcaExplorer software package to enhance commonly performed analysis steps with an interactive and user-friendly application, which provides state saving as well as the automated creation of reproducible reports. pcaExplorer is implemented in R using the Shiny framework and exploits data structures from the open-source Bioconductor project. Users can easily generate a wide variety of publication-ready graphs, while assessing the expression data in the different modules available, including a general overview, dimension reduction on samples and genes, as well as functional interpretation of the principal components. pcaExplorer is distributed as an R package in the Bioconductor project ( http://bioconductor.org/packages/pcaExplorer/ ), and is designed to assist a broad range of researchers in the critical step of interactive data exploration.
2019-06-18
Federico Marini (16:49:49): > I’m updating the package to respect thedata(allen)deprecation
Federico Marini (16:49:59): > If noone has already started that yet
Kevin Rue-Albrecht (16:52:19): > nope, i’ve got 2 ‘active’ branches but neither deals with that
Federico Marini (16:53:09): > k
Kevin Rue-Albrecht (16:53:09): > 1. halloween feature > 2. allowing a dimensionality reduction component on the X-axis of sample-level plots
Federico Marini (16:53:20): > 1. I’m ready:smile:
Federico Marini (16:53:26): > DRACARYS
Federico Marini (16:53:29): > gimmethat
Federico Marini (16:53:45): > I feel it, Kev. You want to have the fireworks for Bioc2019
Federico Marini (16:53:51): > no need to hide it
Kevin Rue-Albrecht (16:53:57): > loool … haven’t found a dracarys video to overlay yet
Federico Marini (16:54:54): > this one full screen?https://www.google.com/imgres?imgurl=https%3A%2F%2Fmedia1.tenor.com%2Fimages%2F670dcf40a997847ffc1204ad7e971e68%2Ftenor.gif%3Fitemid%3D14128677&imgrefurl=https%3A%2F%2Ftenor.com%2Fview%2Fdrogon-dracarys-dragon-breaths-fire-got-game-of-thrones-gif-14128677&docid=55x94K8823LvMM&tbnid=EY5WgBe9EmJD0M%3A&vet=10ahUKEwi11ZOG9fPiAhWkuXEKHWKtDYMQMwhQKBEwEQ..i&w=268&h=228&client=safari&bih=960&biw=1640&q=dracarys%20gif&ved=0ahUKEwi11ZOG9fPiAhWkuXEKHWKtDYMQMwhQKBEwEQ&iact=mrc&uact=8
Kevin Rue-Albrecht (16:55:27): > I’ve had a cursory look at the JS code of the bugs, but in any case that requires image sprites to be drawn, and I’m notthatskilled
Federico Marini (16:55:30): > a modal dialog wildly opening:smile:
2019-06-19
Kevin Rue-Albrecht (07:20:36): > FWIW, I waste some time last night trying to get that one to work:https://joaopereirawd.github.io/animatedModal.js/ - Attachment (joaopereirawd.github.io): animatedModal.js > animatedModal.js is a jQuery plugin to create a fullscreen modal with CSS3 transitions. You can use the transitions from animate.css or create your own transitions.
Kevin Rue-Albrecht (07:29:04): > I gotta say, RStudio is getting funky these days:smile: - File (PNG): image.png
Kevin Rue-Albrecht (07:29:54): > It’s the second time this week that opening RStudio itself does that (I’m not talking about a pop-up sub-window like git)
Charlotte Soneson (07:31:17): > I have had the same problem - closing and re-opening usually works…
Kevin Rue-Albrecht (07:32:30): > Absolutely. I was just curious how frequent that was, aside from me. Probably a clunky version, I’ve updated relatively recently
Federico Marini (09:41:32): > I have it since a while when I use projects from mounted folders
Kevin Rue-Albrecht (11:02:09): > @Federico Marini, you’re usually good at keep track of the various iSEEverse weblinks, is there an obvious one that I forgot?https://kevinrue.github.io/slides/bioc2019-workshop/#/11
Federico Marini (11:57:35): > iSEE paper is missing
Federico Marini (11:57:38): > from LTLA
Federico Marini (11:57:53): > funny enough, each of us has one:smile:
Kevin Rue-Albrecht (11:58:44): > there we go, I did wonder whether Aaron has a repo: he has the source of the article
Kevin Rue-Albrecht (12:13:16): > Just looking through the repo… we haven’t mergeddownsample_randomlyyet? Looks like it was building before the wedding. Not sure what’s keeping ushttps://github.com/csoneson/iSEE/commits/downsample_randomly
Kevin Rue-Albrecht (12:13:59): > boy my head hurts from switching between projects
Charlotte Soneson (12:15:20): > Good catch. I don’t think there is anything keeping us.
Kevin Rue-Albrecht (12:16:37): > I was just cleaning my local branches. Good ofgitto warn about stuff not merged tomaster
Kevin Rue-Albrecht (15:57:36): > The ‘discreet’ result of?iSEEWorkshop2019on the AMI of Bioc2019:
Kevin Rue-Albrecht (15:58:06): - File (PNG): Screenshot 2019-06-19 20.56.35.png
Kevin Rue-Albrecht (15:58:39): > :sweat_smile:
Charlotte Soneson (16:02:04): > :upside_down_face:
Kevin Rue-Albrecht (16:02:28): > Fair to say that one can’t confuse the help page with another package
Kevin Rue-Albrecht (16:04:03): - File (PNG): image.png
Federico Marini (19:25:24): > eheeh
2019-06-20
Kevin Rue-Albrecht (16:17:31): > announcement:tada:1.5.2 pushed upstream
Kevin Rue-Albrecht (16:38:02): > Among the pretty easy things that can be done, I’m just thinking that there could be a dropdown with the various ‘default’ themes of ggplot2: > > theme_grey, theme_bw, theme_linedraw, theme_light, theme_dark, theme_minimal, theme_classic, theme_void > > without going into the detailed settings within each theme of course, but I guess it could help for some point coloring schemes, I suppose
2019-06-22
Federico Marini (03:28:28): > no problem to anticipate with the upcoming release of cowplot?
Aaron Lun (04:07:28): > I’m going to sleep. ¯*(ツ)*/¯
Kevin Rue-Albrecht (04:09:36): > Boarding gate next to right next to mine is SF… next time@Aaron Lun!
Kevin Rue-Albrecht (04:11:29): > Btw, i don’t know why exactly, but since the iSEE slowdown I can’t work that late any more . Normal person life kicking in?:sweat_smile:
Charlotte Soneson (04:11:54): > You’re getting old:wink:welcome to the club
Charlotte Soneson (04:12:14): > Safe travels!
2019-06-23
Federico Marini (08:15:00): > new_allenis merged. Whenever you have time@Kevin Rue-Albrecht, could you push to upstream?
Kevin Rue-Albrecht (08:15:37): > I’ll do now
Federico Marini (08:16:58): > supi, thanks
Federico Marini (08:17:04): > Flight went well?
Federico Marini (08:17:32): > I’m the only iSEEer on european soil as of now:disappointed:
Kevin Rue-Albrecht (08:18:55): > Done
Kevin Rue-Albrecht (08:19:26): > Somebody’s gotta guard the house:house_with_garden:
Federico Marini (08:19:31): > Will the workshops be streamed as far as you know?
Federico Marini (08:19:41): > :woofwoof:
Federico Marini (08:20:11): > I’ll be sitting on the porch with my Moscow mule trying to survive the heat wave
Kevin Rue-Albrecht (08:20:15): > Flight went perfect. Then I stayed out to catch up with an old friend last night. Beat the jet lag like a boss
Federico Marini (08:20:32): > :dealwithit-parrot:
Kevin Rue-Albrecht (08:20:42): > I don’t know anything about streaming this year
Federico Marini (08:23:58): > I’ll stay woke to see if anything gets posted
Kevin Rue-Albrecht (21:51:34): > Check that out, I found a Mexican beer about “el iSEE” ! - File (JPEG): Image from iOS
2019-06-24
Federico Marini (03:23:29): > Baron Kevin was in town
Federico Marini (03:23:32): > Oh God
Federico Marini (03:23:40): > It is one letter away
Federico Marini (03:23:43): > Kevin Baron
Federico Marini (03:23:46): > Kevin Bacon
Federico Marini (03:24:06): > dunno if it counts towards lowering your Bacon-Erdös number
Stephanie Hicks (10:31:51): > Great job@Kevin Rue-Albrecht! - File (JPEG): Image from iOS
Kevin Rue-Albrecht (10:34:58) (in thread): > Thanks!!
2019-06-26
Federico Marini (12:16:29): > Go go go iSEE team:person_doing_cartwheel:
Federico Marini (12:17:33): > Blaze minds as you can:slightly_smiling_face:
Kevin Rue-Albrecht (12:58:37): > Robert Castello suggested we advertised the voice recognition as ‘accessibility feature’
2019-06-29
Aaron Lun (23:37:35): > lol -30% coverage. Kevin’s going to throw a fit.
2019-06-30
Federico Marini (03:28:22): > Aaron what the hell:stuck_out_tongue:
Federico Marini (03:28:29): > I did not test it yet
Federico Marini (03:28:38): > but it is a nice way to wake up to
Federico Marini (03:29:09): > The feature requestress (does it conjugate like this?!) as well:slightly_smiling_face:
Federico Marini (03:29:43): > This change threw some fuel in a firey idea I had
Federico Marini (03:29:58): > some kind of demo “for anything”
Federico Marini (03:30:19): > We’d need to think of privacy, security, and stuff
Federico Marini (03:30:29): > but what if we had a domain likeiSEE.it
Federico Marini (03:30:39): > (or .io, FWIW)
Federico Marini (03:31:27): > Maybe one way would be with a repo spawning a binder instance - but dunno if that works for shiny apps
Kevin Rue-Albrecht (04:58:04) (in thread): > iSEE what you did there:smirk:
Kevin Rue-Albrecht (05:01:18): > I just landed from the overnight flight, si I’m not promising anything today, but I’ll have a look to assess the situation over the week. Unit tests served us well so far :)
Kevin Rue-Albrecht (07:23:26): > Arf.. Being on the bus back to Oxford, I have the ‘chance’ of realizing that the unit tests rely on a working internet connection now, even if theallendataset is already cached locally: > > > test_check("iSEE") > Loading required package: scRNAseq > Loading required package: scater > Loading required package: ggplot2 > Error in curl::curl_fetch_memory(url, handle = handle) : > Timeout was reached: Operation timed out after 10002 milliseconds with 0 out of 0 bytes received >
Kevin Rue-Albrecht (07:24:43): > As proven here, I do have an internet connection, butcurl_fetch_memoryfails frequently (roughly 8 out of 10)
Kevin Rue-Albrecht (07:44:09): > mmmh.. with respect toiSEE-observers.Rmost of those functions have asessionargument which is used in non-optional parts of the functions. I guess I’ll just mark the file for “no coverage”. Ah.. I see, it’s all code from the shiny server anyway, so it was under “no coverage” before already.
Kevin Rue-Albrecht (14:12:09): > @Aaron Luni’ve moved a few functions around inobservers.R, to separate the ones that don’t have asessionargument (and thus should be testable)
Aaron Lun (14:12:25): > There’s not many that don’t have a session argument.
Kevin Rue-Albrecht (14:13:15): > Although Travis/Codecov are happy again, there are a few things that I should be able to cover in the next few days (we need time to interactively test the updated app anyway)
Kevin Rue-Albrecht (14:14:58): > Flying back East is more painful. Fighting to stay up another couple of hours
Kevin Rue-Albrecht (14:24:53): > So,iSEE()+ upload an RDS file myself works for me. It takes a fair amount of time, but then again, at least it works.
Aaron Lun (14:26:27): > I don’t think it takes any longer than initializing the app directly, it’s the same observers that are getting set up.
2019-07-01
Kevin Rue-Albrecht (16:41:40): > @Federico Marinidoes your HTML trick in the README work on all web browsers? I suppose that most readers will understand that clicking the arrow will show the hidden content, and I admit that the README is otherwise verbose (Aaron made a comment about that a while back), but I still wonder whether I like it. Perhaps it’s just about rephrasing > > The user interface ofiSEEweb-applications currently offers the following features: > Something like adding “(click to expand)“?
Kevin Rue-Albrecht (16:47:44): > Or it could be because the collapsed part includes section headers of the same level as## Functionalities
Kevin Rue-Albrecht (17:13:32): > I’ve pushed a proposed update onneater_readme. Let me know.
Kevin Rue-Albrecht (17:15:20): > On a separate note, to comment on my recent renaming ofiSEE-observers.Rtoobservers, I can’t remember for what historical reason we haveR/files callediSEE-main.RandiSEE-extras.R. I believeiSEE-pkg.Ris a standard name automatically generated bydevtoolsto store the?iSEE-pkgman page. But aside from that, all other files aren’t prefixed byiSEE
Aaron Lun (17:17:05): > because there was a main app, and then we had extra functions that we split out of the main app, but then everything in the package was technically an extra function, so we distinguished the ones we split out withiSEE-extra.R.
Kevin Rue-Albrecht (17:19:24): > Oh. So you’d argue that it’d be better to re-rename toiSEE-observers.R. Not that it’s crucial to anything, but if I follow your logic, the observers are pretty much something split right out of the main app too
Aaron Lun (17:19:54): > ¯*(ツ)*/¯
Aaron Lun (17:19:57): > Don’t really care.
Kevin Rue-Albrecht (17:20:04): > Meh. Same here
Kevin Rue-Albrecht (17:20:21): > Too lazy to change it back.
Federico Marini (17:50:10) (in thread): > Theneater_readmestarted out mostly as a way to try it live on a repo
Federico Marini (17:50:27) (in thread): > I did not change any text on purpose
Federico Marini (17:50:58) (in thread): > and yes, there might be better ways of folding - I mean better spots to start it
2019-07-04
Kevin Rue-Albrecht (03:59:31): > Brief mention of iSEE:https://helenajambor.wordpress.com/2019/07/02/happy-birthday-vizbi/ - Attachment (helena * jambor): Happy Birthday VIZBI > Note: This was published as guest blog at EMBL events > 10 years after it all started, VIZBI came back to its original stomping grounds, the ATC at EMBL in Heidelberg. As its name suggests, VIZBI “Visualizing Biological Data” is a blend of several worlds. Of biology, with its long history in visualizations that goes back to Ancient Greek text books, and of art and scientific illustration. > Venn diagram of VIZBI disciplines: microscopy and EM data, transcriptomics and computer science. (Note: a 5-circle Venn cannot show all possible overlaps, which is fully intended here) > > VIZBI is also inseparable from computer science and its tools to transform big data into human readable entities. And finally, VIZBI incorporates concepts of design and visual perception to make visualizations engaging and enlightening. > Highlighting spectacular biological images > At VIZBI 2010, microscopic images were omnipresent. Back then, I was embarking on my postdoc project, a large-scale microscopy screen of RNAs in cells. My memories tell me that this was the main focus of the conference. Indeed, a quick check of the 2010 program confirms that almost the entire community of light sheet microscopy and image processing were in attendance at the first ever event. > VIZBI 2019 continued to highlight spectacular biological images. A phenomenal augmented reality installation showed them in 3D, EM-tomography simulations by Peijun Zhang animated the 64-million atoms assembling into HIV particles, and Lucy Collinson shared the high numbers of high-resolution EM data collected at the Francis Crick Institute. This large amount of data is annotated with the help of amateurs, for example in their citizen science project at the Zooniverse “Etch a cell”. > Colourful confocal images or images of tissues also provided the inspiration to many works of illustrators on display that combined science and art, for example the double win of best poster and best art to a depiction of tubulin in a mitotic spindle by Beata Mierzwa @beatascienceart, a hugely talented artist and scientist (who also sells cool cytoskeleton-printed leggings and mini-brain organoid dresses). > Data visualization > At VIZBI 2019, visualizations of data – as opposed to images – gained a much more prominent spot. All keynote speakers were from the technology side. Hadley Wickham presented the history of ggplot2. Ggplot2 (and yes, there once was a ggplot1!) is the R universe for visualizing pretty much everything that comes in numbers and is now merged into the tidyverse. Being a visualization talk, all slides were themselves beautiful, I love the tidyverse playfully represented as stars of our universe! The second keynote was by Janet Iwasa who presented her animation work that heavily relies on 3D and computer graphics software used for animation films. Instead of earning her money in the film industry, she decided to put it to good use for biology. Janet first used her skills in her PhD project to visualize motor proteins “walking” along the cytoskeleton, and these days produces Oscar®-worthy movies showing biology, such as the origin of life or the life cycle of HIV. And everyone take note: all her films start as a storyboard on paper, which is what I teach as good practice for all visualization designs. > Making the invisible visible > The third keynote was by Moritz Stefaner, a data designer who is enticed by biological data but appalled by the time-scales in biological projects (too long!). Luckily, he hasn’t given up on us just yet, and keeps producing phenomenal visualizations. For example, showing absence and loss is notoriously hard, but Moritz found a beautiful way to make the invisible visible in his designs for “Where the wild bees are” with Ferris Jabr for Scientific American. > Making absence visible, a project by Keynote speaker Moritz Stefaner. Photo: H.Jambor > Moritz left us hungry for more when also showing his data-cuisine project, that visualizes data about food and turns food into data: the number of berries picked in Finland become a layered dessert, and common causes of death are encoded as praline fillings – you never know which one you’ll get! (Luckily this was with Belgium pralines, so all deaths are sweet.) > Feedback wanted! > Visualizations of data were in the spotlight of many other projects too. This is of course owed to the many possibilities of large-scale methods that swamped biology with data in recent years: RNAseq, inexpensive genome sequencing, mass-spec at fantastic scales, robotics driven biochemistry and medicine, image processing that turns images into insights by quantifying signals and so on. RNA sequencing, for example, fuelled Susan Clark’s project tracing methylations in cancer, Phillippe Collas’ ambitious endeavour to understand 3D genome architecture, and is empowered by Charlotte Soneson’s “iSEE” software to interactively analyse data from high throughput experiments and the project of Kirsten Bos tracing human pathogens back thousands of years by sequencing tiny dental samples. And of course, of the biggest data projects in biology is the ENSEMBL genome browser, which was officially released as pre-alpha version VIZBI (check it out: ), the very approachable Andy Yates and his team are looking for feedback! > Technical Challenges > Visualizations of high-dimensional datasets are not without problems. The technical challenges were addressed by David Sehnal who showed computational infrastructure to visualize protein structures (MolStar). The mathematical problems of dimensionality reductions were a topic of Wolfgang Huber’s talk, and a tool to visualize, and thereby find(!), batch effects, “proBatch”, was presented in the flash talk by Jelena Čuklina (they welcome beta-testing by users!). Teaching science visualizations, I often see a great need to discuss ethical and practical aspects. Critically assessing limitations and challenges of scientific visualizations might be a topic to be expanded in future, when VIZBI enters its second decade. This should be coupled with visual perception research, after all, we are no longer limited by computational power, but rather by what our eyes and brains can comprehend (see Miller 1956). > Flash talks > “Data dancing” © Alex Diaz > > Speaking of flash talks: the conference organisers did such a great job in highlighting every single one (!) of the posters by one-minute talks. I tremendously enjoyed them, admittedly in part because I have a short attention span. Among the talks and art was also “Data dancing” by Alex Diaz. He showed that art and beauty can also be found in statistics and numbers blossoming like flowers across the page. On that note: see you next year in San Francisco! > > P.S. Many more highlights I was unable to cover here. Check for all posters and slides of the flash talks, check #VIZBI on twitter and my public collection of participants twitter handles ().
2019-07-05
Federico Marini (03:33:17): > nice, a long lasting memory post-conference
2019-07-06
Aaron Lun (12:37:52): > Had an earthquake last night.
Federico Marini (13:55:53): > all good in the hood now?
Aaron Lun (14:07:36): > My blinds were swinging and I couldn’t figure out why because I don’t have any open windows.
Aaron Lun (14:07:45): > Eventually I realized it was an earthquake.
Aaron Lun (14:07:51): > I just threw my hands up and enjoyed the ride.
Stephanie Hicks (21:00:30): > :scream:
Aaron Lun (21:14:23): > I’m on the 22nd floor. If it’s strong enough to bring down the building, I’m fucked.
Aaron Lun (21:14:34): > Nothing to do but to chill.
Stephanie Hicks (21:15:11): > 22nd!
Stephanie Hicks (21:15:15): > how many floors are there?
Aaron Lun (21:15:34): > 22.
Aaron Lun (21:15:39): > Penthouse corner suite
Aaron Lun (21:15:44): > that’s what selling out gets you.
Stephanie Hicks (21:16:27): > waaaaat
Stephanie Hicks (21:17:07): > well, in that case, yes enjoy the ride.:wink:
Aaron Lun (21:17:24): > hopefully more of these earthquakes should push the rent down.
Aaron Lun (21:17:37): > If we had one every week, maybe no one would want to live here.
Aaron Lun (21:17:42): > And then I’d get supercheap rent.
Stephanie Hicks (21:19:01): > and by supercheap, i somehow think it would still be more expensive than most places in the US lol
Aaron Lun (21:22:44): > If I can get it under 3000/mo, I’d be happy.
2019-07-10
Federico Marini (10:17:22): > Hi iSEEers, quick report on the situation of the achievements: - File (JPEG): 20190709_123746.jpg
Federico Marini (10:17:41): - File (JPEG): 20190710_111438.jpg
Federico Marini (10:17:53): - File (JPEG): 20190710_155619.jpg
Federico Marini (10:18:12): > I still have to learn how tonothave the fanboy face on
Federico Marini (10:18:21): > but I promise to work on that:wink:
Stephanie Hicks (10:18:34): > :camera:
Federico Marini (10:20:14): > I got to talk to Curtis and Joe re: contest prize and co. > In brief: > - there seems to be a second package of goodies coming > - the 1:1 meeting will still take place, we just have to reschedule > - I asked Joe specifically about lasso in shiny, he said Carson Sievert (plotly’s papi) will be onto Shiny and (verbatim) “he’s the best person that could work on that feature”
Charlotte Soneson (10:29:05): > :100:
Federico Marini (10:37:59): > From Jim (Hester) I heard travis has been bought by some other company
Federico Marini (10:38:11): > and he is exploring a plan b “just in case”
Federico Marini (10:38:52): > (https://blog.travis-ci.com/2019-01-23-travis-ci-joins-idera-inc) - Attachment (blog.travis-ci.com): Travis CI joins the Idera family > When we started working on a Continuous Integration solution back in 2011, it was hard to imagine what Travis CI would become. Years later, we’re still continuously working to make our community an…
Aaron Lun (11:57:54): > what is this? useR?
Kevin Rue-Albrecht (12:43:10): > Yes
Kevin Rue-Albrecht (12:43:47) (in thread): > Nice to see that I’m not the only OCD person who aligns their stickers to perfection!
Kevin Rue-Albrecht (12:47:58) (in thread): > Hadley’s smile: “someone get me out of here:grimacing:”
Aaron Lun (12:48:03): > Think Mike is there.
Kevin Rue-Albrecht (12:48:04) (in thread): > :stuck_out_tongue_winking_eye:
Kevin Rue-Albrecht (13:23:20): > While we’re on the subject of pictures with celebrities - File (JPEG): File from iOS
Stephanie Hicks (13:24:54): > :joy:
Kevin Rue-Albrecht (13:24:58): > Which makes me realise that I talked to him about iSEE but didn’t give him any sticker.
Aaron Lun (20:01:56): > <!channel>Comments on #299? Otherwise we should merge.
2019-07-11
Federico Marini (02:19:00) (in thread): > I also see a color gradient here
Federico Marini (02:19:12) (in thread): > I caught him cold-blodded:smile:
Federico Marini (03:36:36): > Joe Cheng just dropped the bomb
Federico Marini (03:36:37): > https://rstudio.github.io/shinymeta/ - Attachment (rstudio.github.io): Record and Expose Shiny App Logic using Metaprogramming > Provides functions that make it easier to construct Shiny apps that can dynamically construct reproducible R scripts.
Federico Marini (03:39:46): > tidyeval is going to be in there as well
Charlotte Soneson (03:58:08): > Was following online:slightly_smiling_face:- this looks cool. It wasn’t immediately obvious to me that it would solve everything that we need to do in iSEE (like, figuring out which order to return the code based on the dependencies between the panels), but it looks promising!
Federico Marini (04:01:02): > Probably iSEE per se is fine
Federico Marini (04:01:10): > but the next one can benefit right away of this
Kevin Rue-Albrecht (04:01:19): > Neat! Didn’t reference iSEE as source of inspiration ?:cry:
Federico Marini (04:01:27): > (at least, not really, for Bioc, till it is on CRAN)
Federico Marini (04:01:39): > > Neat! Didn’t reference iSEE as source of inspiration ?:cry: > check twitter
Federico Marini (04:01:45): > he did twice:smile:
Federico Marini (04:02:00): > the first time was damn too fast to fire up the cam
Federico Marini (04:03:12): > If you want to see it for a third time
Federico Marini (04:03:14): > go tosli.do
Federico Marini (04:03:20): > enter useR2019
Federico Marini (04:03:25): > post a question
Federico Marini (04:03:30): > and upvote it to hell:smile:
Charlotte Soneson (04:07:13): > It’s about a third intohttps://youtu.be/5KByRC6eqC8:slightly_smiling_face:(can’t put timestamps since it’s live:stuck_out_tongue:)
Federico Marini (04:09:33): > didn’t post it here for that reason, but thanks C
Kevin Rue-Albrecht (04:35:17): > Just coming out of post office with more t shirts. Say thanks to Curtis! - File (JPEG): Image from iOS
Federico Marini (04:36:30): > Will definitely do! Great
Kevin Rue-Albrecht (04:37:06): > He sent them himself
Kevin Rue-Albrecht (04:37:22): > … or his PA maybe:stuck_out_tongue_winking_eye:
Charlotte Soneson (09:37:26) (in thread): > Here it is:https://youtu.be/5KByRC6eqC8?t=1118 - Attachment (YouTube): useR! 2019 - Keynote : Joe Cheng
Kevin Rue-Albrecht (13:57:13): > Update on the T-shirts, I got: > 1. “I love the way you R” size M (~@Aaron Lun’s 1st choice) > 2. “I think therefore I R” size M (@Aaron Lun’s 2nd choice) > 3. “I love the way you R” size S (my 2nd choice) > So if that’s OK with you guys, I’ll send > - “I love …” to@Aaron Lun- “I think …” to@Federico Marini
Aaron Lun (13:57:44): > Was that my first?
Aaron Lun (13:57:50): > I thought 2 was my first.
Aaron Lun (13:57:55): > But whatevs.
Aaron Lun (13:58:14): > BECAUSE I DON’T LOVE ANYONE ELSE’s R.
Kevin Rue-Albrecht (13:58:23): > I’m not sure what wasactuallyyour first, but I’m reporting them as I recorded them onhttps://docs.google.com/document/d/1EtibBy_8kZSPH6CrycA7cEjQV6UEG0ecrJuWac31gXI/edit - File (Google Docs): iSEE-shiny-contest-prize-process
Kevin Rue-Albrecht (13:59:28): > Also, we’re still 1 t-shirt short (total 5 instead of 6), but they sent another copy of the Bookdown book.
Kevin Rue-Albrecht (13:59:40): > @Aaron LunI don’t remember you being a fan of paper books:stuck_out_tongue:
Aaron Lun (13:59:47): > I don’t recall that either.
Aaron Lun (13:59:56): > Mostly use them as laptop stands.
Kevin Rue-Albrecht (14:00:26): > well, this one is only useful to adjust your screen by ~0.5cm
Kevin Rue-Albrecht (14:01:11): > … that said you never know when you’ll need that little extra optimisation ^^
Kevin Rue-Albrecht (15:21:50) (in thread): > dammit she’s good
2019-07-12
Federico Marini (01:45:20): > guess we’ll buy the missing “shiny” tshirts:slightly_smiling_face:
Federico Marini (01:45:53): > stickers, any?
Federico Marini (01:46:09): > or do I have to plunder their desk for the giveaway?
Kevin Rue-Albrecht (03:15:43): > Eh… no extra sticker in that package. Enveloppe of 3 t shirts and 1 book “only”
Kevin Rue-Albrecht (03:16:23): > Curtis can correct me if i missed the secret compartment
Kevin Rue-Albrecht (03:17:08): > Btw. I just peaked at the complete slide deck for your talk@Federico Marini: did you get to the speech recognition?
Federico Marini (03:22:36): > Of course:smile:
Federico Marini (03:22:43): > you went live kev
Kevin Rue-Albrecht (04:06:20): > :joy::+1:
Kevin Rue-Albrecht (12:15:24) (in thread): > One sec before you merge that. I must have forgotten to push the tiny case where to coveriSEE()(literally no input). I’m sure I wrote that somewhere
Aaron Lun (12:15:48) (in thread): > FINISH HIM
Kevin Rue-Albrecht (12:18:20) (in thread): > Aargh.. I did push it.. onquickerboot
Aaron Lun (12:20:56) (in thread): > Just merge quickerboot as well, scRNAseq should build in the next run.
Kevin Rue-Albrecht (12:24:12) (in thread): > I’ve just relaunched Travis for that reason. The last PR build only failed because scRNAseq hadn’t propagated yet
Aaron Lun (12:24:33) (in thread): > Oh, I forgot about our builds.
Aaron Lun (12:24:55) (in thread): > Well, scRNAseq hasn’t propagated yet as ExperimentData packages don’t rebuild as frequenty.
Kevin Rue-Albrecht (12:25:19) (in thread): > Once it goes through I’ll merge
Kevin Rue-Albrecht (13:17:08) (in thread): > Thinking of quickerboot here’s another thought: I’ll probably throw in shortly another “mode” that launches the interface with 0 active panel (however many are available). I’ve got this feeling that for the purpose of EDA a good fraction of users would enjoy an app that launches much faster at the cost of adding panels, rather than waiting for 1 panel of each to load and then removing unused ones
Kevin Rue-Albrecht (13:18:13) (in thread): > Kind of iSEE_void(…) or iSEE_minimal(…)
Kevin Rue-Albrecht (13:24:58): > Ugh. > > * installing **source** package 'scRNAseq' ... > **** using staged installation > **** R > **** data > **** inst > **** byte-compile and prepare package for lazy loading > Error: package or namespace load failed for 'DelayedArray': > (converted from warning) multiple methods tables found for 'type' > Error: package 'DelayedArray' could not be loaded > Execution halted > ERROR: lazy loading failed for package 'scRNAseq' > * removing '/home/travis/R/Library/scRNAseq' > * restoring previous '/home/travis/R/Library/scRNAseq' > Error in i.p(...) : > (converted from warning) installation of package 'scRNAseq' had non-zero exit status > Calls: <Anonymous> ... with_rprofile_user -> with_envvar -> force -> force -> i.p > Execution halted > The command "Rscript -e 'devtools::install(dependencies=TRUE)'" failed and exited with 1 during . > > https://github.com/csoneson/iSEE/pull/301
Aaron Lun (13:25:38): > Looks like something upstream.
Aaron Lun (13:30:21): > because scRNAseq doesn’t evenuseDelayedArray.
Kevin Rue-Albrecht (13:46:05): > scRNAseq(Imports)SummarizedExperiment(Depends)DelayedArray
Aaron Lun (13:47:37): > Yes, that’s what I mean.
Aaron Lun (13:47:45): > someone else is hitting something upstream
2019-07-16
Aaron Lun (00:01:02): > build is okay for scRNAseq gogogogogo
Kevin Rue-Albrecht (04:02:03): > Thanks for the notice. I just tried to nudge Travis again, but scRNAseq 1.11.8 hasn’t propagated yet. Travis still installs 1.11.6 . Let’s give it a few more hours
Federico Marini (16:14:43): > Without sounding too melodramatic, one thing for you all@Charlotte Soneson@Kevin Rue-Albrechtand@Aaron Lun. > Thank you for making up the chance to work on this. I had one of the most amazing conference experiences at useR2019 where I could present iSEE’s latest content, in a pretty much compelling way which almost heads towards a science slam:slightly_smiling_face:
Federico Marini (16:15:13): > I could feel like walking a good amount of inches above ground after the whole exposure we got
Federico Marini (16:15:50): > That was the presentation, for the records:https://federicomarini.github.io/useR2019/
Charlotte Soneson (16:22:44): > That’s awesome:slightly_smiling_face:well done!:clap:
Federico Marini (16:23:32): > As I said: thank you all. Without your contribution I’d pretty much be doing something else, most likely duller
Aaron Lun (18:30:29): > Man, your presentations are way more polished than mine.
Aaron Lun (18:30:35): > I’m all like, “here it is. Use it or not.”
Aaron Lun (20:57:33): > 1.1.8 is out.
Aaron Lun (20:57:58): > Whoever merges, don’t forget to close Anna’s issue.
2019-07-17
Davide Risso (06:06:29): > Just randomly eavesdropping on this channel, but great presentation@Federico Marini! A gif mastery that I’ve only seen in Jeff Leek’s presentations!:wink:
Federico Marini (06:18:32): > Wow this is more than enough to have me blush:slightly_smiling_face:
Kevin Rue-Albrecht (06:24:05): > I’ve triggered again a Travis build this morning, it does fetch scRNAseq 1.1.8 > That said, the Travis build still fails to install the scRNAseq package due to an issue related toDelayedArray > > * installing **source** package ‘scRNAseq’ ... > **** using staged installation > **** R > **** data > **** inst > **** byte-compile and prepare package for lazy loading > Error: package or namespace load failed for ‘DelayedArray’: > (converted from warning) multiple methods tables found for ‘type’ > Error: package ‘DelayedArray’ could not be loaded > Execution halted > ERROR: lazy loading failed for package ‘scRNAseq’ > * removing ‘/home/travis/R/Library/scRNAseq’ > * restoring previous ‘/home/travis/R/Library/scRNAseq’ > Error in i.p(...) : > (converted from warning) installation of package ‘scRNAseq’ had non-zero exit status > Calls: <Anonymous> ... with_rprofile_user -> with_envvar -> force -> force -> i.p > Execution halted >
Kevin Rue-Albrecht (06:33:48): > What bugs me is that it seems to be a Travis-specific issue, because I can update my Bioc installation just fine with BiocManager.
Charlotte Soneson (06:46:01): > I had this same error for another package (on travis), but this morning it was magically resolved…
Charlotte Soneson (06:47:53): > Well, actually my error message was slightly different: > > type() only supports array-like objects and ordinary vectors. See ?type in the > DelayedArray package. >
Kevin Rue-Albrecht (06:58:58): > Perhaps it’s just about trying again a few kore turns until all the latest package versions re picked up. > Good to know it wasn’t just iSEE.
Kevin Rue-Albrecht (12:54:27): > Probably not someone you know@Charlotte Sonesonas they just sent me an email about iSEE, but looks like an ex neighbor or yourshttps://fgcz.ch/the-center/people/rehrauer.html
Charlotte Soneson (12:55:26): > Oh I certainly know Hubert:slightly_smiling_face:
Kevin Rue-Albrecht (12:56:45): > Another request for a generic iSEE app where users can load their own dataset
Kevin Rue-Albrecht (12:56:58): > > We have a shiny-server running and we do want that our users can upload their SingleCellExperiment object as an .rds file and visualize it. > > Did you ever consider such a use case?
Charlotte Soneson (12:57:06): > :smile:
Kevin Rue-Albrecht (12:57:22): > I love questions about (just) solved problems
Kevin Rue-Albrecht (12:58:15): > Btw I triggered another Travis build that has now passed the 6min mark where scRNAseq installation used to crash
Kevin Rue-Albrecht (12:58:24): > Hopefully I can merge to master tonight
Charlotte Soneson (12:58:39): > :+1:
Kevin Rue-Albrecht (15:00:30): > OMG I can’t believe it: Travis PASSES again!
Kevin Rue-Albrecht (15:01:32): > Now, that wasmaster, I’ll run the PR’s again so that it gets the green light for merging
Aaron Lun (15:43:37): > lol
Kevin Rue-Albrecht (16:35:14): > Ok.. So decision time: > Apparently only@Aaron Lun’s PRreboot(300) passes Travis:https://travis-ci.org/csoneson/iSEE/pull_requestsFor some reason,@Federico Marini’s PRquickerboot(301) since theReprocessedAllenDatacommit :https://github.com/csoneson/iSEE/pull/301
Kevin Rue-Albrecht (16:35:38): > Shall I merge PR300 while we work on 301? And if so, shall I bump up the version to make a new release for it?
Kevin Rue-Albrecht (16:41:14): > In any case, I think I will add a new commit anyway, to document > > options(shiny.maxRequestSize=50*1024^2) > which is a useful one that currently is only mentioned in PR300 (https://github.com/csoneson/iSEE/pull/300)
Kevin Rue-Albrecht (16:44:06): > Aargh, wait a second, in PR 300, there is still an item that Aaron left for me: > > @Kevin Rue-AlbrechtYour colormap check now occurs inside the server so it needs to raise a notification and quit gracefully rather than spewing errors out. > Haven’t looked at that yet. Though I’m not sure exactly whether@Aaron Lunhad a MWE to share perhaps?
Aaron Lun (16:54:21): > No, I don’t have an MWE, it’s just a prediction from static analysis.
Aaron Lun (16:54:47): > Basically you need to wrap the check inside atryCatchblock that raises a notification instead of actually erroring.
Aaron Lun (16:55:05): > To illustrate, see what happens when you upload an SE that is not compatible with the provided EMap.
Kevin Rue-Albrecht (16:59:13): > I see (oh oh). I’ve actually run into an example in the last few minutes: basically fixing up the unit tests onquickerboot, given that importing fewer assays creates this scenario of incompatible ECM, and it does throw an error. I haven’t tested out how it plays out in a live app, but I can imagine that it’s not pretty
Kevin Rue-Albrecht (17:59:17): > @Aaron LunI’m aware it was 18 days ago, but can you remember what “Bugfixes to observers” implied here? (https://github.com/csoneson/iSEE/pull/301/commits/b915ad98738b0b37397268bb917c6fb92edf656d) > I’m not sure if anything was broken already or you meant “updates following the file upload feature added”
Aaron Lun (18:01:36): > probably the latter
Kevin Rue-Albrecht (18:02:14): > Anyway,quickerbootPR (301) passes Travis now. It was only about unit tests needing an update because the test ECM was not compatible anymore withReprocessedAllenData(assays = "tophat_counts")
Kevin Rue-Albrecht (18:03:49): > One update worth mentioning, before we merge is the updatediSEEman page (https://github.com/csoneson/iSEE/blob/quickerboot/man/iSEE.Rd) > I’ve added a note about setting the max file upload size
Kevin Rue-Albrecht (18:05:17): > I gotta go sleep now though. No work on OSCA tonight. Had to choose:slightly_smiling_face:
2019-07-18
Federico Marini (05:21:41): > @Kevin Rue-Albrechtthank you for handling the email with Hubert
Federico Marini (05:22:01): > Probably we should ask him to post it anyway to the support bioc forum
Federico Marini (05:22:08): > guess he won’T be the last one asking
Federico Marini (05:22:17): > so we can just copypaste our reply
Kevin Rue-Albrecht (06:33:41): > Well, I’ve pointed him to the GH issue, which is the best we can do anyway, when people don’t find (or Google) the issue themselves
Kevin Rue-Albrecht (06:33:58): > Maybe we just need to make that issue more discoverable
Kevin Rue-Albrecht (06:36:00): > Or better, expand the documentation (e.g., vignettes)
Kevin Rue-Albrecht (06:36:45): > God I wish I had the time and freedom to follow up on all those ideas
Kevin Rue-Albrecht (13:22:48): > I’ll put the entireobservers.Runder “no coverage”, as it’s impossible to coverobserveEvent(...)blocks programmatically (that I’m aware of)
Aaron Lun (13:23:24): > yes.
Kevin Rue-Albrecht (13:25:57): > … plus it’ll bump up the unit test coverage back to an acceptable value:grimacing:
Kevin Rue-Albrecht (16:07:29): > Travis just completed PR 301. > Technically it’s ready for merge. > My question is whether we want to version bump as part of the same PR or if there is anything else that needs to be implemented before making a new release?
Aaron Lun (16:09:09): > Just bump it.
Kevin Rue-Albrecht (16:39:20): > https://github.com/csoneson/iSEE/issues/303
Aaron Lun (16:40:37): > @Federico Marinifill in kevin on what we discussed yesterday.
Federico Marini (16:41:12): > So that we are all on the same page: indeed some shortcuts for the modes could be a very nice addon
Federico Marini (16:41:33): > And since this release cycle is relatively tranquil (sorry to jinx it by saying it loud)…
Kevin Rue-Albrecht (16:44:21): > Is there a second half to your phrase? > > And since [….]
Federico Marini (16:48:46): > … it would just make sense to do that
Federico Marini (16:49:01): > yes there was. But I just stood up to catch me a panna cotta:smile:
Kevin Rue-Albrecht (16:50:23): > ah ah! got you there:stuck_out_tongue:
Aaron Lun (16:56:24): > o
Kevin Rue-Albrecht (17:46:36): > git push upstream master(-> 1.5.4):white_check_mark:
Aaron Lun (17:54:29): > :+1:
Kevin Rue-Albrecht (17:58:34): > https://github.com/csoneson/iSEE/commit/353de19b4ad9f4747f47b78ad1e3eb5c211ba9b7
Federico Marini (17:59:21): > me likes it
Aaron Lun (17:59:29): > No need toreturn, just finish with the iSEE statement.
Aaron Lun (17:59:58): > Suggest sticking to camelCase.
Aaron Lun (18:00:09): > Also have a cuter name. Need another iSEE pun.
Federico Marini (18:00:25): > don’t call me in
Federico Marini (18:00:36): > I just had a flurry of ideas for R-themed cocktails
Federico Marini (18:00:49): > that should be there for eRum2020
Federico Marini (18:01:17): > with my favorite ones being
Federico Marini (18:01:22): > tequila summarise
Federico Marini (18:01:27): > and long format ice tea
Aaron Lun (18:01:40): > just haveseeVoid.
Aaron Lun (18:01:44): > andseeGating.
Federico Marini (18:02:03): > if we ever use it for ChIP-seq…
Federico Marini (18:02:08): > seeSaw
Aaron Lun (18:03:38): > Hey, that sounds familiar.
Federico Marini (18:04:10): > Let’s have a glmlet to celebrate that
Kevin Rue-Albrecht (18:18:39): > > Also have a cuter name. Need another iSEE pun. > I meant this one as a reference toggplot2::theme_void
Aaron Lun (18:21:33): > yeah, I didn’t get that.
Kevin Rue-Albrecht (18:21:40): > iSEE:zap:QC:the_horns:
Kevin Rue-Albrecht (18:22:18): > That would be my mode for viewing the scater::calculateQCMetrics()
Kevin Rue-Albrecht (18:23:23): > In case this one is too subtle as well - File (JPEG): Image from iOS
2019-07-20
Kevin Rue-Albrecht (09:48:37): > Any preference betweenmodeVoid,modeEmpty,modeZero,modeNothingto rename the mode that starts with 0 panels?
2019-07-21
Charlotte Soneson (15:35:48): > To me, I thinkmodeEmptyis most intuitive
Kevin Rue-Albrecht (17:25:58): > Good point. They all sounded kind of the same to me, but this one does sound more a appropriate. Thanks!
2019-07-25
Aaron Lun (22:32:57): > @Federico MariniHow did you make those GIFs?
2019-07-26
Federico Marini (16:41:58): > which ones, in the presentation?
Federico Marini (16:42:03): > the ones stepping in or so?
Aaron Lun (16:42:23): > yeah
Federico Marini (16:42:30): > these are normal images, with a wisely designed css to animate them
Federico Marini (16:42:44): > I can invite you to the repo for the source of it
Federico Marini (16:43:40): > what basically does it is just to add the class attributes of the slide
Federico Marini (16:43:41): > sorry
Federico Marini (16:43:42): > no css
Federico Marini (16:43:43): > js
Federico Marini (16:43:49): > too much sun
Aaron Lun (16:46:20): > How did you take those images then? Video capture?
Federico Marini (16:46:42): > yep
Federico Marini (16:46:46): > ah
Federico Marini (16:46:47): > wait
Federico Marini (16:46:56): > not the “splashy” ones
Federico Marini (16:46:58): > yes
Federico Marini (16:47:01): > screencast
Federico Marini (16:47:10): > then to gif via a few steps
Federico Marini (16:47:19): > which I should have documented somewhere
Federico Marini (16:48:50): > say, like this;
Federico Marini (16:49:07): > > ffmpeg -i mov_toursteps.mov -f gif -r 25 - | gifsicle --delay=3 > gif_toursteps.gif >
Aaron Lun (17:42:58): > Is this on a mac? What software did you use for the screencast?
Federico Marini (17:50:14): > quicktime, plain and simple
Aaron Lun (17:50:24): > WTF is quicktime?
Federico Marini (17:52:49): > quicktime player
Federico Marini (17:52:50): > :smile:
Federico Marini (17:53:17): > that + new screen recording
Kevin Rue-Albrecht (18:39:56): > Does that thing still exist?:stuck_out_tongue: - File (PNG): image.png
2019-07-27
Aaron Lun (14:15:29): > man, I’m super worried about this guy.https://support.bioconductor.org/p/123265/#123275
Kevin Rue-Albrecht (14:45:47): > Ouch. Poor soul. Seems like he needs a bioinformatics 101 more than any advice on sparse matrix representations tbh
Federico Marini (15:27:01): > jeez aaron
Federico Marini (15:27:02): > https://support.bioconductor.org/p/123265/#123266
2019-07-28
Aaron Lun (13:56:15): > We should addmodeSingle, which just generates an app containing a single panel of the requested type.
Aaron Lun (13:56:43): > Possibly mimicking some ofscater’s functions, e.g.,modeColDatawith a similar set of direct arguments for the parameters.
Kevin Rue-Albrecht (14:06:23): > True. > That said, I can’t wait to havemodeSingleandmodeBatchelorin the same package:rolling_on_the_floor_laughing:
Federico Marini (15:33:34): > :dealwithit-parrot:
2019-08-02
Aaron Lun (14:19:09): > @Federico Marini@Kevin Rue-Albrecht@Charlotte SonesonWe really, really, really need that interactive chapter up and running for Rob’s book. Can someone give me a schedule? I already have after-dinner time full with the other chapters.
Kevin Rue-Albrecht (14:44:59): > I was hoping to look into it over the weekend. For prototyping example apps, I can use the free shinyapps account that I got from the contest, but there’s only another 6 months or so on it. After that I’ll pull the plug before they take from my credit card…
Aaron Lun (14:45:20): > Get@Federico Marini’s shiny server?
Kevin Rue-Albrecht (14:45:25): > Or that.
Aaron Lun (14:45:49): > Where is this guy anyway?
Kevin Rue-Albrecht (14:46:03): > Ah you don’t have Twitter:
Kevin Rue-Albrecht (14:46:32): > He was clearly enjoying a summer break:blush:
Kevin Rue-Albrecht (14:47:08): > Also, you kind of scared me about this restructure of the book, the simple single cell workflow and the package man pages. It sounded like a “no one touches anything” moment. I meant to ask
Aaron Lun (14:47:23): > Just do it, it’s fine.
Aaron Lun (14:47:27): > Text text text
Kevin Rue-Albrecht (14:47:31): > Yup
Kevin Rue-Albrecht (14:54:40): > I just haven’t been able to keep up the double workload between oxford and bioc lately. I still haven’t figured out a way/job opportunity to mix business and pleasure. If you have tips, I’ve got 13 months left with Steve.:innocent:main restriction is that milady (Anna) likes oxford.
Kevin Rue-Albrecht (14:56:11): > I believe Genentech takes remote consulting?:yum:
Kevin Rue-Albrecht (14:56:56): > Or i could have a word with Joe whenever that shiny winners Skype call finally happens:kissing:
Charlotte Soneson (15:42:26): > I should also be able to take a look at this tomorrow.@Kevin Rue-Albrecht- I wanted to ask one thing about the structure. I read through the current text and it kind of focuses on interactivity for sharing of information, rather than for analysis/exploration. I was wondering if we should start more generally - I just felt that it was a bit difficult to move on from the current start.
Federico Marini (16:55:09) (in thread): > Still lost in the hills of tuscany, busy with some dolce far niente:slightly_smiling_face:
Federico Marini (16:55:24) (in thread): > I’ll be soon back into civilization
Federico Marini (16:57:00) (in thread): > can be an option
Federico Marini (16:57:10) (in thread): > let me know what and how:slightly_smiling_face:
Aaron Lun (16:57:28) (in thread): > @Kevin Rue-Albrechtshould coordinate with you on that.
Aaron Lun (17:03:38): > @Kevin Rue-AlbrechtWork for industry and get your life back.
Federico Marini (17:07:43) (in thread): > I’ll be all ears - whatever medium fits best
Federico Marini (17:08:16): > Gotta vouch for him A-boy:slightly_smiling_face:
Kevin Rue-Albrecht (17:13:30): > Thanks. It’s more about knowing where to look, who to ask, and what to do to get there. It’s funny how easy it is to forget those things when every day is mostly worrying what story to squeeze out of data. I miss the good old software development and getting “in the zone”.
Kevin Rue-Albrecht (17:14:47) (in thread): > Absolutely. Feel free to edit, delete and move things around. This is mostly the result of one evening of brainstorming with@Rob Amezquita. I was brain frozen with a white page for weeks on how to get started, so at that point we just threw down some ideas to have some material to work with.
2019-08-03
Kevin Rue-Albrecht (06:24:55) (in thread): > The structure is clearly a weak point. Most likely because I started writing with only the first section “Motivation” and then drifted off > Making a list of section headers to guide the flow would definitely be a big help and even allow collaborative writing of various section in parallel.
Kevin Rue-Albrecht (06:56:07) (in thread): > Also, the main title “Sharing and Interactive Interfaces” was there before I wrote anything. That’s also why I somewhat restricted myself to that aspect. > > Funnily enough, that makes me realize that flipping the order to “Interactive visualization and sharing” would encourage me to more naturally write about visualizing for analysis/exploration first, and then opening on how to share interactive apps with others. > Thoughts?
Aaron Lun (13:53:02): > Funnily enough, I anticipate that we will have an opening soon.
Aaron Lun (20:41:08): > hello?
2019-08-04
Kevin Rue-Albrecht (04:59:42): > Yup?
Kevin Rue-Albrecht (05:02:04): > If you’re worrying about osca, rather than pushing/pulling, Charlotte and I have revived the google docs for drafting a series of sections herehttps://docs.google.com/document/d/15j-8ODkH7h1MbugDXyaVQrnoGXNY3DeaGmUkALsOM1c/edit# - File (Google Docs): OSCABase - Interactive Interfaces and Sharing
Kevin Rue-Albrecht (05:07:38): > Ah, and by the way: I finally just deployed this on myShinyapps.ioaccount:https://kevinrue.shinyapps.io/iSEE/I have enabled uploads up to 1GB (options(shiny.maxRequestSize=1024^3)), but I may still need to fine tune the server settings from theshinyapps.iodashboard. I always found the documentation of that one a bit lacking and confusing, so it may take me a few trials and errors
Aaron Lun (14:09:34): > sweet
Kevin Rue-Albrecht (14:38:20): > Let me know if and how you break the app. I suspect it won’t handle excessively large objects, or HDF5-backed datasets (?)
Kevin Rue-Albrecht (17:51:35): > Getting there. Slowly, but rather surely, thanks to Charlotte’s help. I gotta go sleep now, but I just laid down a decent draft for a section about an example app for QC.
Kevin Rue-Albrecht (17:52:44): > I think she added the section title “Annotation and identification of cell populations”, not sure what exactly would go there. Discussion for tomorrow, I guess
Kevin Rue-Albrecht (17:53:14): > Anyway, hopefully we’ll push something over the week, more likely next weekend
2019-08-05
Charlotte Soneson (08:03:02): > (Somewhat) unrelated - I’m presenting a poster on iSEE in our annual meeting in September. Draft below, comments welcome. - File (PDF): Soneson_annual_meeting_2019.pdf
Kevin Rue-Albrecht (08:03:49): > :party_parrot::star-struck:
Kevin Rue-Albrecht (09:03:34): > Suggestionfor the third (last) green paragraph: > > Any class extending theSummarizedExperimentclass can be used. For instance, theSingleCellExperimentclass implements the additional slotreducedDimfor storing a set of reduced dimension representations (e.g., PCA, tSNE, UMAP). The presence of this slot enables the Reduced dimension plot panel type.
Kevin Rue-Albrecht (09:07:52): > Challenge: minimize lines crossing, avoid if possible at all.Suggestion: moving the following > > Colour the points by any provided annotation, or by observed values for features or samples (e.g., gene expression levels) > above the following > > Select one or more sets of points in one panel, and transfer this selection to > one or more other panels > would result in a single line crossing instead of two > (Also, even historically in terms of implementation, it’s easier to talk about coloring points in a single panel before talking about transferring selections between two panels.)
Kevin Rue-Albrecht (09:11:25): > Current: > > Eight panel types are provided > Suggestion: > > Eight built-in panel types are provided
Kevin Rue-Albrecht (09:11:51): > Suggestion: > End each text snippet with a full stop. At the moment, some do, some don’t.
Kevin Rue-Albrecht (09:16:46): > Current: > > Access the quick start guide, or provide a customised step-by-step guide through the displayed panels > Suggestion: > > Launch a guided tour: either the default quick start introduction to the interface, or a bespoke step-by-step guided tour adapted to any data set. > (something like that)
Kevin Rue-Albrecht (09:17:36): > Current: > > subsampling points for faster plot rendering > Suggestion: > > subsampling points for faster plot rendering (use the “Visual parameters” box)
Kevin Rue-Albrecht (09:18:43): > THE END. (I’ve gone through the poster)
Charlotte Soneson (09:21:52): > Thank you!:star-struck:all fixed
Kevin Rue-Albrecht (09:25:20): > Really good looking poster!:+1:Sorry not sorry@Aaron Lun:stuck_out_tongue:Which makes me wonder if he saved the components (or it is slots?) of his poster:sweat_smile:
Charlotte Soneson (09:44:55): > What do you do with old posters anyway? Most of mine are in a corner in our apartment…
Aaron Lun (11:49:04): > Oh, mine’s hung up in CRUK.
Aaron Lun (11:50:11): > I don’t know if I’m meant to put my CRUK address or my Genentech address on this thing. I would say go with the CRUK one, because (i) it’s how we published it and (ii) avoids getting the company lawyers involved for IP issues.
Charlotte Soneson (11:51:14): > Ok, will do
Aaron Lun (12:14:05): > Basically, if I could possibly be interpreted as a representative of Genentech for some event, I have to get authorization.
Aaron Lun (12:14:13): > Sounds crappy but they pay me superbucks so whatever.
Charlotte Soneson (12:16:34): > Yeah, I also somehow noticed the difference when I moved from the university to a pharma-associated research institute.
Kevin Rue-Albrecht (16:53:06): > FYI, trying to set up an example for the “Annotation and identification of cell populations” section, I’ve run into the following issue for gaz’illion-th time:https://github.com/rstudio/shinydashboard/issues/328I’m not staying up much longer so feel free to subscribe to the issue and keep the conversation going if they need more feedback. It’s a pretty common issue for us, given the number of columns that we typically face incolData/rowData. > > Minimal reprex included, so no excuse for not checking it out:wink:
2019-08-08
Federico Marini (05:25:50): - File (PNG): image.png
Federico Marini (05:26:08): > I could not see the tweet as it is a private account, but now, “AH AH, iSEE”
2019-08-09
Kevin Rue-Albrecht (07:54:42): > @Federico MariniDid you come across this candidate meme for iSEE presentations?https://pbs.twimg.com/media/D8N3qCUXYAAD7PU?format=jpg&name=4096x4096 - File (PNG): image.png
Federico Marini (08:12:13): > ehehehe
Federico Marini (08:12:22): > I’ll definitely put it in the science slam
2019-08-10
Aaron Lun (00:05:41): > It’s quiet around here.
Aaron Lun (00:05:43): > Too quiet.
Federico Marini (07:30:51): > Soon I’ll start throwing in some more memes
Federico Marini (07:30:53): > promised
Kevin Rue-Albrecht (09:04:56): > I know… I miss the days of careless peak iSEE development. It’s frustrating because there is still so much that could be done.
Kevin Rue-Albrecht (09:06:23): > Unfortunately, my hands are basically tied these days. Can’t save any time to focus on anything. I haven’t done any scRNA-seq in such a long time that OSCA almost feels like a swansong… Speaking of which the plan is to copy and push the Google Docs of the interactive chapter toOSCABasethis afternoon
Kevin Rue-Albrecht (11:13:54): > This paper writing business is probably not for me in the end. I like software development too much:male-technologist:
Kevin Rue-Albrecht (11:16:14): > God.. and it was really time to copy that Google Docs to OSCABase.. GDoc served its purpose, but what a pain to move things over. Mostly for the cross references that we couldn’t really type in there. But also because I’m doing the final edits at the same time I guess
Aaron Lun (14:01:57): > Industry! Industry! Industry!
Kevin Rue-Albrecht (14:03:57): > Hell yeah. I just need to figure out how that works / the ropes and learn to read a job description properly this time
Kevin Rue-Albrecht (14:04:57): > Nice bio in the OSCA contributors section btw@Aaron Lun
Aaron Lun (14:06:33): > I wondered who would read it.
Kevin Rue-Albrecht (14:56:48): > Yeah.. I’m not your usual reader.. guess who PR’ed a typo in the introduction of the bookdown book:stuck_out_tongue:https://github.com/rstudio/bookdown/pull/740
Aaron Lun (21:20:55): > Just addedlfcthreshold for wilcox tests
Aaron Lun (21:20:57): > Oh jeez.
Aaron Lun (21:20:59): > So pain
2019-08-11
Kevin Rue-Albrecht (13:06:24): > I updated the PR following Fede’s review. I haven’t applied all suggestions: my replies on the ones I didn’t do are in the GDoc
2019-08-14
Federico Marini (09:23:30): > iSEErs and not only:https://mastering-shiny.org - Attachment (mastering-shiny.org): Mastering Shiny > The official guide to the Shiny web application framework for R.
2019-08-15
Stephanie Hicks (15:29:35): > hi#iseeteam — I’m looking to see if we could set up a time to talk about how I can help support you and become more involved with the team. As Bioc is now funded by CZI for three years and I am committed to the Interactive and Scalable Data Visualization portion, I definitely want to see if we could determine some goals / products of things that you would like to see created.
Aaron Lun (15:31:28): > Yes, I have a feature wishlist.
Stephanie Hicks (15:31:31): > I can hire someone (e.g. a postdoctoral fellow) who would be able to support this effort too, but I want identify what would be the goals first
Stephanie Hicks (15:31:44): > that will help me figure out the right background / training for the person to hire
Stephanie Hicks (15:40:02): > in case it’s useful, this is what we wrote in the grant, but my sense is that as long as it is working towards the goal of interactive and scalable data viz in R/Bioc, it would be fine: > > Interactive and scalable data visualization is crucial for understanding HCA data across samples (e.g., experimental replicates, patients, cells) and features (e.g., genes, transcripts). We will implement interactive graphics for millions of cells, specifically heatmaps and dendrograms, using the SummarizedExperiment Explorer (iSEE) package. iSEE is built as a Shiny application. iSEE allows examination of gene and cell specific meta data and analysis results, dynamic linking between plots, and code tracking for reproducibility. We will (a) use standard Bioconductor infrastructure for scalable visualization of large-scale and remote data. We will (b) develop data subsampling strategies for efficiently plotting large datasets. We look forward to (c) hosting iSEE on the DCP.
Charlotte Soneson (15:40:29): > That sounds great. I’d be available for a call Monday to Wednesday next week, then I’m on holidays for the rest of August, then I’m available again from September.
Stephanie Hicks (15:43:51): > Considering everyone’s time zones, I’m going to guess lunch my time is easiest. Is Monday 6pm GMT+2 / 12pm EST / 9am PST reasonable?
Aaron Lun (15:45:33): > Sounds fine to me.
Charlotte Soneson (15:45:38): > Me too
Kevin Rue-Albrecht (16:25:53): > Is that 4 or 5pm UK time? I thought UK was GMT, but somehow Google tells me GMT+2 is only 1h ahead of UK. Perhaps daylight saving time is offsetting things
Kevin Rue-Albrecht (16:28:55): > double checking the time looks like 5pm form me, which is perfect on Monday
Stephanie Hicks (16:31:57): > :thumbsup:
Kevin Rue-Albrecht (16:41:44): > Do you already have a timeline for you to (find and) hire someone? I remember reading that the award earliest start date was December this year, correct?
Stephanie Hicks (17:05:26): > Turns out the award started July 1. I just found that out today…:face_palm_star_trek:
Kevin Rue-Albrecht (17:07:19): > oh ok:rolling_on_the_floor_laughing: > > to talk about how I can help support you > Not that I’m available for hire but I’d love to find a way to keep contributing and that would definitely require me finding support one way or another. I don’t see much Bioconductor momentum in Oxford, and it is explicitly not a priority in the group where I am hired until September next year, which isn’t helping. Though I keep wondering whether that could actually be an opportunity to create my niche in that space (i.e. “the Oxford Bioconductor branch”). Happy to discuss ideas on Monday though. > More importantly, curious to hear Aaron’s ideas for future developments. AFA I’m concerned I mostly left some ideas hanging in GitHub branches. I’ll try and collect my ideas for Monday.
Kevin Rue-Albrecht (17:15:13): > On a separate note, I didn’t want to hijack the developer forum today for iSEE talk,@Aaron Lun, but is there any foreseeable impact ofaltExpon iSEE?
Aaron Lun (17:16:51): > Possibly. Depends on how much effort one expects to the user to put in.
Aaron Lun (17:17:47): > Probably easiest to write a function that “flattens” all aspects of the SCE. We already do that at the start, we should just formalize it into a proper function.
Aaron Lun (17:18:38): > And there are more sophisticated possibilities that leverage delayed evaluation.
Kevin Rue-Albrecht (17:20:29): > I wonder howggbiohandles stuff. It must be flattening stuff at some point to work in the ggplot framework
Kevin Rue-Albrecht (17:26:19): > FYI, to throw in ideas for Monday:https://docs.google.com/document/d/1qClgSS4pT3cCnS5B-EgBlcXX9WbEjti7n2ukdeb80yE/edit?usp=sharing
Federico Marini (17:26:31) (in thread): > I’m in as well
Federico Marini (17:27:25): > Cannot guarantee the environment around me is scream-free if I am home already:slightly_smiling_face:
Kevin Rue-Albrecht (17:29:01): > As long as you’re here Fede, would you like to merge your branchruntravisrun? Doesn’t hurt and would get rid of an otherwise superfluous branch
Federico Marini (17:30:06): > I’m literally falling asleep so I’d leave critical ops for tomorrow
Federico Marini (17:30:17): > but yes, I’ve basically nothing against it
Kevin Rue-Albrecht (17:31:53): > Cool, thanks. I don’t see anything critical in it, and it doesn’t affect bioc-build, only our Travis CI setup. But yeah, after 8 month pending, tomorrow’s still good:stuck_out_tongue:
Kevin Rue-Albrecht (17:32:52): > neater_readmedoesn’t hurt either , while you’ll be at it
Kevin Rue-Albrecht (18:15:16): > Going to bed. I started throwing down some discussion points in the Google Docs above
Stephanie Hicks (18:15:26): > @Kevin Rue-Albrechtcould you give me access to the google doc?
Kevin Rue-Albrecht (18:15:29): > can anyone confirm that you have editing power?
Kevin Rue-Albrecht (18:15:33): > ah, thanks ^^
Kevin Rue-Albrecht (18:16:32): > it says now “Anyone who has the link can edit”
Kevin Rue-Albrecht (18:18:36): > Everyone feels free to edit/add/remove anything. No need for “Suggesting mode”, except if you really have a doubt about something. Otherwise knock yourselves out
Stephanie Hicks (18:18:40): > Thanks I can see it now
Kevin Rue-Albrecht (18:21:16): > PS: I’m genuinely curious to see an iSEE instance dealing with millions of cells. That’s probably a good challenge to force us to optimize a bunch of functions and usage of out of memory representations, I imagine.
Stephanie Hicks (19:12:51): > I’ve learned quite a bit about on disk data representations (between mbkmeans and working with WGBS dna methylation data). So happy to help contribute my knowledge there
2019-08-16
Charlotte Soneson (11:48:49): > iSEEfails on the devel build machines (since some time); from what I can see the error occurs here: > > > sceX > class: SingleCellExperiment > dim: 0 0 > metadata(3): SuppInfo which_qc log.exprs.offset > assays(0): > rownames: NULL > rowData names(0): > colnames: NULL > colData names(0): > reducedDimNames(0): > spikeNames(1): random > altExpNames(1): ERCC > Warning message: > 'type=' is deprecated. > > reducedDim(sceX) > Error: subscript contains out-of-bounds indices > In addition: Warning message: > 'type=' is deprecated. > > Also lots of warnings abouttype=being deprecated.
Aaron Lun (11:49:35): > That second thing shouldn’t be happening.
Aaron Lun (11:49:42): > The first thing also shouldn’t be happening either.
Aaron Lun (11:50:17): > These are reminiscent of SCE bugs from the refactor, but I thought I fixed them all.
Charlotte Soneson (11:50:51): > The above is from my testing locally - build system says only > > * checking tests ... > Running 'testthat.R' > ERROR > Running the tests in 'tests/testthat.R' failed. > Last 13 lines of output: > Error: testthat unit tests failed > In addition: Warning messages: > 1: In .local(object, ...) : using library sizes as size factors > 2: 'centreSizeFactors' is deprecated. > See help("Deprecated") > 3: 'isSpike<-' is deprecated. > Use 'isSpike<-' instead. > See help("Deprecated") > 4: 'spikeNames' is deprecated. > See help("Deprecated") > 5: 'isSpike' is deprecated. > See help("Deprecated") > 6: 'type=' is deprecated. > 7: 'type=' is deprecated. > Execution halted >
Kevin Rue-Albrecht (11:53:30): > I thought that was just a transient bug while some upstream package updates were propagating. From what Aaron says, it sounds like it. That said, let me know if there’s a chance it’s on our side.
Aaron Lun (11:53:58): > well, it wouldn’t be if charlotte is biovcalid
Charlotte Soneson (11:54:03): > Yeah - I just got an email that it had been failing since at least July 31
Charlotte Soneson (11:55:28): > Should be ok > > > BiocManager::version() > [1] '3.10' > > BiocManager::valid(lib = .libPaths()[1]) > [1] TRUE >
Kevin Rue-Albrecht (14:38:26): > I’ve updated to BiocValid = TRUE, and I can confirm that I run into: > warning when setting up the SCE > > > source("tests/testthat/setup_sce.R") > Loading required package: scRNAseq > Loading required package: scater > Loading required package: ggplot2 > snapshotDate(): 2019-08-13 > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > downloading 0 resources > loading from cache > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > downloading 0 resources > loading from cache > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > downloading 0 resources > loading from cache > Warning messages: > 1: In .local(object, ...) : using library sizes as size factors > 2: 'centreSizeFactors' is deprecated. > See help("Deprecated") > 3: 'isSpike<-' is deprecated. > Use 'isSpike<-' instead. > See help("Deprecated") > 4: 'spikeNames' is deprecated. > See help("Deprecated") > 5: 'isSpike' is deprecated. > See help("Deprecated") > 6: 'type=' is deprecated. > 7: 'type=' is deprecated. > > and error as reported by the test > > Error: Test failed: 'memory setup works correctly' > * subscript contains out-of-bounds indices > 1: iSEE:::.setup_memory(se = sceX, redDimArgs = NULL, colDataArgs = NULL, featAssayArgs = NULL, rowStatArgs = NULL, > rowDataArgs = NULL, sampAssayArgs = NULL, colStatArgs = NULL, customDataArgs = NULL, customStatArgs = NULL, > heatMapArgs = NULL, redDimMax = 5, colDataMax = 3, featAssayMax = 1, sampAssayMax = 1, rowStatMax = 2, > rowDataMax = 3, colStatMax = 2, customDataMax = 2, customStatMax = 3, heatMapMax = 2) at :144 > 2: DEFFUN(se, cur_max) at /Users/kevin/git/iSEE/R/iSEE-extras.R:253 > 3: ncol(reducedDim(se)) at /Users/kevin/git/iSEE/R/defaults.R:303 > 4: reducedDim(se) > 5: reducedDim(se) > 6: .local(x, ...) > 7: internals[, type] > 8: internals[, type] > 9: extractCOLS(x, j) > 10: extractCOLS(x, j) > 11: normalizeSingleBracketSubscript(i, xstub) > 12: NSBS(i, x, exact = exact, strict.upper.bound = !allow.append, allow.NAs = allow.NAs) > 13: NSBS(i, x, exact = exact, strict.upper.bound = ! >
Aaron Lun (14:38:57): > shit, that shouldn’t happen.
Aaron Lun (14:39:25): > and thishasa reducedDim entry, right?
Kevin Rue-Albrecht (14:39:48): > Yes > > > sce > class: SingleCellExperiment > dim: 20816 379 > metadata(3): SuppInfo which_qc log.exprs.offset > assays(3): tophat_counts counts logcounts > rownames(20816): 0610007P14Rik 0610009B22Rik ... Zzef1 Zzz3 > rowData names(3): num_cells mean_count letters > colnames(379): SRR2140028 SRR2140022 ... SRR2139341 SRR2139336 > colData names(23): NREADS NALIGNED ... passes_qc_checks_s nested > reducedDimNames(2): PCA TSNE > spikeNames(1): random > altExpNames(1): ERCC > Warning message: > 'type=' is deprecated. >
Kevin Rue-Albrecht (14:40:47): > > SingleCellExperiment kevin$ grep deprecated R/* > R/SCE-sizeFactors.R: .Deprecated(msg="'type=' is deprecated.") > R/SCE-sizeFactors.R: .Deprecated(msg="'type=' is deprecated.") >
Kevin Rue-Albrecht (14:41:19): > (I’ve got a clone of the SCE package on my machine)
Aaron Lun (14:42:10): > Dammit, I’ll look at it later.
Kevin Rue-Albrecht (14:55:33): > hang on, I think I’m onto something. In oursetup_sce.Rwe have > > sizeFactors(sce, "random") <- runif(ncol(sce)) > > while I just figured out that > > > sizeFactorNames(sce) > [1] "random" > Warning message: > 'sizeFactorNames' is deprecated. > See help("Deprecated") > > I’ll update myself on the recent changes in SCE that I’ve missed. Apparently size factors can’t be named any more. The price of living on devel:slightly_smiling_face:
Kevin Rue-Albrecht (15:13:24): > no one touches anything (in iSEE), I’ve started fixing a few things, and I’m getting close to the source of the issue for the unit test
Aaron Lun (16:00:45): > That’s deprecated, but it shouldn’t have crashed. Can you see why it crashes? Seems like a crack in SCE.
Kevin Rue-Albrecht (17:39:17): > Isn’t this supposed to returnNULLrather than crashing? > > > reducedDim(SingleCellExperiment()) > Error: subscript contains out-of-bounds indices >
Aaron Lun (17:45:30): > yes it should. Throw that as an issue and will deal with it tonight.
Kevin Rue-Albrecht (17:46:38): > I was just looking at it between Slack, dinner and movie. Trying to sneak my name into SCE package:stuck_out_tongue:
Kevin Rue-Albrecht (17:47:05): > doesn’t sound like a tough one
Aaron Lun (17:47:05): > Oh, if you want that, I have some hardcore work for you.
Kevin Rue-Albrecht (17:47:40): > been screwed enough by the lincRNAs, time to screw them back and blow off steam in some useful code
Aaron Lun (17:47:57): > Good, you can fill in the QC plots on the book.
Aaron Lun (17:50:01): > Just go through the chapters I marked as done, and clean them up, please. And fix the basic analysis chapter, you know how I like to write.
Aaron Lun (17:50:28): > Don’t worry about the SCE thing, the code is pretty FUBAR so I’ll take care of that.
Tim Triche (18:24:01): > @Tim Triche has joined the channel
Kevin Rue-Albrecht (18:25:17): > @Aaron Lunso… all theisSpikecalls should just disappear now from the iSEE internals? how do we handlealtExps?
Aaron Lun (18:25:51): > Yes, isSpike should disappear, but I don’t think anyone wasreallyusing them anyway.
Aaron Lun (18:26:43): > altExps are tricky. They are SEs in their own right, and one could conceivably make another app with altExps.
Aaron Lun (18:26:58): > I suggest ignoring them for the time being, the app isn’t acrchitucterd to deal with them.
Kevin Rue-Albrecht (18:27:11): > > I suggest ignoring them for the time being. > good, i was about to type that
Kevin Rue-Albrecht (19:13:15): > OK.. so here is the status of the latest push > locally > -covr::package_coverage()passes 100% > - devtools::check() fails for some really unclear reasonFatal error: length > 1 in coercion to logicalwhich doesn’t point to any line file/line in the package. Possibly an RStudio issue. I haven’t tried R CMD check. I’m leaving that up to Travis
Kevin Rue-Albrecht (19:14:27): > and Travis is taking a lot of time to update the package cache because of the extended period of time without a successful build.
Kevin Rue-Albrecht (19:16:45): > Though I’m stupid: I can’t expect Travis to pass until my PR is merged toSCEand the bumped version propagates to devel:face_palm_star_trek:
Kevin Rue-Albrecht (19:27:05): > Yup, it was an RStudio thing.R CMD checkpasses with my local copy of theSCEPR > > Status: 1 WARNING, 2 NOTEs > * checking examples ... WARNING > Found the following significant warnings: > > Warning: 'centreSizeFactors' is deprecated. > Warning: 'centreSizeFactors' is deprecated. > Warning: 'centreSizeFactors' is deprecated. > > * checking installed package size ... NOTE > installed size is 24.8Mb > sub-directories of 1Mb or more: > R 2.3Mb > doc 21.8Mb > > * checking dependencies in R code ... NOTE > Unexported object imported by a ':::' call: 'S4Vectors:::selectSome' > See the note in ?`:::` about the use of this operator. >
Kevin Rue-Albrecht (19:27:29): > G’night
Aaron Lun (19:27:49): > oyasumi
Aaron Lun (19:28:04): > whoops, this isn’t my anime chat.
2019-08-17
Kevin Rue-Albrecht (07:35:15): > Merged and pushed upstream. Seehttps://github.com/csoneson/iSEE/pull/309to check for any issue. Looked OK to me (when I ran an app interactively).
Kevin Rue-Albrecht (07:35:58): > That should get us back on track on both Travis and bioc-devel
Kevin Rue-Albrecht (08:27:44): > Uh. Funny summertime: July 2019 was our highest monthly download rate so far
Aaron Lun (13:23:33): > Hold on. I don’t understand why the protection in 309 insideredDimDefaults()is necessary. If there are no reduced dimensions, theredDimArgsshould just get subsetted out to zero. Where is this getting used?
Kevin Rue-Albrecht (13:26:25): > Let me get you the line of code, but yes: all default functions get called even if the DataFrame is 0-row
Aaron Lun (13:27:09): > Well that’s fine, but why are the values being used if the DF is zero row?
Kevin Rue-Albrecht (13:27:10): > https://github.com/csoneson/iSEE/blob/master/R/iSEE-extras.R#L239
Kevin Rue-Albrecht (13:27:17): > DEFFUN
Aaron Lun (13:27:35): > I know that it’s called, but they shouldn’t have been used.
Aaron Lun (13:27:50): > That’s whatfeasibilitywas to defend against.
Kevin Rue-Albrecht (13:29:35): > I know what you mean by feasibility, but I don’t know what you mean by “getting used”. Or actually, we still create all the DataFrames of arguments, even for panels that will never show up in the app instance, so perhaps yes, I see what you mean there, “if max == 0 then don’t create the dataframe:
Kevin Rue-Albrecht (13:30:05): > go have a look at the SCE PR if you want. I can have a look at this now
Aaron Lun (13:31:07): > No, it’s fine to create the DF. But if it’s not feasible,numbershould have been set to zero, so the values inside the DF shouldn’t matter.
Kevin Rue-Albrecht (13:34:21): > They don’t matter indeed, but the default value of the second RD axis was still computed asmin(2L, ncol(reducedDim(se, type = 1L))), which causes an error whenreducedDimcrashes in the absence of RD
Kevin Rue-Albrecht (13:34:46): > It didn’t crash whenNULLwas returned in the absence of RD
Aaron Lun (13:35:47): > Right, so just add the protection for that single field.
Kevin Rue-Albrecht (13:36:38): > Exactly. I just thought that - as none of the fields matter - I’d put them all under the same protection. Perhaps that’s confusing for nothing
Kevin Rue-Albrecht (13:37:54): > In fact, I don’t need my new protection, I could justif waszero ...
Aaron Lun (13:38:38): > Well, it would still need to put something for that field.
Aaron Lun (13:39:00): > Hm. This should really be something that’s caught by validity checks later.
Kevin Rue-Albrecht (13:39:02): > of course, nevermind, you’ll see what I mean
Aaron Lun (13:43:06): > I think a sensible policy would be to never attempt to touchseifwaszero=TRUE, because if you’re returning an empty DF, why would you ever need to look at any values inse? This would need to apply to all default argument functions.
Kevin Rue-Albrecht (13:43:47): > yup
Aaron Lun (15:21:17): > Why did we call itdev_discreteand notdef_discrete? Did someone just press the wrong key? I see that f is right next to v on the keyboard.
Aaron Lun (15:26:02): > Looking at my power bill. I realized my energy-saving practices would save me all of…. $2 a month.
Aaron Lun (15:26:16): > It is literally not worth my time to go and switch off the lights.
Aaron Lun (15:26:37): > Anyway, time to go and buy a new chair. Back in a few hours.
Kevin Rue-Albrecht (16:25:23): > > Why did we call itdev_discreteand notdef_discrete? Did someone just press the wrong key? I see that f is right next to v on the keyboard. > I think so, I remember correcting something like that in a place, but obviously not everywhere, if there are some left
Aaron Lun (18:04:30): > oh yeah, this chair is everything i hoped for
Aaron Lun (18:04:35): > it’s got a mesh back
Aaron Lun (19:08:33): > Oh geez, you can really feel the quality of this chair.
Aaron Lun (19:08:40): > It’s 150% more expensive than my previous one.
2019-08-18
Kevin Rue-Albrecht (05:14:53): > :smile:
Kevin Rue-Albrecht (11:03:53): > I just pushed 1.5.7 to GitHub. I tested out in a bunch of cases interactively, and it all seems to behave
Kevin Rue-Albrecht (13:02:12): > AFA I’m concerned, the PR is ready to merge. I’ve just gone through the diff again, and I’m fine with it
Aaron Lun (13:05:26): > Looks good. I would push the NULL check aboveset_default_assay(), as this just returns a single argument so there’s no need for multiple checks (i.e., one check to pass it a NULL, another check inside it to behave sensibly with NULL) - you might as well have a single check to replace the point where you pass it a NULL, like you do with the other things likecovariates.
Aaron Lun (13:05:46): > Also, document why se is allowed to be NULL in the internal docs for the add_general_parameters_*.
Kevin Rue-Albrecht (13:08:54): > Gotcha. My nose so close to the code I didn’t see the obvious there. > I documented theNULLin one of those functions. Let me check again
Kevin Rue-Albrecht (13:15:13): > There we go, it’s documented in.add_general_parameters, which shares its man page withadd_general_parameters_for_column_plotsandadd_general_parameters_for_row_plots
Aaron Lun (13:15:38): > aw geez taxes
Aaron Lun (13:15:41): > Aw man
Kevin Rue-Albrecht (13:16:31): > you don’t have shiny app for that?
Kevin Rue-Albrecht (13:16:55): > How’s that for doc? > > The default argumentse=NULLis intended to populate the DataFrameincomingwith dummy values of the appropriate type for each column. This avoids the need to query theseobject for information that will ultimately not be used.
Aaron Lun (13:19:43): > We should also put a comment somewhere thatnumber=0guarantees thatseis not touched. Possibly mention that at the start of every function, below the “Transient 1-row DF,” comment.
Kevin Rue-Albrecht (13:20:18): > Good point
Kevin Rue-Albrecht (13:22:17): > I’d argue that it’s a good info to put in the internal doc page. You sound like I should mention italsoat that point in the code? what’s the position on DRY?
Kevin Rue-Albrecht (13:23:13): > Though those are public facing functions, perhaps we don’t want to confuse users [with info on the doc page]?
Aaron Lun (13:23:20): > Huh?
Aaron Lun (13:23:30): > Just put a comment below the traisnet DF comment.
Kevin Rue-Albrecht (13:23:35): > :+1:
Aaron Lun (13:23:46): > Otherwise we’ll forget why dwe did it.
Kevin Rue-Albrecht (13:24:43): > Definitely. I was just wondering for a second whether thisnumber=0is something that should be mentioned on the man page as well. But I don’t see a reason why users would call those functions withnumber=0
Kevin Rue-Albrecht (13:25:13): > It’s just something that we need to remember for internal development, rather.
Aaron Lun (13:26:16): > Ys.
Kevin Rue-Albrecht (13:28:22): > > redDimPlotDefaults <- function(se, number) { > # Ensure that we define all the fields with the right types, using a transient 1-row DF > # number=0 guarantees that se is not touched to define dummy values of the right type > waszero <- number == 0 > ... >
Aaron Lun (13:28:41): > Yep.
Kevin Rue-Albrecht (14:22:11): > Alright, it’s all passed Travis. Time for merge and push upstream I believe. Better [know if we] crash bioc-build earlier than later
2019-08-19
Kevin Rue-Albrecht (04:05:14): > @Federico Marinidid you spot this ?https://www.biorxiv.org/content/biorxiv/early/2019/06/10/618439.full.pdf
Kevin Rue-Albrecht (04:05:40): > @Charlotte Sonesonmight be curious too
Federico Marini (04:31:19): > I did - has a slightly different deployment model
Federico Marini (04:31:34): > yet it is nice to see alternatives to the YAShinyApp model
Kevin Rue-Albrecht (04:32:11): > speaking of YAShinyApp, did you see~Hadley’s tweeter announcement yet?~the news about Hadley?
Kevin Rue-Albrecht (04:33:04): > https://qz.com/1661487/hadley-wickham-on-the-future-of-r-python-and-the-tidyverse/ - Attachment (Quartz): What’s next for the popular programming language R? > Star programmer Hadley Wickham hopes R will become more diverse and play better with other languages.
Kevin Rue-Albrecht (04:34:52): > Well, you’re the one who starred “https://github.com/hadley/mastering-shiny” recently:wink:
Federico Marini (04:40:42): > Looks good on many sides
Kevin Rue-Albrecht (04:41:26): > also, i laugh a bit when people star and/or forkhttps://github.com/kevinrue/isee-shiny-contest… there’s almost nothing useful in there aside from links to the package itself ^^
Federico Marini (04:42:22): > We should open up a contest for the 100th star for iSEE
Federico Marini (04:42:38): > this would force me towards making at least the metallic-style iSEE stickers
Kevin Rue-Albrecht (04:54:36): > argh there were some updates to hugo-academic it seems, and my bioc2019 slides are brokenhttps://kevinrue.github.io/teaching/
Federico Marini (08:32:58): > Did we converge on a medium for today’s conference call?
Kevin Rue-Albrecht (08:34:55): > I think Skype usually worked for us. Are there other choices available / preferences? Hangout? Bluejeans?
Federico Marini (08:38:08): > same here, skype has always been good
Federico Marini (08:38:23): > I’ll open it up in advance so that whatever update will be taken care in advance:smile:
Kevin Rue-Albrecht (08:39:32): > Also, I’m just throwing again the link to the Google Docs of ideas I’ve brainstormed:https://docs.google.com/document/d/1qClgSS4pT3cCnS5B-EgBlcXX9WbEjti7n2ukdeb80yE/edit#. > I’m not sure if the pinned linked went unnoticed or you all have your own notes elsewhere, but I haven’t seen any suggestion from anyone else yet. Aaron mentioned he’s got a bunch of them. I look forward to hearing them
Federico Marini (08:40:46): > If we stick to that:@Stephanie Hicks, my username there is fede.maro; then there’s kal.sunrider, lottlotta, and aaronlun110
Stephanie Hicks (08:41:22): > skype works for me! i’m hicks.stephanie
Federico Marini (08:41:24): > > the pinned linked went unnoticed > It did go unnoticed, sorry
Stephanie Hicks (08:41:41): > want me to initiate a call with everyone?
Stephanie Hicks (08:42:10): > @Kevin Rue-Albrechti did see the google doc, but i’ve been trying really hard to intentionally not work on the weekends lately
Stephanie Hicks (08:42:22): > In the office now. Will add comments before our meeting today though
Kevin Rue-Albrecht (08:42:44): > quick time zone check: the call is in 3h20, right?
Federico Marini (08:42:56): > yeps
Stephanie Hicks (08:42:56): > yes that’s the time I have
Kevin Rue-Albrecht (08:44:25): > Great. It’s all in place then.
Kevin Rue-Albrecht (08:49:48): > No worries about the GDoc. I am just really looking forward to ideas people have about improvements and future developments. Just frustrated I can’t keep working on it freely
Stephanie Hicks (08:53:53): > Thanks for your help coordinating@Kevin Rue-Albrecht— very appreciative! and I understand your frustration. Hopefully we will be able to move the project forward with the right person.:slightly_smiling_face:
Aaron Lun (11:59:29): > crap skype
Stephanie Hicks (12:01:15): > no prob let me know when it’s working on your end
Aaron Lun (12:01:50): > first I have to download it.
Stephanie Hicks (12:01:58): > :thumbsup:
Kevin Rue-Albrecht (15:29:52): > @Aaron LunI assume > > Warning: 'centreSizeFactors' is deprecated. > > is a warning that we just need to live with for iSEE R CMD check until the deprecation cycle is over?http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/malbec1-checksrc.html
Aaron Lun (15:58:18): > depends on where it’s coming from. Looks like anormalize()call is still floating around somewhere? ¯*(ツ)*/¯
Kin Lau (17:52:59): > @Kin Lau has joined the channel
2019-08-20
Aaron Lun (00:30:41): > I was wondering why I was feeling so cold today.
Aaron Lun (00:30:48): > Then I realized I had a hole in my pants
Aaron Lun (00:30:52): > and not just a small hole either.
Aaron Lun (00:31:07): > A large rip right on my upper back thigh
Aaron Lun (00:31:12): > Man, if was drafty
Federico Marini (03:43:17): > :smile:
Federico Marini (03:43:29): > I had a similar experience while playing golf
Federico Marini (03:43:33): > with kids
Kevin Rue-Albrecht (07:40:56): > http://dredge.bio.unc.eduthere are 3 links in there, with template apps, e.g.http://dredge.bio.unc.edu/human-neuronal-tissue
Kevin Rue-Albrecht (07:42:28): > I can’t wait for my next project to draw the various tissues surveyed, only so that user can click on the visual picture instead of selecting it from a dropdown menu … - File (PNG): image.png
Kevin Rue-Albrecht (07:43:11): > The GUI offers two images like that, so that users can do interactive differential expression between “this area” and “that area”
Kevin Rue-Albrecht (07:43:30): > I still can’t decide whether it’s genius, or completely useless
Kevin Rue-Albrecht (09:04:05): > That’s an interesting oneshiny::debounce() > > Transforms a reactive expression by preventing its invalidation signals from being sent unnecessarily often. This lets you ignore a very “chatty” reactive expression until it becomes idle, which is useful when the intermediate values don’t matter as much as the final value, and the downstream calculations that depend on the reactive expression take a long time. debounce and throttle use different algorithms for slowing down invalidation signals; see Details.
Kevin Rue-Albrecht (09:43:46): > Anyone remembers where is our latest list of omics experiment types that are represented asSummarizedExperimentobjects in Bioc packages?
Aaron Lun (11:18:03): > If I didn’t know what the tissues were beforehand, damned if the picture’s going o help me.
Kevin Rue-Albrecht (11:20:47): > Spoke too soon again: their “Mouse embryonic tissue” has dropdown selections instead. It would have been nuts if they didn’t have that to start with, before the visual diagrams were implemented.
Kevin Rue-Albrecht (11:21:59): > Anyway. Let’s be clear, I’m not drawing organs and cell types usinggeom_polygonanytime soon
Kevin Rue-Albrecht (11:23:28): > Oh wait, thereisgeom_map()that could probably be hacked to draw organisms and cell atlases I suppose
Aaron Lun (11:34:29): > CUSTOM PANEL.
Aaron Lun (11:34:44): > Anything I don’t want to do is going to be slapped with that label.
Federico Marini (14:32:45): > wait I recall gg_anatomy or something similar
Aaron Lun (14:33:29): > @Kevin Rue-AlbrechtDid you track down where those centreSizeFactors calls were coming from? They should be destroyed.
Federico Marini (14:34:18): > https://github.com/jespermaag/gganatogram
Federico Marini (14:34:39): > this does deserve a custom panel:slightly_smiling_face:
Kevin Rue-Albrecht (14:54:38) (in thread): > It happens during normalise() if I recall correctly
Kevin Rue-Albrecht (14:56:21) (in thread): > here I believe:https://github.com/davismcc/scater/blob/master/R/normalizeSCE.R#L76
Aaron Lun (15:23:02) (in thread): > Why do we have normalize calls?
Kevin Rue-Albrecht (15:40:50) (in thread): > that’s how we originally preprocessed the SCE in examples, tests, and vignette
Aaron Lun (16:37:18) (in thread): > Probably can strip them out and replace with logNormCounts, then.
Kevin Rue-Albrecht (17:35:56) (in thread): > I’ve reinstalled SCE from GitHub, and I get the following error when trying to compiled the “basic.Rmd” vignette of iSEE > > > sce <- logNormCounts(sce) > Error in value[[3L]](cond) : failed to normalize 'altExp(x, 1L)' > Error in assay(x, exprs_values): 'assay(<SingleCellExperiment>, i="character", ...)' invalid subscript 'i' > 'counts' not in names(assays(<SingleCellExperiment>)) >
Kevin Rue-Albrecht (17:36:15) (in thread): > Looks like it comes from here > > for (i in use_altexps) { > tryCatch({ > altExp(x, i) <- FUN(altExp(x, i), size_factors = original, > center_size_factors = center_size_factors) > }, error = function(err) { > stop(paste0(sprintf("failed to normalize 'altExp(x, %s)'\n", > deparse(i)), err)) > }) > } >
Aaron Lun (17:36:43) (in thread): > Presumably the spike-ins don’t have “counts” field.
Aaron Lun (17:36:48) (in thread): > pretty much what it says, really.
Aaron Lun (17:37:13) (in thread): > Setuse_altexps=FALSEto ignore the alternative experiments during normalization.
Kevin Rue-Albrecht (17:37:20) (in thread): > oh right,altExpareSE objects, so they have their assays, OK, still catching up a bit
Aaron Lun (17:37:52) (in thread): > That’s one of the difficulties with sticking everything together, because the altexps may not have the same assays as the mainexp.
Kevin Rue-Albrecht (17:38:16) (in thread): > Ugh.. they havetophat_countsindeed, as only our vignette only renamed the “main” assays
Aaron Lun (17:38:42) (in thread): > should be easy enough withassayNames(altExp(x))[1] <- "counts".
Kevin Rue-Albrecht (17:41:35) (in thread): > Yup. Not it saying it isn’t easy [to workaround], I was just wondering was would look the most like a typical workflow: > - renaming and normalizing the altExp, > - oruse_altexps=FALSE
Aaron Lun (17:42:02) (in thread): > Or you could leave them as “tophat_counts” and just useexprs_values="tophat_counts"inlogNormCounts.
Kevin Rue-Albrecht (17:42:32) (in thread): > works for me
2019-08-21
Aedin Culhane (02:06:47) (in thread): > There is a library on gxa github that might be useful. I’ll pull info tomorrow.
Aedin Culhane (02:07:30) (in thread): > Great find.
Kevin Rue-Albrecht (03:41:56) (in thread): > Could be fun with atlas projects. Hell maybe even cell type signatures
Federico Marini (08:49:56) (in thread): > Yep
Federico Marini (08:50:08) (in thread): > I actually meant it for real as a suggestion
Federico Marini (08:50:22) (in thread): > not a “custom panel label” in@Aaron Lun’s sense:smile:
Kevin Rue-Albrecht (08:55:23) (in thread): > Like you meant voice recognition as a real suggestion too, right?:grimacing:
Federico Marini (08:58:33) (in thread): > Oh that was a totally different story
Kevin Rue-Albrecht (09:35:48) (in thread): > I like that story:grin:
Federico Marini (15:06:21): > Related to the GH issue
Federico Marini (15:06:40): > https://github.com/csoneson/iSEE/issues/312I checked nowhttps://bioconductor.org/packages/release/bioc/vignettes/iSEE/inst/doc/basic.html#4_description_of_the_user_interface
Federico Marini (15:07:20): > We are not mentioning the canvas in the header panel
Federico Marini (15:07:36): > and we are keeping the sidebar
Federico Marini (15:07:56): > Guess it would make sense to update this part, in both release and devel
Kevin Rue-Albrecht (15:14:02): > Re/Open an issue to remind me/us when anyone gets the chance
Federico Marini (15:58:54): > I’ll reopen the issue then
Kevin Rue-Albrecht (19:22:30): > Looks like it’s building on Travis. It did pass on my laptop, and i’ve just opened a basic app. If anyone feels like testing and merginghttps://github.com/csoneson/iSEE/commits/kra-lognormcountswhile I’m offline, go ahead
2019-08-27
Federico Marini (03:58:02): > Guess who dreams to see a piece of iSEE in the cover art for the OSCA paper:slightly_smiling_face:
Kevin Rue-Albrecht (05:23:42): > Did you see news about that paper? Last time I checked the ms tracking system they were still dispatching it back to reviewers
Kevin Rue-Albrecht (05:24:00): > Nvm
Kevin Rue-Albrecht (05:24:12): > I just saw the messages in the other channel
Federico Marini (05:25:32): > :slightly_smiling_face:
2019-08-30
Federico Marini (11:18:15): > Someone forgot us:https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-6053-y - Attachment (BMC Genomics): Single Cell Explorer, collaboration-driven tools to leverage large-scale single cell RNA-seq data > Single cell transcriptome sequencing has become an increasingly valuable technology for dissecting complex biology at a resolution impossible with bulk sequencing. However, the gap between the technical expertise required to effectively work with the resultant high dimensional data and the biological expertise required to interpret the results in their biological context remains incompletely addressed by the currently available tools. Single Cell Explorer is a Python-based web server application we developed to enable computational and experimental scientists to iteratively and collaboratively annotate cell expression phenotypes within a user-friendly and visually appealing platform. These annotations can be modified and shared by multiple users to allow easy collaboration between computational scientists and experimental biologists. Data processing and analytic workflows can be integrated into the system using Jupyter notebooks. The application enables powerful yet accessible features such as the identification of differential gene expression patterns for user-defined cell populations and convenient annotation of cell types using marker genes or differential gene expression patterns. Users are able to produce plots without needing Python or R coding skills. As such, by making single cell RNA-seq data sharing and querying more user-friendly, the software promotes deeper understanding and innovation by research teams applying single cell transcriptomic approaches. Single cell explorer is a freely-available single cell transcriptomic analysis tool that enables computational and experimental biologists to collaboratively explore, annotate, and share results in a flexible software environment and a centralized database server that supports data portal functionality.
Kevin Rue-Albrecht (11:24:49): > > Comparison to other software > they only mention cellxgene
Federico Marini (11:25:42): > Yups
Kevin Rue-Albrecht (11:36:49): > it’s crazy - i know we’ve massively slowed down development and advertising in the last couple of months, but seriously, don’t we have a minimum of visibility with Bioc? an acknowledgment that we exist would be nice
Aaron Lun (11:37:09): > I wuldn’t worry about it.
Federico Marini (11:40:07): > I think power users do appreciate it “silently”
Federico Marini (11:40:19): > we don’t do Seurat-like ads, that’s true
Federico Marini (11:44:05): > (I would lie if I say I don’t feel sorry of the stars being still below 100)
Federico Marini (11:44:17): > :stuck_out_tongue:
Kevin Rue-Albrecht (13:45:19): > A long-term low-priority idea just popped back into mind: > briefly, wrapping all the settings DataFrame’s into a single object of class e.g.iSEEParampros and cons? > (-) the current startup script is pretty clear and more durable thansaveRDS, which could save an object that may not be valid in the future, without the script used to create the object > (+) the configuration of individual panels could be separated from more general app arguments
Kevin Rue-Albrecht (13:46:35): > > ip <- iSEEParams( > redDimArgs = NULL, colDataArgs = NULL, featAssayArgs = NULL, > rowStatArgs = NULL, rowDataArgs = NULL, sampAssayArgs = NULL, > colStatArgs = NULL, customDataArgs = NULL, customStatArgs = NULL, > heatMapArgs = NULL, redDimMax = 5, colDataMax = 5, > featAssayMax = 5, rowStatMax = 5, rowDataMax = 5, > sampAssayMax = 5, colStatMax = 5, customDataMax = 5, > customStatMax = 5, heatMapMax = 5, > initialPanels=NULL) > iSEE <- function(se, > ISEEParam = ip, > annotFun=NULL, > customDataFun=NULL, > customStatFun=NULL, > customSendAll=FALSE, > colormap=ExperimentColorMap(), > tour=NULL, > appTitle=NULL, > runLocal=TRUE, > voice=FALSE) >
Kevin Rue-Albrecht (13:47:02): > Haven’t thought through the whole thing, there may be more pros/cons
Kevin Rue-Albrecht (14:10:49): > (+) accessors might be more convenient than$andlist()hacks for setting the values in the various columns and rows of the DataFrame
Kevin Rue-Albrecht (14:11:10): > anyway, just throwing the idea here, to see if it’s even worth an issue on the repo
2019-08-31
Federico Marini (04:32:42): > A solution to keep backwards compatibility:
Federico Marini (04:33:12): > - we have the iSEEparam parameter which is a list with all the params > - AND we keep the params to be specifically overridden?
Federico Marini (04:33:55): > say, what is not specified in the iSp gets the default, and what is also explicitly defined as param, that is the value taken
Federico Marini (04:34:06): > De facto we would have “mode- objects”
Federico Marini (04:34:32): > As a saturday morning thought, I can say I like ti
Federico Marini (04:34:34): > it
2019-09-02
Charlotte Soneson (09:15:54): > Last chance for comments on the poster for our annual meeting; will send it for printing tomorrow:slightly_smiling_face: - File (PDF): Soneson_annual_meeting_2019.pdf
Federico Marini (09:20:16): > We are really splitting the single hair here: > - t-SNE instead of tSNE? > - color instead of colour? (we usecolorin the app)
Federico Marini (09:20:48): > Otherwise I stick to my “great job” note of 6 weeks or so ago
Kevin Rue-Albrecht (10:11:02): > Same thing here: > - actual arrows instead of->? (bottom left text). To be honest I can’t even find how to that > > - moving the following text higher, and stretching it to a width similar to the author and affiliation text boxes > > iSEE is an R package for reproducible, interactive data exploration. The interface consists of a set of panels, displaying different aspects of the input object. Eight built-in panel types are provided, but custom panels can easily be defined if more precise control over the output is desired. > - Minor edits > > Export R code that exactly reproduces all figures currently visible. > > Export R code that allows initialization of iSEE in the current configuration and layout. > > Launch a guided tour: either the default quick start introduction~to the interface~, or a custom step-by-step tour adapted to any data set.
Kevin Rue-Albrecht (10:12:13): > Here is a quick illustration of what I mean about the text to move (I just put a red box to highlight the change). - File (PDF): Soneson_annual_meeting_2019.pdf
Charlotte Soneson (12:03:25): > Thanks! I’ll fix those things:slightly_smiling_face:
Kevin Rue-Albrecht (14:02:42): > Uh. The GitHub README is probably lagging behind the bioc-build: it reports error on devel, when the daily report looks all green (https://github.com/csoneson/iSEE)
Aaron Lun (14:04:59): > This is typical.
Federico Marini (15:54:16): > In a more sober condition:stuck_out_tongue:I think the ideas of iSEEmodes as param is actually quite good
Federico Marini (15:54:33): > in terms of quite damn good
Federico Marini (15:55:11): > I pointed out in the outlook for the upcoming workshop that one thing to concretely work on is to lower the barriers for effective first adoption of iSEE
Kevin Rue-Albrecht (16:02:37): > Good, thanks for the confirmation. I cannot agree more about “lower the barrier”, even more so after my recent review experience
Kevin Rue-Albrecht (16:09:49): > Damn (pun intended - 666). I noticed that I’m just 4 commit behind Aaron right now… yeah ok on this repo - File (PNG): image.png
Federico Marini (16:10:21): > Let’s spot a typo for you then:smile:
Federico Marini (16:11:02) (in thread): > But it is. Thing is: the tool works. so this gets us some power users, that stick to us (I guess)
Federico Marini (16:11:42) (in thread): > The occasional ones, we need to tie them in by showing how to do not “just many things” but probably “exactly that thing”
Kevin Rue-Albrecht (16:13:53) (in thread): > There are a few easy issues, e.g.https://github.com/csoneson/iSEE/issues/296
Kevin Rue-Albrecht (16:14:21) (in thread): > I’m just mentally exhausted when I get home lately
Federico Marini (16:16:34) (in thread): > actually, more than a typo: we do still have to fix the vignette description of the sidebar+which we do nto have any more
Kevin Rue-Albrecht (16:17:07) (in thread): > I think so, the issue is still open, and I haven’t done it myself
Federico Marini (16:21:45) (in thread): > it was more a reminder to myself
Federico Marini (16:21:59) (in thread): > but I’m also quite drowned out
Federico Marini (16:22:11) (in thread): > more because of the local wine fest tho
Kevin Rue-Albrecht (16:22:30) (in thread): > lol… i wish
Kevin Rue-Albrecht (16:33:24): > tada:https://github.com/csoneson/iSEE/issues/314
Aaron Lun (18:41:39): > I AM SO BORED
2019-09-03
Kevin Rue-Albrecht (09:55:52): > random ideas of the day (feasibility and effort unknown): > -iSEE_preview(sce, redDimPlotArgs, ..., output_dir): dump all the plots and scripts that the iSEE interface would show, without having to launch the interface itself and export the script, edit it, and save the plots - save time > -iSEE_random(sce): launch an instance with panel settings automatically populated with random covariates sampled from the object - Chatroulette-style analyses, a chance to find unexpected associations
2019-09-06
Aaron Lun (20:28:19): > so bored
Aaron Lun (20:28:22): > belting out some cher
Aaron Lun (20:31:36): > switched to whitney
2019-09-07
Kevin Rue-Albrecht (06:49:09): > I thought you’d more Queen, “Don’t stop me now”-style
Aaron Lun (06:55:21): > I am actually still awake.
Aaron Lun (06:55:27): > Had a bout of gastro so I couldn’t sleep.
Aaron Lun (06:55:29): > Just writign code.
Kevin Rue-Albrecht (06:57:07): > Ouch - sorry to hear. Take care of yourself though. I’m coming down with a cold, finally, after all those changes of temperature this summer
Aaron Lun (06:57:58): > That’s cause we’re getting old
Kevin Rue-Albrecht (06:58:25): > damn you, smaller thymus:fist:
Aaron Lun (07:17:46): > Should probably wash my hands before I start cooking
Aaron Lun (07:17:58): > And not use the same scissors for cooking and cutting my hair
2019-09-16
Kevin Rue-Albrecht (05:59:59): - File (JPEG): Image from iOS
Federico Marini (06:58:30): > yet another browser?
Federico Marini (06:59:14): - File (PNG): image.png
Federico Marini (06:59:18): > HOLISTIC VIEW
Federico Marini (06:59:23): > we forgot that in our ms
Aaron Lun (11:29:06): > Think it’s just the rebranded product from their acquisition of spatial transcriptomics.
Kevin Rue-Albrecht (12:08:07): > Oh yeah it is. I just came out of their seminar today. I should have clarified
2019-09-17
Laurent Gatto (04:43:53): > @Laurent Gatto has joined the channel
Federico Marini (14:57:24): > Another idea to support themodesidea
Federico Marini (14:57:59): > it would be a good way to change the setup on the same dataset in one shot, if it is possible to upload
Federico Marini (14:58:31): > thinking of this, because otherwise when we upload an RDS “hub-like” we only have access to the default config
Kevin Rue-Albrecht (15:00:50): > Not sure what ” in one shot” means in this case
Kevin Rue-Albrecht (15:04:52): > On a related note though, I took some time recently to think further about theiSEEParamsclass, and ran into the conceptual issue of the validity method; similarly to theExperimentColorMapclass, it would need a separate method to test whether theparamsand thesceobject are compatible
Federico Marini (15:10:42): > > Not sure what ” in one shot” means in this case > Without changing manually all settings
Federico Marini (15:10:51): > but more uploading the config’d mode
Kevin Rue-Albrecht (15:12:56): > right, well that goes back to storing it all in one object then
Federico Marini (15:13:35): > pretty much, yes
2019-09-18
Federico Marini (11:28:41): > We’re failing on devel
Federico Marini (11:28:46): > > 10: plotReducedDim(object = object, ncomponents = ncomponents, dimred = reddim_name, > ...) > 11: stop(sprintf("'ncomponents' is larger than 'ncols(reducedDim(object, '%s'))'", dimred)) >
Federico Marini (11:59:40): > unrelated: 99:star:
Federico Marini (12:00:05): > Aaron activate your Public relationships machinery:party_parrot:
Kevin Rue-Albrecht (12:19:33): > Actually, I’ve gone to a R user group meetup in Oxford on Monday, and mentioned it around. That said, I don’t recognize the last couple of star’ers as Oxford people
Federico Marini (15:18:46): > btw, we are also here
Federico Marini (15:18:47): > https://bio.tools/isee - Attachment (bio.tools): iSEE > Provides functions for creating an interactive Shiny-based graphical user interface for exploring data stored in SummarizedExperiment objects, including row- and column-level metadata. Particular attention is given to single-cell data in a SingleCellExperiment object with visualization of dimensionality reduction results.
Federico Marini (15:38:35): > > Actually, I’ve gone to a R user group meetup in Oxford on Monday, and mentioned it around. That said, I don’t recognize the last couple of star’ers as Oxford people > I’ll be this friday in Köln to present
Kevin Rue-Albrecht (15:52:41) (in thread): > Nice. > Not sure if they’re website is meant to update with new versions, but their “Source package ›” points to version 1.0.1
Federico Marini (16:10:38) (in thread): > noticed
Federico Marini (16:10:44) (in thread): > but hey.
Charlotte Soneson (17:07:37): > Institute retreat poster session done, and I think we have a bunch of new users:slightly_smiling_face:recurrent question on how to use iSEE with microscopy images, so I may end up looking into that with some of them.
2019-09-19
Kevin Rue-Albrecht (03:01:15): > > microscopy > ? > are we talkingactualimages?:star-struck:
Charlotte Soneson (03:02:33): > Yes
Federico Marini (03:05:47): > Huch?
Federico Marini (03:05:49): > Cool?
Charlotte Soneson (03:18:16): > Don’t know yet exactly what they want to do, but could be a nice use case.
Kevin Rue-Albrecht (03:18:55): > Spatial iSEE
Kevin Rue-Albrecht (03:24:49): > Wow. I just realized thatEBImagealready existed in 2010:https://bioconductor.org/help/course-materials/2010/BioC2010/100730-seattle-huber.pdf
Kevin Rue-Albrecht (03:32:03): > What I’d be curious is whether it’d be useful /possible to convert microscopy images into the “country map” styles of R plots > like that kind of thing:https://ggplot2.tidyverse.org/reference/map_data.html - Attachment (ggplot2.tidyverse.org): Create a data frame of map data — map_data > Easily turn data from the maps package in to a data frame suitable for plotting with ggplot2.
Kevin Rue-Albrecht (16:57:01): > > We’re failing on devel > I’m not sure why we fail on devel only on 2 of the 3 machines:http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/malbec1-checksrc.html
Kevin Rue-Albrecht (17:00:32): > I’d hope it’s just about some new package versions that haven’t propagated to “tokay” and “merida”
Federico Marini (17:01:17): > let’s wait’n’see
Kevin Rue-Albrecht (17:02:10): > Unrelated: if you wanna update the docs before the release, there’s this one:https://github.com/csoneson/iSEE/issues/312
Kevin Rue-Albrecht (17:02:40): > Personally, I’ve lefthttps://github.com/csoneson/iSEE/issues/302unattended for far too long
2019-09-21
Kevin Rue-Albrecht (11:45:42): > @Aaron LunHow’s that forhttps://github.com/csoneson/iSEE/issues/302? - File (PNG): image.png
Kevin Rue-Albrecht (11:52:40): > Also, I updatedisColorMapCompatibleto display all errors, instead of just the first one. - File (PNG): image.png
Kevin Rue-Albrecht (11:55:27): > I’ll push the updated code as soon asR CMD checkpasses locally. Here is a MWE made up of our unit tests bits for testing: > > stopifnot( > require(scRNAseq), > require(scater) > ) > > # Example data ---- > sce <- ReprocessedAllenData(assays = "tophat_counts") > > sce <- logNormCounts(sce, exprs_values="tophat_counts") > sce <- runPCA(sce) > sce <- runTSNE(sce) > > rowData(sce)$num_cells <- rowSums(assay(sce, "tophat_counts") > 0) > rowData(sce)$mean_count <- rowMeans(assay(sce, "tophat_counts")) > # Add a groupable field in rowData > rowData(sce)$letters <- sample(letters[1:3], nrow(sce), TRUE) > > sizeFactors(sce) <- runif(ncol(sce)) > > colData(sce)[["nested"]] <- DataFrame( > nested1 = runif(ncol(sce)), > nested2 = sample(letters, ncol(sce), TRUE), row.names = colnames(sce) > ) > > > # Example custom color maps ---- > > COUNT_COLORS <- function(n){ > c("black","brown","red","orange","yellow") > } > > FPKM_COLORS <- viridis::inferno > > TPM_COLORS <- viridis::plasma > > ASSAY_CONTINUOUS_COLORS <- function(n){ > c("black","purple","yellow") > } > > QC_COLOR_FUN <- function(n){ > qc_colors <- c("forestgreen", "firebrick1") > names(qc_colors) <- c("Y", "N") > return(qc_colors) > } > > > ecm_manyAssays <- ExperimentColorMap( > assays = list( > counts = COUNT_COLORS, > tophat_counts = COUNT_COLORS, > cufflinks_fpkm = FPKM_COLORS, > cufflinks_fpkm = FPKM_COLORS, > rsem_tpm = TPM_COLORS, > another = TPM_COLORS, > yet_another = TPM_COLORS, > last_one_i_promise = TPM_COLORS, > oh_well = TPM_COLORS > ) > ) > > iSEE(sce, colormap = ecm_manyAssays) >
Aaron Lun (13:24:14): > looks good
Federico Marini (13:24:50): > :thumbsup:
Aaron Lun (13:26:11): > but can’t you just show the errors in the notification as well?
Aaron Lun (13:33:43): > No need to tell them to run something, which would be kind of meaningless for uploaded files.
Kevin Rue-Albrecht (14:06:43): > I initially did that, but as you can see [in the image I posted just above], there can be many errors, and the notification bubbles are not very wide, so the error messages would end up wrapped over multiple lines each:face_vomiting:
Kevin Rue-Albrecht (14:10:16): > … and we exposedisColorMapCompatiblein the NAMESPACE, users might as well learn to use it
Aaron Lun (14:11:00): > Well, uploaders will have no idea what’s wrong. Why don’t you catch the first error and report it rather than suggesting them to run a function that they might not be able to run?
Kevin Rue-Albrecht (14:14:03): > Right. I didn’t think of that use case. Makes sense to report all errors in that case then
Aaron Lun (14:15:52): > Well, that’s why you print just the first error.
Kevin Rue-Albrecht (14:54:07): > Alternative solution - File (PNG): image.png
Kevin Rue-Albrecht (14:58:50): > Inspired by Aaron’sif is.null(se)approach; ifcolormapis not compatible, interrupt the server initialization and setoutput$allpanelsto an alternativetagList.
Kevin Rue-Albrecht (15:08:08): > @Federico Marinido you remember how toshow(colormap)in a UI output like the above?showprints to the console, it doesn’t return a character string to print like in the screenshot above
Kevin Rue-Albrecht (15:10:15): > @Aaron Lundo you mind trying the latest commit on branchkra-ecm-serverwith > > ecm_manyAssays <- ExperimentColorMap( > assays = list( > counts = COUNT_COLORS, > tophat_counts = COUNT_COLORS, > cufflinks_fpkm = FPKM_COLORS, > cufflinks_fpkm = FPKM_COLORS, > rsem_tpm = TPM_COLORS, > another = TPM_COLORS, > yet_another = TPM_COLORS, > last_one_i_promise = TPM_COLORS, > oh_well = TPM_COLORS > ) > ) > > iSEE(colormap = ecm_manyAssays) > > and thesce.rdsthat we use for our vignettes?
Federico Marini (15:12:42) (in thread): > renderPrint + verbatimTextOutput?
Kevin Rue-Albrecht (15:13:02) (in thread): > ahhhh I couldn’t remember, thanks
Kevin Rue-Albrecht (15:13:34) (in thread): > that said, I was hoping for something that would fit inside > > output$allPanels <- renderUI({ > tagList( > h1("Invalid color map"), > column(pre(conditionMessage(err)), width=12) > ) > }) >
Kevin Rue-Albrecht (15:17:14) (in thread): > I was hoping to show users the colormap that was packaged with the app, because users only have thesce.rdsfile that they upload. They don’t have the colormap itself.
Kevin Rue-Albrecht (15:18:25) (in thread): > Oh well. It’s already a good start like this
Kevin Rue-Albrecht (15:19:36): > If you’re happy then we could merge and move on.
Kevin Rue-Albrecht (16:01:18): > @Federico MariniI managed to prototype it locally, but I wouldn’t like to commit it, it’s not really useful. I’d stick with the last one above - File (PNG): image.png
Federico Marini (16:31:40): > would it make sense to make it “accessible” as a modal
Federico Marini (16:31:52): > that pops up and displays useful stuff about the ECM
Federico Marini (16:31:53): > ?
Kevin Rue-Albrecht (17:37:09): > It just feels a bit overkill until we’re given some real use cases where an app is preloaded with an ECM while users can upload random SCE
Kevin Rue-Albrecht (17:40:11): > Probably a better argument yet: the last screenshot shows you how an ECM is displayed, which ellipses a number of colormaps when there are more than 4. > Even if we offered a modal, we’d still have to find a clear way to show a comprehensive content of the ECM… perhaps even including the body of the various functions, as discrete colormaps expect specific factor level names … ugh
Kevin Rue-Albrecht (17:51:09): > @Aaron LunSorry to ping you again. I’m gonna hit the hay. I’m happy enough with the latest implementation (screenshot below) as it gives all the error messages in the interface. So, if you give me the green light I’ll merge tomaster. > As said, the only information not given to the user in this implementation is the content of the ECM object itself if it’s an instance designed to upload objects. But at that point I guess our guidelines would probably be that users should contact the app maintainer and ask what kind of ECM they preconfigured and why… right? - Attachment: Attachment > Alternative solution
Kevin Rue-Albrecht (17:57:10) (in thread): > Actually, given that the latest implementation instantiate a dashboard where the header menu is visible, we could dump the ECM content in something similar to thesessionInfomodal. > Still, we would still have the issue of displayingcomprehensivelythe content of the ECM, to make this any helpful
Aaron Lun (20:11:15): > I don’t understand what the problem was with showing a session notification.
Aaron Lun (20:11:43): > Just throw the first error as a warning in the session notification. Having the app throw up wholesale is too strict when the uploader can’t really do anything about it.
Aaron Lun (20:13:30): > In other words; if we could trust uploaders to set up their SCE properly, they would just run their own instance.
2019-09-22
Kevin Rue-Albrecht (04:22:18): > > Well, uploaders will have no idea what’s wrong. Why don’t you catch the first error and report it rather than suggesting them to run a function that they might not be able to run? > 1. Showing a notification while letting server initialization to complete requires the app to define a compatible ECM to replace the incompatible one. I previously solved that by replacing the custom ECM by the default empty ECM (mostly because that was the most immediate “compatibility fix”). Again this comes back to “why did the developer preconfigure a custom ECM, and why would users upload incompatible objects”. I imagine the use case to be a group of analysts who share a specific workflow (e.g. ARMOR) and set up an app to visualize the output using a shared custom colormap. In which case, the objects should be immediately compatible with the app. If users want to upload incompatible object, well … they might as well deploy their own app, no? > 2. If we display only the first error and there are 10 errors, users would have to upload their SCE 10 times to fix all the issues one after the other. Perhaps I could trigger 1 notification per error? > 3. You said it yourself: if users cannot run the functionisColormapCompatible, I thought we might as well run it for them and display all the errors (back to point 2.) > 4. If an app is set up with a custom ECM, I’d argue thatweare not the ones who need to trust uploaders, the developer who preconfigured and deployed the app is the one who’d have to give users the proper guidelines/instructions. > 5. The main interface has more “real estate space” than the small notification bubbles to display error messages. (back to point 2.) > > All that said, I can revert back to the notification strategy if you think that it’s the best option for us and for users. > I must admit that one advantage of the notification is that the server properly implements the observers for the dashboard header menus; right now, the button are visible but the observers aren’t active
Kevin Rue-Albrecht (10:08:34): > Anyway - stand by, I’m reverting to the notification and empty ECM implementation.
Kevin Rue-Albrecht (10:28:22): > Here’s how it looks now. > - notification stay for 10s each > - 1 warning notification to say that that the default (empty) ECM is applied because the SCE is not compatible with the custom ECM > - 1 error notification per compatibility error between the SCE and ECM > - oh, and the panels show up as usual, they just took more than 10s to show up, so I couldn’t screenshot the notificationandthe panels - File (PNG): image.png
Aaron Lun (13:01:29): > That looks better. 1) Soft fail is necessary. 2) If you can show all errors, that is fine. 3) Yes. 4) Fine, but a soft fail. 5) All errors are currently shown via notifications, and it’s bad to have another mode for just this thing.
Kevin Rue-Albrecht (13:03:18): > Cool. As far as I can see, it’s ready for merge tomaster.
Kevin Rue-Albrecht (13:04:40): > FYI, I’m too lazy to run a deprecation cycle for the renaming ofisColorMapCompatibletocheckColormapCompatibility, and the change of return value from logical(1) to a character vector of error messages. Bad practice, but I seriously doubt anyone was using that function yet…
Aaron Lun (13:05:10): > ¯*(ツ)*/¯
Aaron Lun (13:05:17): > It’s probably fine
Aaron Lun (13:05:21): > I’ll just direct all complaints to you.
Kevin Rue-Albrecht (13:05:26): > Deal.
Kevin Rue-Albrecht (13:19:26): > Alright. It’s gone upstream.
Aaron Lun (13:20:43): > :neutral_face:
Aaron Lun (13:20:54): > :moneybag:
Aaron Lun (13:21:00): > :shamrock:
Kevin Rue-Albrecht (13:35:24): > Hm.. My comic book skills must be rusty
Kevin Rue-Albrecht (13:36:05): > On a separate note, I just earned my 1st authorship back by 18 commits:stuck_out_tongue: - File (PNG): image.png
Aaron Lun (13:37:53): > Curses
2019-09-23
Kevin Rue-Albrecht (08:59:48): > @Federico Marini - File (PNG): image.png
Federico Marini (13:05:07): > I saw that on Sat and promptly messaged Charlotte:smile:
Kevin Rue-Albrecht (13:43:56): > ha ha! iSEE sub-channels eh?:stuck_out_tongue:
Federico Marini (14:08:59): > eehhehe
Federico Marini (14:09:20): > it’s the italian connection
Federico Marini (14:09:25): > huh, about that
Federico Marini (14:10:14): > Aaron should also get some fine italian ma’am, then we all original iSEEers would have an Italian in the couple
Federico Marini (14:10:15): > :smile:
Kevin Rue-Albrecht (14:28:15): > Unrelated: why the hell do we have > > setClass("ExperimentColorMap", > contains="Vector", > > ???https://github.com/csoneson/iSEE/blob/master/R/AllClasses.R#L76ECM is not a vector… > I might have accidentally copy pasted from another S4 class definition. > I canmaybesee a point to have global metadata, but it clearly doesn’t fit withelementMetadata
2019-09-24
Kevin Rue-Albrecht (05:53:06): > https://immunology.sciencemag.org/content/4/39/eaaw8405 > > All visualizations were generated with the iSEE package (68). - File (PNG): image.png
Kevin Rue-Albrecht (05:57:23): > Mmh.. it mostly their Figure 2 > > Fig. 2. Expression of FX by tumor-associated immune cells. > And “amusingly”, they don’t have a legend for panel C (tSNE)
Federico Marini (06:37:27): > I tried to have it put in, they did not like it
Kevin Rue-Albrecht (06:39:39): > Oh damn. Didn’t see your name there:joy:
Kevin Rue-Albrecht (06:39:53): > I really shouldn’t work before noon:sleeping:
Federico Marini (06:42:01): > > Oh damn. Didn’t see your name there > Thought you were just being provocative
Federico Marini (06:42:02): > :smile:
Federico Marini (06:42:12): > I told them one can do lots of figures with that
Federico Marini (06:42:22): > -> they went all in with “all viz”
Federico Marini (06:42:25): > nice touch
Kevin Rue-Albrecht (06:44:23) (in thread): > No provocation at all. That was just raw reaction to the missing figure panel legend:sweat_smile:
Kevin Rue-Albrecht (06:45:59): > Nice touch, but I must say it was a bit of a letdown when I scrolled through all the figures and realized that “all viz” was essentially restricted to Figure 2. I even went to the SM (supp material) in hope of more viz:yum:That said, it can’t be all iSEE (yet):wink:
Kevin Rue-Albrecht (06:46:43): > I’m just greedy ^^
Federico Marini (06:48:38): > > I even went to the SM (supp material) > Gotcha, extra download for the altmetric scores:grin:
Federico Marini (06:50:42) (in thread): > Thing is those cluster were “just clusters” with no major need to see what it was. Indeed a little puzzling
Federico Marini (06:50:48) (in thread): > but back then there was no SingleR
2019-09-25
Kevin Rue-Albrecht (12:35:50): > Random idea of the day: a free text panel type for users to type notes during their analysis. I’ve run out of real estate space on my screen to fit a text editor next to the iSEE window
Kevin Rue-Albrecht (12:36:25): > Bonus point if it immediately renders as Markdown
Kevin Rue-Albrecht (12:38:19): > … come to think of it, it’s almost allowing users to write their manuscript in the UI …Manuscript text panel 1?
Federico Marini (12:42:19): > would be neat to have
Federico Marini (12:42:30): > say, record the stream of thoughts
Federico Marini (12:42:55): > me likes it
Federico Marini (12:43:56): > another aceEditor, and some includeMarkdown uiOutput that picks on that and renders it?
Federico Marini (12:44:04): > Ok, entering inception mode
Federico Marini (12:44:09): > brain’s fuse
Federico Marini (12:44:13): > need a beer:smile:
Kevin Rue-Albrecht (12:45:56): > Yup. Initially, I thought of it because I wanted to keep a list of gene to examine visible alongside the app (i.e., bringing textintothe app). But then I realized that it could be just as useful to take stuff out of the app. Could be in the same panel, e.g.: > > # To do list > - a > - b > # Observations > Blah. >
Federico Marini (12:46:17): > mostly,blah:slightly_smiling_face:
Kevin Rue-Albrecht (12:47:24): > And then the famousactionButtonI’ve wanted since the start: “Export as ”
Kevin Rue-Albrecht (13:12:53): > Oh man, and the ability to cross reference plot panels would be fun.:sweat_smile:
Federico Marini (14:28:02) (in thread): > Write up M&M formatted for Naturebutton
2019-09-26
Kevin Rue-Albrecht (17:40:19): > Drumrolls please for branchkra-param
Kevin Rue-Albrecht (17:43:01): > To be enjoyed with a side of: > > example(iSEEParam) # ?iSEEParam > initialPanels <- DataFrame(Name="Column data plot 1", Width=6) > iSEE(sce, PARAM=isp, initialPanels=initialPanels) >
Kevin Rue-Albrecht (17:44:00): > Then go for “Display panel settings” and scroll to the bottom.
Aaron Lun (17:44:01): > What’s the point of this?
Federico Marini (17:45:07): > I think it is a proof of principle of the iSEEParam idea
Kevin Rue-Albrecht (17:45:17): > Seehttps://github.com/csoneson/iSEE/issues/314
Aaron Lun (17:46:28): > Why?
Kevin Rue-Albrecht (17:46:36): > All DataFrames of parameters packaged in a single object > Makes the call toiSEE(sce, params)muchshorted too
Aaron Lun (17:46:51): > It just shifts the code from one place to another.
Kevin Rue-Albrecht (17:48:29): > I also like the idea of separating the configuration of individual panels (*Args) from the more general app parameters (initialPanels,customFunctions, …)
Aaron Lun (17:48:30): > The benefit (compared to other usages of this PARAM pattern) is not obvious.
Aaron Lun (17:48:43): > Seems like another level of indirection.
Kevin Rue-Albrecht (17:49:26): - File (PNG): image.png
Kevin Rue-Albrecht (17:49:34): - File (PNG): image.png
Aaron Lun (17:49:43): > What’s the deal?
Aaron Lun (17:50:35): > There has to be some added user value from the extra layer
Kevin Rue-Albrecht (17:51:20): > My initial thought was to give users betters accessors to edit the params.
Aaron Lun (17:51:27): > If you separate it like this, now we have to worry about syncing them.
Aaron Lun (17:51:50): > Why do we have to give accessors?
Aaron Lun (17:52:00): > What’s wrong withDataFrames?
Kevin Rue-Albrecht (17:52:37): > Because at the moment, it<panelType>Args$Field[index] <- valuefor the simplest, and<panelType>Args[[*Data]][[index]] <- list(...)for the list-like parameters
Aaron Lun (17:53:20): > Why don’t you write a function that just modifies the DFs?
Kevin Rue-Albrecht (17:53:45): > I’m wondering whether we could help users with some kind ofredDimArgs(params, field, index) <- value, and handle the assignment and type checking to make sure users don’t mess with it
Aaron Lun (17:54:04): > Well, you can’t have the type checking in that method, because you have to check it again anyway when you enteriSEE.
Aaron Lun (17:54:13): > No guarantee that it’s sync’d up to the SE.
Kevin Rue-Albrecht (17:54:16): > yup - I noticed that
Kevin Rue-Albrecht (17:54:23): > also because the SE isn’t stored with the params
Kevin Rue-Albrecht (17:55:08): > i don’t want to jump arguments, but I was also wondering whether a class of params could help with the modes
Aaron Lun (17:56:48): > The modes can handle themselves, they’re dedicated functions.
Aaron Lun (17:57:15): > but<panelType>Args$Field[index] <- valuedoes not get any more concise.
Aaron Lun (17:58:04): > you might have apanelArgmethod for a DF that automates the list/non-list choice.
Aaron Lun (17:59:31): > But there’s no need for a giantiSEEPARAMclass. That would only be necessary if people were repeatedly using the same sets of arguments multiple times, or if you were passing those arguments down to another function and didn’t want to list them out again. (Modes don’t count here, because you need to modify the arguments in modes, so you can’t just pass thePARAMobject directly as we do withBPPARAMorBSPARAM, etc.)
Kevin Rue-Albrecht (18:01:37): > All fair points.
Kevin Rue-Albrecht (18:04:19): > Oh well. The code’s there. The branch doesn’t take much space to have if a use case comes up that makes it handy for some reason. Otherwise it’s not much work to delete if we decide it’s not going anywhere. We’ve got older branches lingering on GH
Aaron Lun (18:05:17): > If you want to make it easier to customize, then making it easier to modify the DataFrames would be a better source of revenue.
Kevin Rue-Albrecht (18:05:52): > So just a function, rather than a full blown class?
Aaron Lun (18:05:56): > Yes.
Aaron Lun (18:06:14): > Just have a getter and setter for DF, Field, Index.
Kevin Rue-Albrecht (18:07:47): > Well, if you have a strategy in mind, feel free to comment on the GH issue of the class (we can rename the issue if needed). I’m done for today, and I need to get some sleep for whatever awaits me tomorrow. Plus I’m away to see friends this weekend.
Kevin Rue-Albrecht (18:10:16): > For the record, I did give some thought to > > Just have a getter and setter for DF, Field, Index. > while writing the class. > The complication I ran into is that not all Fields are equal or exist in all dataframes. That’s why I thought that a class and accessors could handle some kind of validity checks (e.g. “You can only set aSearchcolumn in the*TableArgsDFs)
Aaron Lun (18:11:14): > Well, if they create a DF from theDefaults()arguments, all the fields that are avaialble are known in advance, so as long as you don’t add new fields, you can just throw against that.
Kevin Rue-Albrecht (18:11:25): > True.
Kevin Rue-Albrecht (18:12:00): > Right. And with that, my Xmas plans just filled up:stuck_out_tongue:
Aaron Lun (18:12:16): > Join the club.
Kevin Rue-Albrecht (18:14:19): > G’night
Aaron Lun (18:18:04): > night
2019-10-10
Federico Marini (11:32:25): > Is it me being nitpicking or should we edit this a little?+
Federico Marini (11:32:26): > https://osca.bioconductor.org/interactive-sharing.html#reproducibility - Attachment (osca.bioconductor.org): Chapter 18 Interactive Interfaces and Sharing | Orchestrating Single-Cell Analysis with Bioconductor > Online companion to ‘Orchestrating Single-Cell Analysis with Bioconductor’ manuscript by the Bioconductor team.
Federico Marini (11:32:55): > namely the name of the button, and the fact that we can also export the config of panels?
Aaron Lun (11:34:16): > less talk, more PRs.
Federico Marini (11:34:34): > :slightly_smiling_face:
Kevin Rue-Albrecht (11:34:50): > i.e., let’s do the talking on the PR, more traceable and effective anyway
Federico Marini (11:34:53): > I like this Aaron guy
Federico Marini (11:45:57): > I don’t think I have writing permissions to OSCAbase
Aaron Lun (11:46:54): > Well, that’s why you make a PR.
Federico Marini (11:47:24): > I was working on the pure clone
Federico Marini (11:47:30): > needed to fork it first
Federico Marini (11:50:35): > PR’s in
Federico Marini (11:50:36): > https://github.com/Bioconductor/OSCABase/pull/11
Kevin Rue-Albrecht (11:56:15): > While we’re on that page, would it be more educative to replace lines like > > colDataArgs$YAxis <- "log10_total_counts" > > by > > colDataArgs[1, "YAxis"] <- "log10_total_counts" > > even if the DataFrame is only 1 row and the second form is slightly more verbose, would it be better to implicitly teach users to set the parameters for individual panels?
2019-10-11
Charlotte Soneson (10:19:07): > https://twitter.com/carmonation/status/1182577214463844352 - Attachment (twitter): Attachment > An #iSEE Shiny web tool to interact with these data is available at https://tilatlas.shinyapps.io/B16_CD8TIL_10X/ . Have fun!
Federico Marini (10:31:50): > Wowzers
Federico Marini (11:09:10): > https://www.sib.swiss/david-gfeller-group - Attachment (sib.swiss): Gfeller David > Our aim is to study interactions between cancer and immune cells. We are focusing on molecular and cellular aspects of cancer immune cell interactions.
Federico Marini (11:09:15): > gotta love those pics
Kevin Rue-Albrecht (11:12:24): > bwahhhh they pushed a.DS_Storefile to GitHub:stuck_out_tongue_winking_eye:https://github.com/GfellerLab/TILAtlas/tree/master/data
Federico Marini (11:12:58): > Come on Kev, maybe Santiago is also in the channel:smile:
Kevin Rue-Albrecht (11:13:28): > I was hoping that “bwaaah” and the smiley were screaming “teasing”
Kevin Rue-Albrecht (11:14:15): > I’d feel more embarassed opening an issue or a PR for that ^^
Federico Marini (11:14:35): > Naah. I ’ll just enjoy we got used like we would like to
Kevin Rue-Albrecht (11:14:35): > Although, itisHacktoberfest … It would look better if my PR count also included repos that aren’t just my own
2019-10-16
Aaron Lun (13:08:40): > Can someone confirm that the iSEE paper’s iframes are fucked up?
Aaron Lun (13:09:25): > If we need to update the app, I want to point it away from the CRUK server.@Federico Marini, can you set up an instance on your end?
Aaron Lun (13:09:53): > And can someone try out my suggestion inhttps://github.com/csoneson/iSEE/issues/182?
Federico Marini (13:10:21): > AFAIK I can’t do it with https:disappointed:
Federico Marini (13:11:34): > Re: your question, you mean clicking on the apps to launch?
Aaron Lun (13:11:38): > Yes.
Federico Marini (13:11:39): > because that works on my side
Federico Marini (13:11:51): > I’m on Mac, Safari
Charlotte Soneson (13:12:01): > Also for me, Mac Chrome
Aaron Lun (13:12:11): > So it doesn’t look all wonky for you?
Charlotte Soneson (13:12:31): - File (PNG): Screen Shot 2019-10-16 at 19.10.52.png
Aaron Lun (13:12:31): > I just get “marionilab.cruk.cam.ac.uksent an invalid response.”
Aaron Lun (13:12:37): > WEll. hm.
Federico Marini (13:12:38): > Mac Ff also good
Federico Marini (13:12:44): > Chrome as well
Federico Marini (13:12:50): > Oh, hi Charlotte:slightly_smiling_face:
Charlotte Soneson (13:12:57): > :wave:
Federico Marini (13:13:16): - File (PNG): image.png
Federico Marini (13:13:26): - File (PNG): image.png
Federico Marini (13:13:31): > no wonkyness, all good in the hood
Aaron Lun (13:14:37): > Hm. Must be the indsutry firewall.
Aaron Lun (13:16:40): > Fine, okay. If someone can give 182 a shot, that would make my life easier.
2019-10-17
Federico Marini (10:44:25): > It is thrrrrrrrowback thursday:tada:
Federico Marini (10:44:36): - File (PNG): image.png
Charlotte Soneson (10:46:39): > :smile:
Federico Marini (10:51:09): - File (PNG): image.png
Federico Marini (10:51:24): > Check out the date:slightly_smiling_face:gotta be an astral alignment thing
Kevin Rue-Albrecht (11:14:28): > Gee.. on 24th December… What the hell Kevin:sweat_smile:
Aaron Lun (11:27:02): > 182!
Federico Marini (14:22:12): > Huh?
Aaron Lun (14:23:24): > Read the github issue.
Kevin Rue-Albrecht (14:25:38): > that makes more sense than listening to their albums all day:stuck_out_tongue:
Kevin Rue-Albrecht (14:27:28): > More seriously, I’ll test now(ish, dinner’s about to be ready). I saw your comments, and thought you tested it yourself before you replied to that comment.
Aaron Lun (14:27:58): > no. filling out forms over here.
Federico Marini (14:32:56): > This one says it all - I copypasted his mini apps and it works
Aaron Lun (14:34:24): > But whether it works in iSEE itself.
Federico Marini (14:34:31): > although, for example 2 you need more something likea,b,c;d
Federico Marini (14:34:40): > on it now
Federico Marini (14:45:15): > gets them genes undefined when I copypaste
Federico Marini (14:46:51): > I am changing e.g. > > selectizeInput( > .input_FUN(.heatMapFeatName), > label="Features:", > choices=NULL, selected=NULL, multiple=TRUE, > options=list(plugins=list('remove_button', 'drag_drop'), > splitOn = I("(function() { return /[,;]/; })()"), > create = I("function(input, callback){ > return { > value: input, > text: input > }; > }"))), >
Aaron Lun (15:25:31): > No, it should just be copy and paste with space separation.
Federico Marini (15:36:57): > nah, did not
Aaron Lun (15:37:43): > But the example works. So figure out how to make it work inside.
2019-10-18
Kevin Rue-Albrecht (04:12:51): > So, I’ve identified the issue. > Basically, users currently need to paste the numeric index of the genes of interest, not their name, e.g. paste1 5 10in the selectize and it works for me. > Also, note that I’ve updated the code (relative to Fede above) to split only on spaces. > > selectizeInput( > .input_FUN(.heatMapFeatName), > label="Features:", > choices=NULL, selected=NULL, multiple=TRUE, > options=list( > plugins=list('remove_button', 'drag_drop'), > splitOn = I("(function() { return /[ ]/; })()"), > create = I("function(input, callback){ > return { > value: input, > text: input > }; > }") > )), > > Bottom line is that if we want to allow users to paste values fromrownames(sce), the JS would need to somehow know how to convert from rownames to numeric index, so that it doesn’t create “undefined” entries. I don’t know right now how to do that. > The alternative being to store rownames asheatMapArgs$FeatNameinstead of row indices, but that would raise other kinds of issues. > I haven’t pushed anything (i.e., the code above is the only thing that I changed), I’ve got a rush til next Thursday to prepare an internal seminar.
Federico Marini (08:21:05): > Well done Kev. Sorry for not following up completely yesterday. Had a stupid presentation to give this morning
Federico Marini (08:21:32): > I tried with names and ENSids but gforgot the selection is actually done behind the scenes with integers
Kevin Rue-Albrecht (08:40:04): > Preparing the bioc2019 workshop was an efficient reminder of that specific point in my case ^^
Federico Marini (08:48:41): > Uh, now that you say workshop
Federico Marini (08:48:55): > Do we want to offer a reloaded version of that for the upcoming eRum conference?
Kevin Rue-Albrecht (08:56:18): > Be my guest
Federico Marini (09:03:12): > Asking because I think the first round of submissions/contribs will open soon-ish
Aaron Lun (11:25:38): > Thanks@Kevin Rue-Albrecht. Probably have to fiddle with thecreate=argument for that…
Aaron Lun (11:25:43): > If only we knew more JS.
Federico Marini (11:25:57): > +1
Federico Marini (11:26:23): > I have so tiny yet powerful ideas that could become instant classics as htmlwidgets:disappointed:
Federico Marini (11:27:25): > Charlotte knows already about my wet dream -https://leonawicz.github.io/HtmlWidgetExamples/ex_dt_sparkline.html- based
Federico Marini (11:27:39): > but for DE results, and miniboxplots split by condition for the groups
Kevin Rue-Albrecht (11:33:17) (in thread): > sarcasm, right?
Kevin Rue-Albrecht (14:07:22): > FYI, “space” is probably not the best separator to paste a list of genes in a selectize:https://github.com/selectize/selectize.js/blob/master/docs/usage.md > > searchConjunction: When searching for multiple terms (separated by space), this is the operator used. Can be ‘and’ or ‘or’ .
Kevin Rue-Albrecht (14:08:14): > I humbly suggest,
Aaron Lun (14:09:07): > Bit harder to get people to format it with a comma.
Aaron Lun (14:09:17): > I was hoping that people could just copy-paste from an excel table.
Aaron Lun (14:09:25): > Newlines get converted to spaces.
Kevin Rue-Albrecht (14:11:57): > AFAIK we can have a range of separators, newline is a good one too, I just had to point out the issue with space, if you just type “1 2 3”, you’ll see that any record that matches those numbers/digits (in any order) is shortlisted
Aaron Lun (14:12:48): > If we can’t get the JS to work, we could just have an upload button.
Aaron Lun (14:13:09): > upload a list or csv file, and we fill in the selectize for you.
Kevin Rue-Albrecht (17:44:41): > > document.getElementById("heatMapPlot1_FeatName").parentNode.children[1].childNodes[1].childNodes[0].childNodes > > will get you a list of up to 1000 options i.e. feature (seemaxOptionsherehttps://github.com/selectize/selectize.js/blob/master/docs/usage.md) > That list of 1000 is fetched using jQuery magic > Bottom line is that it’s not possible to match gene names pasted by users against the full list of text options (i.e. feature names) of the dropdown menu, which is a prerequisite to convert feature names into rownames indices)
Aaron Lun (17:46:02): > Bum.
Kevin Rue-Albrecht (17:46:28): > Happy to be proven wrong, but that 1000 options limit looks like something we can’t easily get around
Aaron Lun (17:46:55): > Well, I don’t mind only having 1000 genes, that seems like it would be more than enough for visualization.
Kevin Rue-Albrecht (17:47:49): > > If we can’t get the JS to work, we could just have an upload button. > It could even be a simple text box like the custom panel arguments. Maybe even a modal that opens on request where users paste their gene list, click a “Submit” button, and we process that like any other UI input on the R/Shiny side
Kevin Rue-Albrecht (17:48:22) (in thread): > 100% agreed. That’s also good to keep a fast turnover between client and server side
Kevin Rue-Albrecht (17:50:28): > Handling all that on the R/Shiny side would also allow us toshowNotificationabout invalid gene names in the mix. I was thinking we could ignore the invalid ones, use only the valid ones, and raise a notification to let users know how many were used/ignored.
Kevin Rue-Albrecht (17:51:02): > If all are invalid, it’d be the same as “Clear” the heat map
Aaron Lun (17:52:35): > Well, if it’s a text box, you want to make sure that the text is cleared as soon as they press “go”, otherwise they might get a misleading idea about what genes are valid. It’s also a bit awkward having a text box next to a selectize in the UI, I could imagine people being confused there. It may be cleaner to have an “upload your file” and we just fill out the details, and people can add or delete things afterwards.
Kevin Rue-Albrecht (17:55:26): > I agree on all points. Though I didn’t explain myself clearly about the text box. Instead of your “Upload” button, I’d imagine a button that opens a modal with that text box (i.e., not in the heat map plot panel itself). Then that text box behaves like you described (i.e., clearing on go, etc.). Then - once the list of features was processed and the heat map updated - users can just close the modal to see the result
Aaron Lun (17:55:53): > that’s… tolerable.
Aaron Lun (17:56:11): > If you throw in an upload file button there, I’m game.
Aaron Lun (17:56:16): > Call it “batch upload” or something.
Kevin Rue-Albrecht (17:57:21): > Also, last point, maybe obvious: the text box would always open prefilled with the current list of features, so that users can decide whether to replace (Cmd+A, paste) or append their new list
Kevin Rue-Albrecht (17:58:08): > worst case, if they paste a duplicated feature name, we run auniquewhen processing the updated list
Kevin Rue-Albrecht (18:01:48): > Just to be clear: not gonna happen tonight. Also, as much as I’d like to see the feature, we’re getting awfully close to the release.
Aaron Lun (18:06:46): > Pfft. How bad can it be? We don’t have any reverse dependencies.
2019-10-19
Federico Marini (04:10:15): > Ok, normally my ideas from the saturday morning are pretty weird but:
Federico Marini (04:10:26): > https://twitter.com/USCLONI/status/1184903765289132033?s=20 - Attachment (twitter): Attachment > It’s a big day @USCLONI! We just launched a new smartphone app, Schol-AR, that has the power to revolutionize how scientific data is shared. @TylerArdPhD built Schol-AR so researchers can embed data viz in papers, posters & more using AR. More here: https://news.usc.edu/161786/augmented-reality-app-usc-scientific-data-3d-models/ @USC
Federico Marini (04:10:48): > not saying “iSEE plugin”, but it would be quite fun to see in action
Aaron Lun (04:20:51): > Bit too fun for me.
Federico Marini (05:04:06): > True that
Kevin Rue-Albrecht (07:16:14) (in thread): > It’s not “bad” per se. Actually, I have a pretty clear idea of how I’d like to implement this gene list feature. It’s just that I’ve got an institute seminar to give on Thursday and already not enough time to deal with all the last minute edits and additions that were suggested for the slides
2019-10-21
Federico Marini (03:52:08): > This one had to come
Federico Marini (03:52:09): > http://singlecellvr.com/
Kevin Rue-Albrecht (03:58:05): > What the hell am I supposed to see?
Kevin Rue-Albrecht (03:59:27): > I mean I don’t even get the demo video (“Try it yourself” here:http://singlecellvr.com/about.html)
Federico Marini (04:31:08): > not so much, we probably would need the Oculus rift-like thingy
Federico Marini (16:50:49): > Hey iSEE core, I have come up with a proposal of having a round of feedback collection, maybe in the form (ah ah. ) of a form
Federico Marini (16:51:03): > Here’s my first draft, feel free to edit and comment
Federico Marini (16:51:04): > https://docs.google.com/forms/d/1V0jbj9RjyCWncdmesRL5tQZP12p3hLPRl-_h2tj-o8U/edit?usp=sharing
2019-10-24
Aaron Lun (02:13:46): > I HAVE A DREAM.
Aaron Lun (02:15:37): > I was thinking about whether I could use iSEE for some in-house project, and it wasn’t flexible enough. Or to be specific - the existing panels were too general for our required application. But in the next cycle, we have an opportunity to rectify that and create an API for more integrated deployment of custom panels.
Aaron Lun (02:19:06): > The main idea is that every panel type (even the built-in ones) must follow the API, which then makes it easy to define many custom panels, because you get the same functionality as the built-in ones as long as you follow the API’s contract.
Aaron Lun (02:24:59): > For example, for plots where points correspond to columns of the SE, the contract might be: > - Emits a code chunk that describes the generation of a ggplot object. > - Accepts a column selection. > > And that’s it! If you write a function that follows this contract, you can fit right in. (The various aesthetic selections can be added to the ggplot afterwards, or we can add them to the contract, albeit at the cost of making it a little harder to write the code.)
Aaron Lun (02:27:07): > Then I can write my ownvolcanoPlotormaPlotfunctions, etc. that create highly specialized plots but still benefit from the rest of the iSEE framework. And even better, we can add support for custom UI elements and observers.
Aaron Lun (02:27:45): > It’s quite a bit of work, but this is necessary to get enterprise teams to take up iSEE, because they WILL want to customize things, and if we can’t support that, then they’re not going to use it because they have the manpower to just write their own apps.
Charlotte Soneson (04:02:48): > I agree that this would be super useful if we can pull it off - in fact I had a couple of similar requests from (non-enterprise, but still) people over the past weeks.
Vince Carey (09:08:07): > Just in case this idea of an API for iSEE could resonate with issues in data science microservices I am pinging@Sean Davis
Sean Davis (09:08:22): > @Sean Davis has joined the channel
Kevin Rue-Albrecht (10:16:56): > That would really be an awesome feat to formalize the panel ins/outs into an API. > One challenge that comes immediately to mind is the Heat Map panel, which has a different set of collapsible boxes and a “legend plot” in the “Column data parameters” box. > Anyway, happy to see development in that area!
Sean Davis (10:55:55): > Thanks,@Vince Carey, for the nudge. The idea of a more pluggable architecture is attractive, for sure!
Vince Carey (10:57:33): > Would the dynamic interactive heatmaps of the MD Anderson group have a role here, as a concept or target for reuse?https://bioinformatics.mdanderson.org/public-software/ngchm/
Vince Carey (11:00:17): > Cultivating relationships with cellxgene also seems worthy of consideration.
Federico Marini (11:18:40): > Joining the band in saying it would be quite cool to get it done like that, also for the long term plans and easier extendability
Federico Marini (11:18:52): > We could call it diySEE:slightly_smiling_face:
Sean Davis (12:04:38): > It has a name! 90% done!
Federico Marini (12:07:19): > You know me Sean:stuck_out_tongue:
Kevin Rue-Albrecht (12:17:21): > @Aaron Lunwould that satisfyhttps://github.com/csoneson/iSEE/issues/182? - File (PNG): Screenshot 2019-10-21 21.56.28.png - File (PNG): Screenshot 2019-10-21 21.56.36.png - File (PNG): Screenshot 2019-10-21 21.56.52.png
Kevin Rue-Albrecht (12:19:09): > If you don’t have any major visual issue with it, I’ll push to GH like that and you can play with it. It’d be nice to merge tomasterand pushupstreamin the next couple of days to include in the next release
Kevin Rue-Albrecht (12:21:06): > you know what, it’s pushed to GH. have a play and let me know if it’s ok to merge to master
Aaron Lun (13:33:18): > Looks good, though I’d like a table on the right instead of a list.
Aaron Lun (13:54:51): > Does the right update dynamically with any change to the left?
Kevin Rue-Albrecht (14:21:55): > 1. Shouldn’t be a problem > 2. Yes
Aaron Lun (14:28:56): > A pretty table, with alternating shading.
Federico Marini (14:30:12): > kable would do it by default or am I wrong?
Kevin Rue-Albrecht (14:41:10): > I don’t know yet. Right now it’s an htmlOutput that output up to 10
Kevin Rue-Albrecht (14:43:18): > I could turn use
and
html for a simple table but the shading would then require CSS I imagine.kablemay be worth a try
Kevin Rue-Albrecht (15:34:27): > ok.. so status report: > 1.@Federico Marinithe current layout is afluidRowwith 2 columns of width 6 each. If I placed akable/renderTablein the second column, I would have to lose the “25 entries do not exist in rownames(se) and will be ignored. The first 10 are shown below.” text above and below the table
Federico Marini (15:35:15): > can’t we put more tags in the 2nd column?
Aaron Lun (15:35:30): > I would also be happy with a static text field showing all the failures.
Federico Marini (15:35:48): > renderUI + returning a tagList?
Federico Marini (15:35:58): > (if at all needed)
Kevin Rue-Albrecht (15:36:30): > I’m trying an approach with
and
tags, but then we’d have to figure out the striped rows CSS, which kable does automagically
Kevin Rue-Albrecht (15:37:14) (in thread): > the current approach is arenderUIalready
Kevin Rue-Albrecht (15:37:40) (in thread): > Can you be more specific in your description?
Aaron Lun (15:37:53) (in thread): > Like a code chunk in Rmarkdown.
Aaron Lun (15:38:04) (in thread): > Or specifically the output of a code chunk.
Kevin Rue-Albrecht (15:38:19) (in thread): > right, that’s possible I think
Aaron Lun (15:38:21) (in thread): > Even if it’s just “the top 20 failures shown below”, would be sufficient. Not like they’re going to fix them all at once.
Kevin Rue-Albrecht (15:38:40) (in thread): > Yup, I thought the same when I capped at 10
Federico Marini (15:57:39): > Anyway: well done@Kevin Rue-Albrecht, guess a nice bunch of users might enjoy this
Federico Marini (15:58:14): > … which reminds me of my idea thrown in the lions’ den a few days ago: do we want to prompt some feedback with the form?
Kevin Rue-Albrecht (15:59:12): > I meant to ask why a Google Form when we can have a modal in the apps:stuck_out_tongue:
Kevin Rue-Albrecht (16:00:01): > But apart from that, why not. It’s another way for users to contact us, as long as we don’t confuse them with too many ways to contact us
Aaron Lun (16:00:20): > Depends on what we’re going to do with the feedback. I mean, I don’t have any spare capacity to work on it.
Kevin Rue-Albrecht (16:00:35): > I honestly don’t know how much feedback to expect, and … what Aaron just said
Federico Marini (16:00:35): > ’t was mostly because of the anonymity guaranteed by the form
Federico Marini (16:01:07): > Yes, that is a damn truth- more to check out what people want most, provided we have that spare time
Kevin Rue-Albrecht (16:23:17): > ? - File (PNG): image.png
Aaron Lun (16:23:54): > That’s fine to me.
Federico Marini (16:25:34): > Neat!
Kevin Rue-Albrecht (16:25:38): > The catch here is that the implementation is fairly redundant. There are 3 UI components in a vertical layout in the second column: > 1. the first bit of text > 2. the table > 3. the last bit of text > All 3 of which independently recompute the number of invalid entries, to decide if and what they show
Aaron Lun (16:26:47): > HMmm..
Kevin Rue-Albrecht (16:28:21): > I just pushed. Check it out
Kevin Rue-Albrecht (16:29:28): > That said, it’s not like it’s an intensive part of the app either, we could afford the occasionally triple computation, it’s just checking which items in the list are not inrownames(se), it’s not like it’s computing a UMAP
Aaron Lun (16:40:12): > I think this is the best that we’re going to get.
Kevin Rue-Albrecht (16:40:52): > I’ll take that:slightly_smiling_face:
Aaron Lun (16:41:07): > I’m not sure we need 15 more, though.
Aaron Lun (16:41:14): > I think people could figure out what 25 - 10 is.
Kevin Rue-Albrecht (16:41:51): > agreed, did cross my mind, and 10 is a good number
Kevin Rue-Albrecht (16:44:03): > I’ll push that last one, then I’ll leave at least one of you to play with it and confirm the merge, so that I can push upstream tomorrow
Kevin Rue-Albrecht (16:46:01): > it’s pushed
Kevin Rue-Albrecht (16:47:27): > The only thing I can still think of is aesthetic: whether to show the conditional “The first 10 are shown below.” on a newline (it only shows if there are more than 10 invalid entries)
2019-10-25
Kevin Rue-Albrecht (12:07:23): > I’ve fixed the last roxygen import issue. When it passes Travis, I’ll merge and push upstream. At least users will have a way to input lists of features. I think we can optimise and polish in the next devel
Aaron Lun (12:07:56): > I fell asleep so I forgot about this, but go ahead.
Federico Marini (12:08:51): > :thumbsup:
Federico Marini (12:09:08): > I really like the layout as it is now
Kevin Rue-Albrecht (12:09:55) (in thread): > just to be clear, I’ve removed the “000 more …” message under the table, as suggested by Aaron
Federico Marini (12:13:14) (in thread): > clean UIs ftw:slightly_smiling_face:
Kevin Rue-Albrecht (12:22:40): > Ugh. Can’t believe it. I forgot to version bump
Kevin Rue-Albrecht (12:22:48): > PR of the year
Federico Marini (12:22:56): > its friday
Federico Marini (12:23:12): > I’d request a thorough review of all the others:dealwithit-parrot:
Kevin Rue-Albrecht (12:38:37): > :party_parrot:1.5.12 is now upstream:party_parrot:
Kevin Rue-Albrecht (12:39:40) (in thread): > Andhttps://github.com/csoneson/iSEE/issues/182is now closed too, which is an even bigger deal
Aaron Lun (12:57:58): > can you comment on the related issue and describe the solution?
Kevin Rue-Albrecht (13:41:08): > Sure. > Speaking of which, I’ll also open another issue to point out that the order of feature names in the selectize correspond to bottom-to-top in the heatmap (as row indices increase along the y-axis). It would be more intuitive that features are top-to-bottom instead.
Aaron Lun (13:41:38): > ¯*(ツ)*/¯
Aaron Lun (13:41:40): > okay
Kevin Rue-Albrecht (13:43:20): > It’s not mission-critical, so it can wait the next release, as I may not able to get to it this weekend. I’ve had a look and it’s not just a matter ofrev()unfortunately.
Aaron Lun (13:44:19): > The heatmap code is a pain in the ass. Is this really the simplest way to create a ggplot heatmap? That’s ridiculous.
Kevin Rue-Albrecht (13:46:55): > Honestly, most of the credit for the heat map work goes to@Charlotte Soneson. It was by far the most painful panel to set up, and despite all the work, we all agree that result is not ideal or practical. This would be a feature to revisit is any of us (or the community) can spare the time (which is a big ask for anyone).
Kevin Rue-Albrecht (13:55:22) (in thread): > Not sure you get an email notification for it, but see my messagehttps://github.com/csoneson/iSEE/issues/182
Kevin Rue-Albrecht (14:34:17): > Nevermind, problem solved:https://github.com/csoneson/iSEE/pull/323
Kevin Rue-Albrecht (14:37:18): > itwasrev…. with a twist.
2019-10-26
Kevin Rue-Albrecht (17:09:48): > bingo, 1.5.13 passed the bioc build with all lights green:http://bioconductor.org/checkResults/3.10/bioc-LATEST/iSEE/
2019-10-27
Kevin Rue-Albrecht (14:07:33) (in thread): > I’ve been thinking a bit more about this vision, and as exciting as the idea of it is, I can’t come up with something yet. > The contract above doesn’t deal with the fact that each plot is linked to a range of input widgets. I believe that declaring the widgets (type, value, choices, range) would also have to be part of the API. Otherwise how would users defined new custom panels, if they don’t also define the UI components of that panel type? > … but then the “memory” (i.e. our storage of widget values) would also have to be defined, among which the “constants” that we use as column names for the DataFrams of parameters. > Ugh… my head hurts
Aaron Lun (14:26:35) (in thread): > One of the arguments would be to pass the UI elements themselves.
Kevin Rue-Albrecht (14:29:20) (in thread): > …already laid out in their collapsible boxes as well, I imagine
Aaron Lun (14:41:19) (in thread): > I think we could handle putting things into the boxes, they just have to provide “categories” of parameters.
Kevin Rue-Albrecht (14:44:07) (in thread): > I gotta say, despite the amount of work that I can foresee to get it all into place, I like the way you think.
Kevin Rue-Albrecht (14:47:36) (in thread): > On a separate note, while I don’t have an issue with us holding the record of messages in a channel, we’ll soon need to find a place better suited to keep track of those design choices more efficiently. Perhaps the wiki of the GH repo could be used again for that.
Kevin Rue-Albrecht (14:49:40) (in thread): > Another way to track progress and milestones could be the “Projects” tab, right next to “Wiki”. The kanban board could help manage the process considering that it might not all happen overnight. Also throwing ideas in and out, like we just did
Aaron Lun (18:41:08) (in thread): > ¯*(ツ)*/¯
Aaron Lun (18:41:11) (in thread): > go ahead
Aaron Lun (18:41:24) (in thread): > crunching through and deleting some more code before tomorrow.
2019-10-28
Peter Hickey (23:07:25): > @Peter Hickey has joined the channel
2019-10-30
Federico Marini (17:20:15): > Heya<!channel>, shall we put together a compact post for the news in iSEE’s release for 3.10?
Federico Marini (17:20:32): > maybe even with a mini screencast?
Federico Marini (17:21:08): > we’d have > - sce upload > - gene lists clever handling > Am i forgetting something major?
Aaron Lun (17:24:14): > You’re more than welcome to do that.
Kevin Rue-Albrecht (17:25:05): > According to the NEWS file, we have `modeEmpty(). I forgot about that one (empty UI, where usersaddpanels instead or removing panels from the default UI: launches much faster)
Kevin Rue-Albrecht (17:25:11): > https://github.com/csoneson/iSEE/blob/master/inst/NEWS.Rd
Federico Marini (17:26:19): > Ok, that can be a teaser in case we want to work lil more on that - maybe even crowdsourcing that at eurobioc
Kevin Rue-Albrecht (17:26:59): > The rest of the NEWS look more like fixes/refactoring.
Kevin Rue-Albrecht (17:27:24): > I particularly like version 1.5.2 “
Minor doc fix..”
Charlotte Soneson (17:29:27): > Re: this: Federico and I were thinking about proposing a kind of mini-hackathon/whatever you want to call it for EuroBioc developer day, where interested participants could gather to build new iSEE modes. In that way we might capture panel combinations that would be useful from their point of view, and get some assistance with the implementation.https://community-bioc.slack.com/archives/C8BJLSP8T/p1572470779150200 - Attachment: Attachment > Ok, that can be a teaser in case we want to work lil more on that - maybe even crowdsourcing that at eurobioc
Federico Marini (17:49:05): > > You’re more than welcome to do that. > director’S mode on:wink:
Kevin Rue-Albrecht (17:51:00): > As long as we’re on the subject, I vaguely remember having a discussion about “modeSomething” vs “mode_something”. Can anyone remember if we’re happy with the current former naming?
Aaron Lun (17:51:24): > I thought the former was better.
Kevin Rue-Albrecht (17:55:15): > Ok. So that sorts out naming guidelines for a eurobioc hackhathon
Federico Marini (18:32:51): - File (MPEG 4 Video): iSEE_upload_cut.mp4
Federico Marini (18:49:32): - File (MPEG 4 Video): iSEE_genelist_cut.mp4
Aaron Lun (19:02:22): > Where are the sound effects
Aaron Lun (19:04:56): > Gimme somehttps://www.youtube.com/watch?v=BZgAJ8iaAzc - Attachment (YouTube): How Do I Live- Trisha Yearwood
Federico Marini (19:06:38): > Sorry Master:stuck_out_tongue:
Federico Marini (19:06:57): > got a 3min youtube video course on iVideo as background info
Aaron Lun (19:07:57): > AND I
Aaron Lun (19:08:01): > WILL ALWAYS
Aaron Lun (19:08:03): > LOVE YOU
Aaron Lun (19:08:13): > I think that would really help with the message.
Federico Marini (19:09:28): > YEP
Federico Marini (19:10:06): > I was more for putting “don’T call me up” to highlight how iSEE reduces the iterations from wetlab scientists to bioinformaticians
Federico Marini (19:11:03): > I’ve got a long list of stupid musical references
Aaron Lun (19:11:23): > baby cause you know that you’re everything
Aaron Lun (19:11:24): > good in my life
Aaron Lun (19:11:29): > AND TELL ME NOW
Aaron Lun (19:11:31): > How do I live
Federico Marini (19:11:32): > Wait for GeneTonic to come out
Aaron Lun (19:11:33): > without you
Aaron Lun (19:11:43): > How do I breath without you
Aaron Lun (19:11:45): > if you ever go
Aaron Lun (19:11:48): > How do I ever
Aaron Lun (19:11:51): > ever survive…?
Federico Marini (19:12:15): > https://www.youtube.com/watch?v=HPs7dJvW2X4 - Attachment (YouTube): Valentina Hasan-Ken Lee (Without you, Mariah Carey)
Federico Marini (19:13:14): - File (PNG): image.png
Federico Marini (19:13:20): > Found an outfit for EuroBioc2019
Aaron Lun (19:14:58): > looks like you too.
Federico Marini (19:18:24): > not picking randomly people here, A-boy:smile:
Aaron Lun (19:19:06): > just gimme some whitney
Federico Marini (19:20:46): > I’m more for britney
Aaron Lun (20:46:32): > Just tried opening a jupyter notebook for the first time. Can you believe that I need a special app to open it? Crazy!
2019-10-31
Federico Marini (04:33:17): > You make it sound like a meme sentence:slightly_smiling_face:
Kevin Rue-Albrecht (09:36:20): > Aaaaargh I forgot to point out that we have a Halloween feature in this release. That didn’t make it to Twitter did it?
Federico Marini (09:36:55): > not really, but feel free to jump on the bandwagon:slightly_smiling_face:
Kevin Rue-Albrecht (09:49:29): > On a lower note, the new devel has an issue :http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/malbec2-checksrc.html > > attempt to replicate an object of type 'S4' >
Federico Marini (11:28:53): > Even lower note, but still
Federico Marini (11:29:09): > for the fun of it, I tried to put some instances of iSEE on classical datasets
Federico Marini (11:29:10): > https://github.com/federicomarini/iSEE_instances/tree/master/iSEE_classics
Federico Marini (11:29:19): > namely,mtcarsanddiamonds
Federico Marini (11:29:44): > This should serve more to convince that ideally any rectangular data could be fed into it
Federico Marini (11:31:07): > fun fact:
Federico Marini (11:31:16): - File (PNG): image.png
Federico Marini (11:31:24): > using only x,y,z, for building pca and tSNE
Aaron Lun (11:34:37): > I don’t know what that is.
Aaron Lun (11:34:49): > Also@Kevin Rue-Albrechtdoes this reproduce locally on R-devel?
Federico Marini (11:42:46): > just the tSNE of the diamonds price
Federico Marini (11:43:06): > > Also@Kevin Rue-Albrechtdoes this reproduce locally on R-devel? > I did not make the switch yet
Kevin Rue-Albrecht (11:43:41): > Haven’t tried yet. Haven’t installed the new R and install all the new devel versions
Aaron Lun (11:45:10): > be aware that RGBL is broken, which breaks the EHub and AHub packages.
Kevin Rue-Albrecht (13:54:50): > Well I ran into an error running ./configure on the new R devel tarball but I didn’t look into it further because i had another project to debug.
Charlotte Soneson (14:19:59): > I get this locally with R 4.0/bioc-devel: > > test_voice.R:121: error: .showPanel adds a valid row to the table of active panels > attempt to replicate an object of type 'S4' > 1: .showPanel("redDimPlot", 6, active_panels, width = 4L, height = 400L) at /Users/charlotte/Dropbox/Rpackages/iSEE/SEE/tests/testthat/test_voice.R:121 > 2: rbind(active_panels, DataFrame(Type = mode, ID = id, Width = width, Height = height)) at /Users/charlotte/Dropbox/Rpackages/iSEE/SEE/R/dynamicUI.R:60 > 3: standardGeneric("rbind") > 4: eval(mc, env) > 5: eval(mc, env) > 6: eval(mc, env) > 7: rbind(...) > 8: rbind(deparse.level, ...) >
Charlotte Soneson (14:48:07): > Seems to be an issue withrbinding adata.frameand aDataFrame
Charlotte Soneson (14:52:02): > > > rbind(data.frame(x = 1, y = 2), DataFrame(x = 3, y = 4)) > Error in rep(xi, length.out = nvar) : > attempt to replicate an object of type 'S4' >
Vince Carey (15:44:46): > @Aaron Lunwhat aspect of RBGL is broken. available build reports look ok for 3.10 and 3.11
Aaron Lun (15:45:07): > I was trying to compile it last night, it was complaining that it couldn’t find the boost headers.
Aaron Lun (15:45:54): > Looks like the change must have propagated this morning.
Vince Carey (15:51:11): > yes
2019-11-01
Kevin Rue-Albrecht (05:50:56): > I’ve just fetched and merged the Bioc “release bumps” commits tokra-bump. However, that’s all I can do before logging off for my honeymoon (finally happening!), if anyone can pick this branch up, possibly fix the S4 issue reported above and whatever else comes up, and eventually merge it back tomaster
2019-11-06
Charlotte Soneson (11:48:20): > I fixed the immediate error causing the fail on the Bioc build system, and replaced a couple of deprecatedscran/scaterfunctions in the test code. Pushed tokra-bump.
Aaron Lun (11:49:47): > :+1:
2019-11-07
Kevin Blighe (11:25:32): > @Kevin Blighe has joined the channel
2019-11-12
Charlotte Soneson (02:22:27): > As discussed above, Federico and I were planning a session during the EuroBioc developer day where we (and any other interested participants) would expand the collection of modes available in iSEE. We put together a first description here (to put in the GitHub issue announcing the session):https://docs.google.com/document/d/1aVkoFka-0GAXCsUyhqPYE6yharb4D4eesaOz-KvkJ2o/edit?usp=sharing
2019-11-16
Federico Marini (07:36:39): > FYI: I filed an issue in detail on the bug/odd behavior for hinton plots
Federico Marini (07:36:55): > My 2 cents: It is on Shiny’s
Federico Marini (07:37:18): > whether they can revert it or we need to adapt it it is not clear yet:stuck_out_tongue:
2019-11-17
Kevin Rue-Albrecht (09:04:57): > I’m baaaack:slightly_smiling_face:So, fom what I see,kra-bumpwas merged without a version bump (nor an update to NEWS). Not sure why, but I can make another PR for that.
Charlotte Soneson (09:06:47): > Welcome back! Yes, I mergedkra-bump- I figured that the version bump could happen just before pushing upstream…
Kevin Rue-Albrecht (09:07:53): > Makes sense to keep GH and upstream as synchronised as possible
2019-11-21
Kevin Rue-Albrecht (06:37:26): > Finally updating my installation of R-devel to look intohttps://travis-ci.org/kevinrue/iSEEWorkshop2019 > > 'calculateQCMetrics' is defunct. > Use 'perCellQCMetrics' instead. >
Federico Marini (06:39:13): > the closest thing to the old one isaddPerCellQCto keep the sce
Federico Marini (06:39:34): > Some stats are not generated anymore, if I am not mistaken
Federico Marini (06:39:50): > Guess the best informed opinion can come from Aaron:wink:
Kevin Rue-Albrecht (06:42:19): > I also made a release of the workshop material as it was for the workshop, even though it fails now because of the scater update. > But I assume that the#biocworkshopsrequires an updated version of the workshop that runs on the latest devel. I need to check on the channel what exactly the requirements are
Sean Davis (06:59:45): > We aren’t making updates to the workshop materials from previous conferences, so you are free to do what works for you with respect to making changes. That has been the practice in previous years and will likely continue. We can discuss creating a separate flow for workshops that are updated regularly.
Kevin Rue-Albrecht (07:01:51): > Cool. I wasn’t sure if the book building process might still pull from github
Aaron Lun (11:28:50): > Yes.
2019-11-23
Aaron Lun (14:36:58): > refactoris minimally functional with new UI creation internals, seedynamicUI.Rfor how the new world will operate.
Kevin Rue-Albrecht (14:54:02): > Sometimes I feel like the code base has grown out of control…:rolling_on_the_floor_laughing:That said, this refactoring might actually simplify or at least “DRY” things to some extent
Aaron Lun (17:44:26): > I think we’re already pretty DRY. The main thing here is to split up the observer set-up responsibilities to separate functions.
Aaron Lun (20:57:00): > OMG this is so awesome. Wait a few hours and you’ll see it.
Aaron Lun (22:48:34): > It’s… beautiful.
Aaron Lun (22:48:58): > I’m cooked for the day, but I’ll make a PR later describing just how awesome this will be.
2019-11-24
Aaron Lun (13:50:58): > While I’m doing the architectural changes, I’d like someone to try to pull out the “bare minimum” of what we might require from someone creating their own plot. Don’t worry about observers; it would instead be “give me something like.make_redDimPlot”. > > The question is what does this “something” be. Currently.make_redDimPlotdoes a lot of work (downsampling, defining selections, defining visual parameters) so I don’t want to have to burden custom panel developers with that work. I want to do all of that for them, setting up environment variables containing, e.g., lists of integer vectors from selected points, and then just tell developers to give me a function that generates a code chunk that assumes that such variables are available. > > I would like someone to figure out what this function would look like and to restructureplotting.Rso that it is easy to “plug and play” by passing in such a function.
Kevin Rue-Albrecht (14:03:15): > Have you checked my updates to the GDoc? I tackled listing out this kind of thing, i.e., the arguments of a constructor function for a “new panel” class. (Could be a simple list to start with, though a class would allow built-in validity checks)
Aaron Lun (14:06:34): > Yes, the idea is similar. You’ll notice how clean the new world code is.
Aaron Lun (14:07:02): > Anyway, that bit requires some delicacy that I’ll deal with in the next week + over thanksgiving. The plotting thing I mentioned above is orthogonal so someone can work on that in the meatntime.
Kevin Rue-Albrecht (14:09:29): > I was about to open the code again. I gave a lot of thought to the whole thing randomly through the days recently, but haven’t been able to save time and sit down to work out a detailed plan. Glad you kicked that off. I might only be able to put in some serious coding at the Xmas fireplace though
Aaron Lun (14:12:06): > Anyway,@Federico Mariniand@Charlotte Sonesonmight want to pay attention to this, because this will affect the future of custom panels. Not in time for Euro Bioc, unfortunately, but it will make it much easier to perform native integration with the iSEE architecture going forward. > > Incidentally, all of those ideas for new modes should be collected into aniSEEupackage (iSEE universe). I don’t want to clutter up the main app with extra modes and panel types but it would make sense to have them somewhere for people to easily deploy.
Kevin Rue-Albrecht (14:27:03): > On a similar note, if the update is as considerable as I anticipate, I was also thinking that we might need to bump toiSEE2(more specifically,iSEE-2.0.0)This basically follows the Bioc guidelines, when the API changes. Which could then lead toiSEEu2the fun never ends!
Aaron Lun (14:30:18): > Well, iSEEu would not need to be 2’d, it doesn’t exist yet.
Kevin Rue-Albrecht (14:31:24): > oh… you ruined it now:cry:
Federico Marini (14:39:28): > > Anyway,@Federico Mariniand@Charlotte Sonesonmight want to pay attention to this, because this will affect the future of custom panels. Not in time for Euro Bioc, unfortunately, but it will make it much easier to perform native integration with the iSEE architecture going forward. > We can tell the audience to stay tuned
Federico Marini (14:39:42): > I learned to trust your visions by now
Federico Marini (14:39:43): > :slightly_smiling_face:
Kevin Rue-Albrecht (15:36:25): > @Aaron LunNot sure if you meant that but the PR indicates that there are a number of seemingly internal functions that are exported in the NAMESPACE, e.g..createParamObservers
Aaron Lun (16:06:16): > That’s correct, for use by developers; the dot just indicates that it is not for end-users.
Aaron Lun (20:07:42): > Tables are done.
Aaron Lun (22:56:44): > heatmaps are done. PHeew. That’s a messed up panel type.
Aaron Lun (22:56:49): > More code than the rest of the panel types combined.
2019-11-25
Federico Marini (15:51:33): > Picking up on iSEEu: shall we try the markdown roxygen for this project?
Federico Marini (15:51:57): > It is quite easy to setup and I became quite acquainted in the latest packages
Federico Marini (15:52:31): > main pros: works super for\code{}
Federico Marini (15:52:48): > link+code ->[func()]
Federico Marini (15:53:21): > I can branch it out and you can have a look:wink:
Charlotte Soneson (15:53:29): > Sounds good to me
Federico Marini (15:54:45): > plus, it can help not using up too many chars per line:slightly_smiling_face:
Federico Marini (15:55:07): > (and is nicely highlighted in RStudio, for the ones who use it)
Federico Marini (16:14:08): > Travis setup from the start?
Federico Marini (16:14:24): > (and pkgdown deployment, FWIW?)
Charlotte Soneson (16:16:15): > I added a travis file
Federico Marini (16:17:05): > :thumbsup:
Federico Marini (16:25:12): > CoC?
Federico Marini (16:25:37): > Especially since we are prompting contribs from externals, this one would make more sense here
Charlotte Soneson (16:25:46): > :+1:
Kevin Rue-Albrecht (17:44:21): > :+1::+1::+1::+1:
2019-11-26
Kevin Rue-Albrecht (03:22:45): > From just going though the existing DotPlot panel types, functions like.make_redDimPlotessentially only define the following before calling.plot_wrapperto handle the generic aspects (colors, selections): > * Command to create the data.frame column for X-axis > * Command to create the data.frame column for Y-axis > * Plot title. > * X-axis title. > * Y-axis title. > And that’s it.
Kevin Rue-Albrecht (03:26:31): > (In reference to@Aaron Lun’shttps://community-bioc.slack.com/archives/C8BJLSP8T/p1574621458207200above) - Attachment: Attachment > While I’m doing the architectural changes, I’d like someone to try to pull out the “bare minimum” of what we might require from someone creating their own plot. Don’t worry about observers; it would instead be “give me something like .make_redDimPlot”. > > The question is what does this “something” be. Currently .make_redDimPlot does a lot of work (downsampling, defining selections, defining visual parameters) so I don’t want to have to burden custom panel developers with that work. I want to do all of that for them, setting up environment variables containing, e.g., lists of integer vectors from selected points, and then just tell developers to give me a function that generates a code chunk that assumes that such variables are available. > > I would like someone to figure out what this function would look like and to restructure plotting.R so that it is easy to “plug and play” by passing in such a function.
Kevin Rue-Albrecht (05:04:27): > I refactored the plotting code. Just gotta update the docs now. Will do later today though. - File (PNG): image.png
Kevin Rue-Albrecht (08:50:20): > Eurobioc2019 hackathon:iSEEspheres:slightly_smiling_face: - Attachment: Attachment > I thought this looked interesting: https://www.biorxiv.org/content/10.1101/853457v1
Aaron Lun (12:35:16) (in thread): > Don’t think about the exact code in.plot_wrapperper se. Think more along the lines of “what does a extension need to specify, and what do we provide?”
Kevin Rue-Albrecht (12:36:40) (in thread): > I’m just walking home now, but once there I’ll push what I have and you’ll see that I’ve extracted exactly that
Aaron Lun (12:37:14) (in thread): > In general, we should provide: > * the SE object, asse. > * the parameter list, asparam. > * a list of selections (active and remembered, realized into logical/integer vectors). > And the user-specified function is expected to generate a plot based on these variables in memory.
Kevin Rue-Albrecht (12:39:47) (in thread): > Ok then the first two are taken care of. I think another bit of refactoring is needed to facilitate the third point
Kevin Rue-Albrecht (12:42:21) (in thread): > Anyway, I’m just warming up after a bit of time away from the code base, i got a few visions while reading the new code
Kevin Rue-Albrecht (14:25:38) (in thread): > There’s a branch calledrefactor-krawhere I’ve put my refactoring so far.
Kevin Rue-Albrecht (14:28:51) (in thread): > Basically, the API is for developers to provide a function that takesparamsandse, and returnlist(data_cmds, plot_title, x_lab, y_lab). > The existing (untouched).plot_wrappertakes care of the rest.
Kevin Rue-Albrecht (14:32:08) (in thread): > That said, I want to simplify that further, for both panel developers and ourselves: I think panel developers should provide at least 2 separate functions to return separately some of the elements above, i.e. the API of a new panel would be to provide 2 functions: > * .getDataCommandsXY(params, se) -> character(n) > * .getPlotTitles(params, se) -> list(title="something", x=NULL, y="something")
Kevin Rue-Albrecht (15:33:54): > damn.. I just ran into code wheresomeof the data commands and plot titles interact, I guess it was good with a single function
Kevin Rue-Albrecht (16:52:39): > I’ll add that to the GDoc, but FYI developers will also have to define their own*PlotDefaults()-style function.
Aaron Lun (16:55:26): > am back
Aaron Lun (16:58:38): > but first I need to toilet
Aaron Lun (17:03:06): > okay, am back
Kevin Rue-Albrecht (17:07:17): > Let me know when you’ve had a chance of inspecting my side-side-branch, so that we’re on the same page
Aaron Lun (17:12:15): > alrigh thit me
Kevin Rue-Albrecht (17:16:11): > in the current setup, the API seems to be a panel specific function that takes: > * se > * params > and return > * data_cmds > * plot_title > * y_lab > * x_lab
Aaron Lun (17:16:13): > hold on, got to go again.
Kevin Rue-Albrecht (17:22:09): > You mentioned another input: > > a list of selections (active and remembered, realized into logical/integer vectors). > But I didn’t have to worry about that yet, because it’s taken care of by.plot_wrapperinternals
Aaron Lun (17:59:44): > alright, I’m back.
Kevin Rue-Albrecht (18:00:36): > busy man
Aaron Lun (18:01:18): > call me
Kevin Rue-Albrecht (18:02:03): > Well, you pick. Or what I said above, or a code review, or we can draft stuff on GDoc. > Or right. Calling it is, gimme a sec
Kevin Rue-Albrecht (18:04:16): > Started a call.
2019-11-27
Kevin Rue-Albrecht (13:39:56): > Looking forward to getting this refactoring so that we can look at a#spatial* dimension plot*panel :)
Federico Marini (13:52:51): > @Aaron Lunand@Kevin Rue-Albrecht: we are putting together some “how to contribute to iSEEu” vignette to use as guideline for the participants to dev day
Federico Marini (13:53:00): > if you feel like adding something, please do so:wink:
Federico Marini (13:53:21): > the idea is to keep things simple, more like “the basics”
Kevin Rue-Albrecht (15:42:28): > any particular reason why “> newline” ? > > title: > > iSEEu_contribute_eurobioc2019 >
Kevin Rue-Albrecht (15:50:37): > I’ve pushed a couple of tiny fixes to iSEEu vignette. I can’t think of anything specific to add yet
Federico Marini (17:13:26): > > any particular reason why “> newline” ? > Simply overtook previous vignette’s yaml
Federico Marini (17:13:46): > lazy me
Kevin Rue-Albrecht (17:14:47): > I was just curious about the syntax
2019-11-28
Aaron Lun (12:44:28): > Going to mandate that all row/column/reddim names must be UNIQUE.
Aaron Lun (12:44:35): > This will make life a lot easier.
Aaron Lun (14:32:19): > jesus, each panel is taking an HOUR to convert.
Aaron Lun (17:42:27): > God there’s so many goddamn parameters
Kevin Rue-Albrecht (18:12:51): > Good so it wasn’t just me
Aaron Lun (18:18:05): > in any case, you can track the current progrsess at refactor-aa
Aaron Lun (18:24:53): > Just need to get through the tables before dinner.
Aaron Lun (20:05:33): > Hello?
Aaron Lun (20:05:37): > So quiet.
Aaron Lun (20:05:45): > Even subway closed at 1pm.
Aaron Lun (20:47:34): > I was eating a subway by myself in the park for lunch. And pigeons were trying to steal my food. And some homeless guy took a dump in front of me.
Aaron Lun (20:47:42): > This thanksgiving thing sucks.
Aaron Lun (22:56:19): > off to watch some anime for a bit.
Aaron Lun (22:59:30): > :konata:
Aaron Lun (23:07:39): > :chuuni:
Aaron Lun (23:26:09): > Bum, couldn’t find a good goku gif.
2019-11-29
Aaron Lun (04:06:32): > That’s it, I’m calling it a day for today.
Aaron Lun (04:06:35): > :chuuni:
Kevin Rue-Albrecht (06:05:13): > Thanks Aaron.
Kevin Rue-Albrecht (06:10:16): > On a separate note.#spatialgot me thinking about an iSEE-friendly class for spatial data. > I’ve put togetherhttps://github.com/kevinrue/SpatialCellExperiment, heavily borrowing fromSingleCellExperimentcode to add aspatialDimslot that behaves similarly to a singlereduceDimelement (cells can have multiple reduced dimension results, but a single set of real coordinates). > Happy to hear feedback from you guys, from an iSEE perspective, before making noise on#spatial
Federico Marini (06:12:20): > Maybe you also want to ping@Davide Rissoas he fathered with Aaron the good SCE?
Aaron Lun (13:31:20): > I don’t think there’s much difficulty in storing spatial coordinates with the existing S(C)E classes, it’s just some more column metadata.
Aaron Lun (13:31:51): > That also includes morphological features that are determined via image analysis.
Aaron Lun (13:32:19): > The real questions are (i) how do we deal with subcellular localization and (ii) do we want to carry around the raw images in an R session?
Aaron Lun (13:32:27): > :chuuni:
Aaron Lun (13:51:53): > Anyway, take it up in#spatial
Aaron Lun (14:09:55): > We’re going to need another book to describe how to extend iSEE.
Kevin Rue-Albrecht (14:15:02): > We’ll need a year long sabbatical too, to write it all
Kevin Rue-Albrecht (14:15:31) (in thread): > You’ll appreciate how I reused the int_colData for that
Kevin Rue-Albrecht (14:23:09) (in thread): > (ii) nope, at least not in the SE-type object anymore. At that point, I’d be happy with polygons, like the country maps:https://cran.r-project.org/web/packages/sf/vignettes/sf1.html#geometrical_operations
Kevin Rue-Albrecht (14:24:02) (in thread): > (i) probably a similar approach, cellular and subcellular localizations modelled like regions of a country
Kevin Rue-Albrecht (15:47:31): > Once we’re done with the whole S4 refactoring, how about we afford a few extra letters and rename > > RedDimPlot(), > FeatAssayPlot(), > ColDataPlot(), > RowDataPlot(), > SampAssayPlot() > > to > > ReducedDimPlot(), > FeatureAssayPlot(), > ColumnDataPlot(), > RowDataPlot(), > SampleAssayPlot() > > ?
Kevin Rue-Albrecht (17:28:48): > @Aaron Lunare you working on it now? I just had a look at the Travis build and I spotted a typo/bug fix that I can solve. > > .allowable_choice_error: no visible binding for global variable > > 'allowed' > > I can have a look at the others too for a bit
Kevin Rue-Albrecht (17:48:22): > well don’t surprise me now, i’m almost done
Aaron Lun (17:57:51): > is it done?
Aaron Lun (17:58:08): > otherwise I’m going to keep up the fight
Aaron Lun (17:58:10): > :chuuni:
Kevin Rue-Albrecht (18:00:23): > 2 more
Kevin Rue-Albrecht (18:00:47): > the heatmap modal threw me a curve ball because it needs to accessinputfor.createRenderedOutput
Kevin Rue-Albrecht (18:10:44): > @Aaron Lun: pushing the code your way
Kevin Rue-Albrecht (18:11:44): > you can pull now
Kevin Rue-Albrecht (18:13:06): > I’ve left the modal thing as aTODO:https://github.com/csoneson/iSEE/blob/230b0ef9cbb0d3a255d5ee65fbf769f24ff94635/R/panel_HeatMapPlot.R#L514
Kevin Rue-Albrecht (18:13:41): > You left…in the generic, but I don’t want to mess with your secret plan.
Kevin Rue-Albrecht (18:13:53): > you seem to use it forcolormapfor other panels
Aaron Lun (18:15:06): > What the hell is that thing?
Kevin Rue-Albrecht (18:15:35): > you’ll have to be more specific
Aaron Lun (18:15:58): > Why are we pulling.heatMapFeaturesTextInputfrom input?
Kevin Rue-Albrecht (18:16:53): > because the modal text field is initialized with the list of features currently present in the heatmap feature selectize
Aaron Lun (18:17:08): > That’s going to be a rpoblm.e
Kevin Rue-Albrecht (18:17:30): > actually, maybe not
Kevin Rue-Albrecht (18:17:40): > let me have a quick look
Kevin Rue-Albrecht (18:18:03): > because it’s nothing more than an input updating another input, nothing we haven’t done before
Aaron Lun (18:19:37): > createRenderedOutputshould never see anyinputdirectly, so that thing will have to move somewhere else.
Kevin Rue-Albrecht (18:19:48): > hang on
Kevin Rue-Albrecht (18:20:52): > i can get the same information frompObjects
Aaron Lun (18:21:37): > Well, I don’t think you can, because those text elements are invalid geature names that never enter into memory.
Kevin Rue-Albrecht (18:22:24): > No no no, I’m talking about the ones thatarein memory. The current set ofvalidfeatures in the selectize
Kevin Rue-Albrecht (18:22:41): > that’s what I use toinitializethe modal text field
Kevin Rue-Albrecht (18:23:04): > then the user can mess with invalid names, it doesn’t matter because only valid ones will make it to pObjects
Aaron Lun (18:23:18): > I’m talking about L516 here.
Kevin Rue-Albrecht (18:24:59): > oh crap
Kevin Rue-Albrecht (18:28:48): > Oh, and L518feature_choicesis not defined. I deleted the fix at the same time as I reverted my failedinputfix. It’s just about usingrownames(se)
Kevin Rue-Albrecht (18:29:51): > oh no wait - silly me, I put the code at L535
Kevin Rue-Albrecht (18:30:01): > it has to be moved one block up
Kevin Rue-Albrecht (18:30:17): > ugh - I shouldn’t do stuff on a Friday night -_-
Aaron Lun (18:30:20): > Putting that aside; there is no location where input and output are available in the same generic.
Kevin Rue-Albrecht (18:30:33): > welcome to heatmap-land
Aaron Lun (18:30:34): > Mostly for security to enforce passage of parameters through pObjects.
Kevin Rue-Albrecht (18:31:52): > Well that heatmap modal is a kind of mini-UI, it has its independent small set of input/output
Aaron Lun (18:32:05): > That is not good.
Aaron Lun (18:32:52): > We should put it in a separate parameter box instead.
Kevin Rue-Albrecht (18:33:49): > mmh.. it’s going to get cramped in a box
Aaron Lun (18:34:41): > Well, whatever you choose, the UI and output needs to be divorced.
Aaron Lun (18:36:39): > But that’s not really the real problem here.
Aaron Lun (18:37:09): > The real problem is that the errors need to be brought up sooner.
Aaron Lun (18:37:29): > “sooner” as in “closer in the Shiny dependencygraph” rather than literally sooner.
Aaron Lun (18:38:29): > The observer responding to the text field should be responsible for triggering the warnings about invalid names.
Aaron Lun (18:39:08): > Something liek this:https://deanattali.com/blog/shinyalert-package/ - Attachment (deanattali.com): shinyalert: Easily create pretty popup messages (modals) in Shiny > A brand new shiny package has entered the world yesterday: shinyalert. It does only one thing, but does it well: show a message to the user in a modal (aka popup, dialog, or alert box). Actually, it can do one more thing: shinyalert can also be used to retrieve an…
Aaron Lun (18:40:29): > hell, even a normal shiny modal inside an observer would sufice.
Kevin Rue-Albrecht (18:40:35): > What if - instead of making it a heatmap-specific rendered output - I could really make it a standalone miniUI, say a “Edit features” modal. Any panel that wants to use that modal would just have to: > * include a button to launch the modal > * pass the current list of features > * receive a new list of features
Kevin Rue-Albrecht (18:41:21): > Anyway. As you said, that’s a separate point for now.
Aaron Lun (18:41:46): > Well, you can do what you want with modals as long as there are no UI elements inside the modal.
Kevin Rue-Albrecht (18:42:44): > uhh… that was the whole point of this modal
Aaron Lun (18:42:46): > It can literally be “here are all the things that failed to import”
Kevin Rue-Albrecht (18:42:55): > oh right
Aaron Lun (18:42:58): > and then we use a conditional UI to hide the free-text import when no one cares about it.
Aaron Lun (18:43:57): > Just have a checkbox with “Advanced import options” or something.
Kevin Rue-Albrecht (18:44:13): > Why not. Maybe it was a bit over the top and costly to do a live detection of the invalid names
Kevin Rue-Albrecht (18:45:36): > > Just have a checkbox with “Advanced import options” or something. > That’s fine foraddingfeatures to the current list. I liked the modal for editing (i.e. the possibility of keepingsomeof the current ones)
Aaron Lun (18:46:42): > Well, you can just stick everything in the modal into this advanced import options section.
Kevin Rue-Albrecht (18:50:09): > Don’t you mean everythingexceptthe dynamic list of invalid features? because that’s the thing that requires access to theinput.
Aaron Lun (18:51:02): > Yes.
Aaron Lun (18:51:10): > Butyou can keep the editor, for example.
Kevin Rue-Albrecht (18:51:30): > Well yes, this one is the core element of the modal.
Kevin Rue-Albrecht (18:52:36): > I just realized, instead of accessinginput, I could also have the editor push/store its content intorObjects. That would have the same behaviour as now, no?
Aaron Lun (18:53:28): > The spirit of the law is that nooutputmanifests from aninputwithout passing throughpObjects.
Aaron Lun (18:54:46): > If we want these unregulated things, then we need to stick them in another generic that can seeinput,outputandse, but notpObjectsorrObjects.
Kevin Rue-Albrecht (18:55:02): > copy that
Aaron Lun (18:55:24): > Anyway, leave it for now, deal with the plotting thing first.
Aaron Lun (18:56:03): > Do we get the same speed from downsampling if we include all points in the ggplot data frame but somehow tell ggplot to skip the plots that are e.g., FALSE for aDowsanpledcolumn?
Aaron Lun (18:57:09): > This would make life a lot easier because I need to compile all plots ahead of time to generate the coordinates required for transmitting selections, and ideally I’d just make the plots and pull out theggplot``data, but the downsampling screws this up.
Kevin Rue-Albrecht (18:58:50): > I think that having a single DataFrame with all points and downsampling status as a column makes more sense from a ggplot/tidy perspective and probably goes just as fast with the subsetting done on the fly
Aaron Lun (18:59:23): > Does it?
Kevin Rue-Albrecht (19:00:07): > Well, 1 plot - 1 DataFrame.
Aaron Lun (19:00:41): > No, the question is whether ggplot is actually smart enough to skip those points.
Aaron Lun (19:00:58): > Or whether it continues to iterate over them and make zero-width points.
Kevin Rue-Albrecht (19:01:17): > ahh - well, you just sayggplot(subset(plot_data, !Downsampled))or > > ggplot(plot_data) + > geom_point(aes(...), subset(plot_data, !Downsampled), color="red") + > geom_point(aes(...), subset(plot_data, Downsampled), color="grey") >
Aaron Lun (19:01:58): > That’s still not good enough, it actually subsets the df.
Aaron Lun (19:02:19): > I want the full DF to persist in the output ggplot object, but for geom_points to recognize that it should only be using a subset.
Kevin Rue-Albrecht (19:03:21): > 2nd example above should do just that
Aaron Lun (19:05:28): > In any case, if the panel is a transmitting plot, it should emit a DF with X, Y and faceting information, so consider that when doing the plotting stuff.
Kevin Rue-Albrecht (19:05:58): > faceting or downsampling, now?
Aaron Lun (19:06:19): > Whatever is required to determine the selected points.
Kevin Rue-Albrecht (19:06:19): > (I assume the whole collection of aesthetics anyway)
Aaron Lun (19:06:43): > I don’t care about the other aesthetics. The DF has to have X, Y and faceting forallpoints.
Aaron Lun (19:07:08): > The trick is to make sure this interacts properly with downsampling.
Aaron Lun (19:07:41): > Even if downsampling is turned on, the emitted DF should still have information for all points, because the downsampling should be purely aesthetic and pertain only to that panel.
Aaron Lun (19:08:53): > Basically, you see how we’re pumping out anxy? I need that in a formal manner.
Kevin Rue-Albrecht (19:09:20): > Gotcha. I’m gonna hit the hay. Just let me know when you’re done and what I should look at when the dust settles.
Aaron Lun (19:09:47): > :chuuni:
2019-11-30
Aaron Lun (18:42:01): > getting my ass kicked here.
Aaron Lun (19:48:50): > IT LIVES
Aaron Lun (19:49:27): > MVP online. I’ll activate the remaining panels later today, but plot transmission won’t work for a while.
Aaron Lun (19:49:37): > :konata:
2019-12-01
Aaron Lun (01:58:47): > Almost all panels back online, excepting the damn heatmap.
Aaron Lun (01:58:52): > :konata::chuuni:
Aaron Lun (14:29:56): > heatmap is back online. MVP is mostly operational. Next task is to fix the various panel organization parameters.
Aaron Lun (17:57:14): > HELLO
Aaron Lun (17:57:36): > holy shit. Euro BioC is on the 9th.@Charlotte Sonesonwhen is the iSEE session?
Aaron Lun (17:57:53): > If someone helps with the documentation, we might be able to get the S4 framework done in time.
Kevin Rue-Albrecht (18:05:01): > Eh not sure what the others are up to, but I’m barely keeping up with your refactoring tbh
Aaron Lun (18:08:29): > WEll you should be able to see that the example works now.
Kevin Rue-Albrecht (18:11:53): > All the panels are up indeed. The plot panels just have an error messageobject '.lassoData' not foundand the heatmap hasinvalid type or length for slot name
Aaron Lun (18:12:29): > Don’t worry about that FTTB
Aaron Lun (18:13:53): > Keep on working on the plotting. Merge refactor-aa into your branch, lots of fixes.
Kevin Rue-Albrecht (18:16:33): > I’m not clear what’s the next step in the process. I’m still not convinced how to go about that plotting branch. We’re talking aboutkraright? I thinkkra2went too far in subfunctions
Aaron Lun (19:26:15): > I thought we had discussed this
2019-12-02
Aaron Lun (02:10:42): > Organization observers are now back up. See if you can spot the subtle change to how new panels are added.
Federico Marini (04:18:19) (in thread): > iSEE session’s on 11th
Kevin Rue-Albrecht (16:01:04): > @Federico Marini@Charlotte Sonesonhow’s the eurobioc business coming up?
Federico Marini (16:02:20): > SeeiSEEu
Federico Marini (16:02:36): > We are using a branch there to collect material
Federico Marini (16:03:00): > contrib101
Federico Marini (16:05:35): > we have a vignette and a small slideshow
Kevin Rue-Albrecht (16:06:54): > I see the vignette. I assume the slideshow is hosted on a different repo? / blogdown
Federico Marini (16:07:17): > hm, no. just different folder
Federico Marini (16:07:29): > inst, as of now
Kevin Rue-Albrecht (16:07:30): > found it, thanks
Federico Marini (16:07:59): > as long as the pkg is an open lab, there is no need to keep all pristine:smile:
Kevin Rue-Albrecht (16:08:44): > oh yeah, i was just curious
Kevin Rue-Albrecht (16:58:05): > It’s started. > Part of my note-taking homework to keep up with Aaron, I’ve kicked off the book(down) project. Don’t forget to subscribe:slightly_smiling_face:
Kevin Rue-Albrecht (16:58:50): > That said, I’ll shortly log off for today.
2019-12-03
Aaron Lun (01:16:13): > brush and lassos are back online.
Aaron Lun (01:16:56): > Be aware that the brush can be quite temperamental; don’t move the mouse while unclicking.
Aaron Lun (01:17:21): > This is an unfortunate consequence of the shiny click observer (required for our lassos) reacting to the unclick.
Kevin Rue-Albrecht (16:48:54): > Ugh. I’ve messed up something when solving conflict mergingrefactor-aaintorefactor-kra. Better to branch offrefactor-aaagain and restart arefactor-plottinginstead. I hadn’t made that much headway anyway
2019-12-04
Aaron Lun (01:05:36): > multiselection observers are back online, and the lasso data error is fixed.
Kevin Rue-Albrecht (12:19:22): > Prototyping manually (i.e., locally, not Travis) the instructions athttps://bookdown.org/yihui/bookdown/github.html, I’ve just deployed thishttps://kevinrue.github.io/iSEE-book/ - Attachment (bookdown.org): 6.2 GitHub | bookdown: Authoring Books and Technical Documents with R Markdown > A guide to authoring books with R Markdown, including how to generate figures and tables, and insert cross-references, citations, HTML widgets, and Shiny apps in R Markdown. The book can be exported to HTML, PDF, and e-books (e.g. EPUB). The book style is customizable. You can easily write and preview the book in RStudio IDE or other editors, and host the book wherever you want (e.g. bookdown.org). - Attachment (kevinrue.github.io): Extending iSEE > This is a minimal example of using the bookdown package to write a book. The output format for this example is bookdown::gitbook.
Kevin Rue-Albrecht (16:16:42): > @Aaron Lunwhile taking more notes about the new implementation, I just realized that we’re calling.cacheCommonInfofor each panel instance during initialization. That’s redundant when there are multiple panels of the same type. > There are a few ways to fix that, I just wondered whether you have a favorite.
Aaron Lun (16:17:11): > Well, actually, if you look inside, you will see that it’s a no-op if the common cache has already been initialized.
Kevin Rue-Albrecht (16:17:43): > omg i’m blind
Kevin Rue-Albrecht (16:18:00): > why do I eventhinkyou’d miss something like that
Aaron Lun (16:19:58): > damn straight
Kevin Rue-Albrecht (16:20:43): > Honestly, having started it, I realize how iSEE is genuinely worth a bookload of explanations. Even with all the internal docs we’ve already written.
Kevin Rue-Albrecht (16:22:58): > I’m not saying the current sections that I have are perfect - I barely started - but at least a book format will help shape the flow of explanation. I can’t deal with jumping between help pages. There’s just so much information everywhere.
Kevin Rue-Albrecht (16:30:53) (in thread): > Nice one on the ability to add an indefinite number of new panels for each type:slightly_smiling_face:
2019-12-05
Aaron Lun (01:35:26): > @Charlotte Soneson@Federico MariniWe won’t make it.
Aaron Lun (01:35:52): > While you should be aware of the upcoming changes, don’t plan your hackathon around it.
Aaron Lun (01:36:50): > The ideas are probably more important than the implementation, though ti should be easy enough to adapt any custom panels to the new world.
Charlotte Soneson (01:38:35): > Yes, I agree. We’ll see how it goes, still have no real idea of how many people will be there and what their interests are - will be interesting in any case.
Federico Marini (03:44:15): > Fair enough for me as well.
Federico Marini (03:44:27): > btw, we missed the birthday of the kiddo
Federico Marini (03:44:33): > or actually it is more like tomorrow
Kevin Rue-Albrecht (04:31:50): > Yup, it’s tomorrow:https://github.com/csoneson/iSEE/commit/2550b701088add87235d004ce4c42bb94e955264
Federico Marini (07:59:55): > Good foster father you are Kev
Federico Marini (08:00:17): > Kiddo turns 2, and you give him a book
Federico Marini (08:00:20): > :slightly_smiling_face:
Kevin Rue-Albrecht (08:00:25): > lol
Kevin Rue-Albrecht (08:01:25): > We could add ashowNotificationon startup with a:birthday:icon on startup
Kevin Rue-Albrecht (08:01:42): > Not sure if it would still make it in the daily build to be released tomorrow
Federico Marini (08:02:27): > :smile:don’t think so
Federico Marini (08:02:38): > we could have it permanent tho
Federico Marini (08:02:44): > if date == birthday
Kevin Rue-Albrecht (08:03:18): > speechless - that’s a fun idea!
Kevin Rue-Albrecht (08:35:19): - File (PNG): image.png
Sean Davis (08:37:52): > This channel is certainly entertaining.
Kevin Rue-Albrecht (08:42:18): > * code: check > * bump: check > * NEWS: check > I think we’re good to go
Kevin Rue-Albrecht (08:44:38): > who wants to write aSys.Date()-specific unit test? > Kidding: this new code is on the server side, which is in the “nocov” area, so no problem
Kevin Rue-Albrecht (08:46:33): > Can’t wait for Aaron to wake up to that now …
Kevin Rue-Albrecht (08:46:47): > @Federico Mariniyou really are a bad influence on me:stuck_out_tongue:
Kevin Rue-Albrecht (08:47:17): > First the voice control, now this
Federico Marini (08:47:18): > just like kids doing stuff while parents are not awake yet:slightly_smiling_face:
Kevin Rue-Albrecht (08:47:57): > wouldn’t you like having agit revertfor when your kids spill stuff eh
Federico Marini (08:48:06): > I had quite a scream when I saw one of the twins showing up at our bed with a handful of hair in his hands
Kevin Rue-Albrecht (08:48:15): > there it is xD
Federico Marini (08:48:22): > the same amount that was missing from the spot on his forehead
Federico Marini (08:48:31): > THE DAY BEFORE A WEDDING
Kevin Rue-Albrecht (08:48:32): > :angel:
Federico Marini (08:48:37): > :heart:
Kevin Rue-Albrecht (13:20:02) (in thread): > > Don’t think about the exact code in.plot_wrapperper se. Think more along the lines of “what does a extension need to specify, and what do we provide?” > I think I just finally understood what you meant there. > It’s not much about reorganizing the existing code insideplot_wrapper, as it is to identify the individual bits and pieces of logic inside that function which decide how the commands are generated, for each individual panel and family of panels. > > So far, I was trying too hard to preserve the existing code logic. However,.plot_wrapperneeds to be broken down into steps that compute (and cache somehow?) certain variables that panel developers can assume to be available to make decisions for generating certain commands (e.g.,.choose_plot_typedefines aplot.typebetween square, violin, horizontal violin, and scatter plot)
Kevin Rue-Albrecht (13:31:39): > Step specific for each dot plot panel: > * Define X,Y and optional faceting info > Among the commons steps for all dot plots: > * add optional aesthetics columns toplot.data(that probably needs to be a generic at the panel family level) > * that.choose_plot_typeis probably one of those things that we’ll compute internally once X and Y are defined, and that we’ll make available to panel developers in an environment or something > * downsampling (we can do internally once X, Y, and optional faceting info are defined) > With all that, panel developers should have all the info they need to generate the set ofggplot2commands, at which point they have to return us at least the list of commands, but probably also thexydataframe and theggplotobject (so that we don’t evaluate the commands multiple times) > > I get that this should all be wrapped within one big.getPlottingCommandsfunction for each panel, but correct me if I’m wrong about the breakdown process above.
Aaron Lun (15:29:03): > The breakdown looks braodly good, see if you can do that for the reddim plots and then I’ll probably haev some more comments.
Kevin Rue-Albrecht (17:08:44): > OK. All my attempts so far: it gets messy when I consider the.choose_plot_typepart: it’s an intersection of all dot plot panel types, no matter what class or family they’re from (#diversebioc…:sweat_smile:)
Aaron Lun (17:09:23): > Don’t expose the chooseplottype.
Aaron Lun (17:09:37): > That’s not necessary, extenders don’t need to know about that.
Kevin Rue-Albrecht (17:11:46): > that’s what I thought too
Kevin Rue-Albrecht (17:17:38): > it’s just nuts when I revisit how many interactions there are in the server code, compared to the relatively straightforward nesting of UI and observers: i.e., the difference betweencallNextMethodsand the serial calls through.plot_wrapper
Aaron Lun (17:20:52): > There should not be a lot of stuff in thecallNextMethodcalls. Just pulling out the visual parameters, etc.
Kevin Rue-Albrecht (17:21:54): > Agreed. I didn’t go anywhere when I considered acallNextMethodapproach
2019-12-06
Aaron Lun (02:36:28): > god I smell like alcohol. Will not write code tonight.
Federico Marini (10:31:16): > Just as a side thought for the core group
Federico Marini (10:31:34): > Do we want to follow up with v2 of the manuscript, once the refactoring is fully done?
Kevin Rue-Albrecht (11:03:51): > Sounds like a good idea … once the refactoring is fully done
Kevin Rue-Albrecht (11:04:17): > I keep running into catch 22 when drafting the refactoring ofplotting.Ron paper
Kevin Rue-Albrecht (18:41:05): > I got a working prototype of refactoring started: see branchrefactor-plotObviously a lot more work needed, but the concept is to have.getCodeChunkas a master wrapper.
Kevin Rue-Albrecht (18:46:52): > The flow is then as described above: > * get the panel specific commands to define X and Y data columns (this will allow us to determine the plot.type (scatter, violin, square) > * get the commands for the optional columns (color, select, …) > * get the commands for the optional downsampling > * get any additional post-processing commands (e.g. horizontal violin) > * get the actual plotting commands (i.e., ggplot)
Kevin Rue-Albrecht (18:51:01): > I’ve only refactored the first point so far intogetCommandsDataXY. The rest is still wrapped in.getCommandsDataExtra(the new equivalent of.plot_wrapper).
2019-12-07
Aaron Lun (00:02:39): > Let’s keep pushing.
Kevin Rue-Albrecht (06:42:13): > Latest plan: Panel developers should provide two functions: one for the XY axes, one for the ggplot’ing. > The reason for two functions instead of one is that once we have the XY information, we can precompute a number of things that will be made available and alter the behavior the function that generates theggplotcommand (e.g., plot.data, plot.type, is_subsetted, is_downsampled) - File (PNG): image.png
Kevin Rue-Albrecht (07:54:07): > I need a different function name for.getCommandsDataXYbecause - on top of generating the commands to initializeplot.datacolumns “X” and “Y”, it also returns the labels for the x-axis, y-axis, and plot title. > I wondered about keeping the name and separating the two jobs (commands and labels), but that would duplicate code.
Aaron Lun (15:16:42): > That seems correct, for panels subclassing from theDotPlotclass.
Aaron Lun (15:19:16): > Well, specifically,DotPlotwill have some kind of.doEverythingPlotRelated“overlord” generic that calls those two functions. Most developers will find it easiest to write methods for those two functions; however, very advanced devs may write a method for the overlord, which is more customizable but requires a lot more care to ensure that it responds to all the various parameters.
Aaron Lun (15:21:08): > Re the function name: you can just say that this function is also expected to generatecommandsthat generate “x_title” and “y_title” in the environment, otherwise we’ll give it some random names.
Kevin Rue-Albrecht (18:27:23) (in thread): > > setMethod(".getCodeChunk", "DotPlot", function(x, all_memory, all_coordinates, se, colormap) {...} > > is.doEverything(Dot)PlotRelated
Aaron Lun (18:28:15): > okay, good.
Aaron Lun (18:28:17): > KEEP PUSHING
Kevin Rue-Albrecht (18:30:20): > I got to the point of moving all of.plot_wrapperinto it now. > I tested almost all the parameters (seeiSEE-main.R). One I couldn’t test isSelectByPlotbecause there’s still a.lassoDataerror hanging around
Aaron Lun (18:32:15): > Had to make a change to process_select_by_choice to bring it back online.
Aaron Lun (18:32:20): > working on it right now.
Kevin Rue-Albrecht (18:32:41): > Anyway. I need to hang up for today. I’ll pick up tomorrow with the point downsampling and hopefully wrapping up with a brand newgetPlottingCommandthat generates purely theggplotcall
Aaron Lun (19:59:58): > refactoring tables now. this is brutal
Aaron Lun (21:32:41): > Tables are online as receivers. After dinner will be the big step; tables as transmitters.
2019-12-08
Aaron Lun (02:23:59): > Tables can now transmit to plots if any search is set on the columns or if a global search is used.
Kevin Rue-Albrecht (13:14:25): > LOL…. better late than never: I just noticed that we have a lonelysethat’s not used in the.square_plotargumentshttps://github.com/csoneson/iSEE/blob/master/R/plotting.R#L1095
Kevin Rue-Albrecht (14:09:16): > Refactored plotting is back online for RedDimPlot
Kevin Rue-Albrecht (14:10:52): > In fact, it’s online for all the panels. Only hickup is the vertical/horizontal violin layout which I seem to have flipped for RowDotPlot’s
Kevin Rue-Albrecht (14:11:45): > :sweat_smile: - File (PNG): image.png
Aaron Lun (14:16:35): > As Captain price often says
Aaron Lun (14:16:37): > KEEP PUSHING
Kevin Rue-Albrecht (14:16:59): > actually@Aaron Lun, there’s something funky going on with the x-axis observer ofRowDataPlot. Not sure if it’s your side or mine. I’ll testrefactor - File (PNG): image.png
Kevin Rue-Albrecht (14:17:52): > notice the 2 continuous variables, the x-axis being ignored
Aaron Lun (14:18:06): > didn’t toucht htat
Aaron Lun (14:18:07): > :chuuni:
Kevin Rue-Albrecht (14:19:17): > investigating now
Kevin Rue-Albrecht (14:33:37): > Fixed. Stupid order of arguments was wrong, even though I named some of them.
Kevin Rue-Albrecht (14:34:14): > I’m gonna have to stop for a while. Could you bring back up the code tracker?
Aaron Lun (14:34:33): > that’s not on today’s todo list.
Aaron Lun (14:34:45): > next stop is to allow plots to transmit to plots.
Aaron Lun (14:34:50): > re. aesthetic parameters
Kevin Rue-Albrecht (14:34:56): > fair enough, I got alapplyto print the code chunks in the console for now
Aaron Lun (14:35:04): > so select a gene in one plot and it uses it to color the points in another plot.
Kevin Rue-Albrecht (14:37:28): > Cool. Well I’ve merged yourrefactorto myrefactor-plotearlier today, solving a couple of minor conflicts, but so far so good. Hopefully, we can keep merging like that as we’re in different areas of the code.
Kevin Rue-Albrecht (14:42:39): > Damn, writing the book will be easier than updating the man pages..
Kevin Rue-Albrecht (14:56:23): > Alright, pausing it here for today. I’ve just deleted a bunch of old world plotting functions that are not in use anymore.
Aaron Lun (14:57:23): > I just renamed a bunch of files (notplotting.R) which may or may not cause merge conflicts.
Aaron Lun (14:58:25): > Well, as long as you didn’t touch anything else, you’d be fine.
Kevin Rue-Albrecht (14:58:26): > is it pushed yet? I might as well merge the conflicts ASAP, before you change code in there as well
Aaron Lun (15:00:52): > Seems like just commands.R.
Kevin Rue-Albrecht (18:51:47): > Woot woot, it works. Interactive and all after a couple of silly fixes. I need to come back at it tomorrow with a fresh mind to figure out what’s next.
Kevin Rue-Albrecht (18:57:10): > Oh wait, it seems like you’ve updated.process_selectby_choiceto return only a character vector now. It used to return the identity of the transmitter before. Crashes my code when I select. > Another reason for looking at it tomorrow.
Aaron Lun (18:59:39): > yes, it’s a character rather than the list.
Kevin Rue-Albrecht (19:00:15): > yup, i saw that, and then you populated the selection environment separately it seems
Kevin Rue-Albrecht (19:03:28): > Done and tested. Really going to sleep this time
Aaron Lun (19:17:10): > You can now select a gene in a rowdata plot and have that selection control the color in column dot plots.
Aaron Lun (21:49:09): > Panel destruction is slightly broken in that, if you delete a panel, its children panels (i.e., those receiving its selections) do not update properly.
Aaron Lun (21:49:14): > But at least it’s back.
Aaron Lun (23:11:38): > Bug fixed, but it relies on a transient invalid state of the memory where the UI is updated but those changes haven’t yet triggered the observers to update the memory (and the plots). This is… not too bad, as this happens all the time anyway, but it’s just on such a larger scale when many such changes happen at once. I don’t think there’s any weird interactions but you never know.
Aaron Lun (23:52:05): > Specifically, we would get problems if another observer tries to access the yet-to-be-updated parameter in memory before the appropriate observer updates that parameter. This can happen because the order of observer evaluations doesn’t seem to be defined if there is no dependency between them.
Aaron Lun (23:55:35): > A better solution would be to manually change things so that the memory is correct prior to re-rendering, but this would require us to also mimic the effects of the appropriate observers (because if the memory is up to date, the observers won’t bother to evaluate, and thus we’d miss out on any additional effects).
2019-12-09
Aaron Lun (00:58:33): > After thinking about it for an hour, it’s probably fine. Probably.
Kevin Rue-Albrecht (07:12:42): > Just sayin’ > (For the record, panel 2’s plotting function is currently onlyggplot()) - File (PNG): image.png
Kevin Rue-Albrecht (07:35:21): > Alright - proof of concept has landed - File (PNG): image.png
Kevin Rue-Albrecht (07:39:56): > @Federico Marini@Charlotte SonesonYou guys might want to include the last screenshot above as a teaser for the upcoming new-world implementation
Federico Marini (07:41:36): > I had my presentation already, damn
Kevin Rue-Albrecht (07:42:23): > It is as simple as > > setClass("RedDimHexbinPlot", contains="RedDimPlot") > > setMethod(".getCommandsPlot", "RedDimHexbinPlot", function(x, param_choices, plot_data, plot_type, labs, is_subsetted, is_downsampled) { > plot_cmds <- list() > plot_cmds[["ggplot"]] <- "ggplot() +" > # Adding hexbins to the plot. > new_aes <- .build_aes() > plot_cmds[["hex"]] <- sprintf("geom_hex(%s, plot.data) +", new_aes) > plot_cmds[["theme_base"]] <- "theme_bw()" > return(unlist(plot_cmds)) > }) >
Kevin Rue-Albrecht (07:43:03): > @Federico MariniI’m sure you can even fit the code chunk above in a single slide xD
Kevin Rue-Albrecht (07:45:20): > FWIW: brushes work natively on new panels. Although right now they just aren’t plotted (the “n points selected” message does show up though)
Kevin Rue-Albrecht (07:47:06): > It just hit me that if (we probably will) we allow panel developers to choose a color for new panel types, it will be … interesting
Federico Marini (07:49:46): > Can we have color gradients:stuck_out_tongue:
Federico Marini (07:49:51): > or
Federico Marini (07:50:02): > “here in the leopard-skin panel you see….”
Kevin Rue-Albrecht (07:50:45): > Gradients within panel families might be an option
Federico Marini (07:50:57): > Did you check by chance if the new world “auto-solves” the 1d- hinton plot shiny brush problem?
Kevin Rue-Albrecht (07:51:08): > e.g.RedDimHexbinPlothaving a different shade of blue toRedDimPlot
Kevin Rue-Albrecht (07:51:38): > let me see what that issue looks like again
Kevin Rue-Albrecht (07:56:31): > Ok - I was still figuring why the brush doesn’t show on new panels - it’s actually related to this new color issue: > * the brush color matches the panel header background color > * as new panels don’t have a color, neither do their brushes, i.e., they’re “NA” and drawn in invisible ink
Kevin Rue-Albrecht (08:01:23): > I’ll check with@Aaron Lunlater, but it’d be about adding 1 or 2 slots in each panel class to holdpanel_colorandbrush_fill_colorThe only alternative that I can think of - keeping the current setup - would be to apply the same color to all non-builtin panels (i.e., if our current setup returnsNA, use default color, e.g. black)
Kevin Rue-Albrecht (08:06:14): > @Federico Marinihttps://github.com/csoneson/iSEE/issues/325still broken. Thanks for the reminder.
Kevin Rue-Albrecht (08:18:34): > I should also highlight that the example above probably isn’t best practice. I just hijackedRedDimPlotto get a quick boilerplate. > Notably, this hexbin layout should not inherit the point-coloring widgets from theRedDimPlotpanel, because color and fill need to behave differently (at the bin level instead of the point level) > To do that,RedDimHexbinPlotshould probably derive directly fromColDotPloteven if that means reimplementing parts of the logic fromRedDimPlotIdeally, in the future we’ll expose some of the internal functions e.g.,.getCommandsDataFacets, so that panel developers can borrow pieces of logic that are common between panels, without having to inherit the whole panel UI and logic.
Kevin Rue-Albrecht (10:52:04): > Proof of concept for creating new panel types in iSEE 2.0 (currently on branchrefactor-plot)https://gist.github.com/kevinrue/d22c3aeac477d5879f48010c3546625b
Federico Marini (10:53:34): > Can you add also a screenshot? “for the record” + for others to check how it would look like?
Kevin Rue-Albrecht (10:53:47): > Notice all thecallNextMethod()which highlight how little effort is needed
Federico Marini (10:53:52): > (plus, I’m still not on devel on the MBP)
Kevin Rue-Albrecht (10:54:50): > gists being code files, I don’t see how you want me to add a screenshot there
Kevin Rue-Albrecht (10:55:14): > but that gist is just the code for the latest screenshot I pasted above
Kevin Rue-Albrecht (10:55:48): > If you like, I could commit code and screenshot toiSEEu
Kevin Rue-Albrecht (10:56:18): > We’ll just have to define a file structure to differentiate between modes and custom panels
Kevin Rue-Albrecht (10:57:34): > Basically: > * mode<NameOfMode>.R > * panel_<PanelClass>.R
Kevin Rue-Albrecht (10:58:03): > to underscore or not to underscore that is the question
Federico Marini (11:07:30) (in thread): > You could add a comment I guess?
Federico Marini (11:08:35) (in thread): > (and copypaste an image - function is underappreciated (or well hidden))
Kevin Rue-Albrecht (11:09:18) (in thread): > Ohhhh. Good one!
Aaron Lun (11:33:17) (in thread): > This is more complex than it seems as the JS needs to be set up to propagate color for the boxes. Just leave it alone for now.
Kevin Rue-Albrecht (11:36:23) (in thread): > Gotcha. Just to play with the brushes, I currently have the uncommited > > stroke_color <- ifelse(is.na(stroke_color), "gray10", stroke_color) > fill_color <- ifelse(is.na(fill_color), "gray90", fill_color) >
Aaron Lun (11:37:11) (in thread): > Well, you can commit that, just add a TODO on top of it.
Aaron Lun (11:37:22) (in thread): > Maybe makeutils_color.Rrather thanpanel_color.R
Kevin Rue-Albrecht (11:38:19) (in thread): - File (PNG): image.png
Aaron Lun (11:39:25) (in thread): > Also don’t bother trying to force a connection from the visual parameters with the plot; I will eventually add some options to allow people to disable irrelevant UI elements.
Kevin Rue-Albrecht (11:39:38): > @Federico MariniI’ve added this screenshot to the gist - Attachment: Attachment
Kevin Rue-Albrecht (11:40:10) (in thread): > Excellent. Exactly my thought
Kevin Rue-Albrecht (11:40:39) (in thread): > For instance, this hexbin panel would need to disable the downsampling UI and logic, to protect end-users from themselves
Aaron Lun (11:40:54) (in thread): > Well, the logic doesn’t need to be disabled, it would just be ignored.
Kevin Rue-Albrecht (11:41:04) (in thread): > or that indeed
Kevin Rue-Albrecht (11:41:49) (in thread): > which reminds me that I also realize that we currently initialize the voice observers even when the JavaScript library isn’t loaded. It doesn’t hurt, but we could condition the observers as we do for the JS
Kevin Rue-Albrecht (12:11:14): > One thing that I’ve been tracking down for a while now and I can’t figure out why is that the app crashes with error > > Warning: Error in rep: invalid 'times' argument > 75: brushedPoints > 73: .get_brushed_points [/Users/kralbrecht/git/iSEE/R/utils_brush.R#131] > 72: observeEventHandler [/Users/kralbrecht/git/iSEE/R/observers_dimname.R#10] > 1: shiny::runApp > > if I try to initialize panels with a brush > > RedDimPlot1[["BrushData"]] <- list( > xmin = -45, xmax = -20, ymin = -35, ymax = -5, > mapping = list(x = "X", y = "Y", group = "GroupBy"), > direction = "xy", > brushId = "RedDimPlot1_Brush", outputId = "RedDimPlot1") > > Anyway. Not top priority, but one of those things we’ll need to look at when cleaning up before merging tomaster
Kevin Rue-Albrecht (14:24:53) (in thread): > Actually, much better than R scripts files iniSEEu, new panels would be better distributed as separate R packages. > This way they can declare their dependency (Imports:) oniSEE, import the NAMESPACE, and define new classes of panels available to end-users.
Aaron Lun (14:25:30) (in thread): > Well, iSEEu is meant to be an R package that does that.
Kevin Rue-Albrecht (14:27:52) (in thread): > Oops. For a second I forgot that iSEEu is formatted as an R package -_-
Kevin Rue-Albrecht (14:29:08) (in thread): > It’s signs like that which let me know when I should stop working for the day:sweat_smile:
Kevin Rue-Albrecht (14:59:00): > wow that last one was fun: there was a bug initializing a new lasso when the previous one was closed - go find that bug except by chance while playing around with the app
Kevin Rue-Albrecht (16:22:25): > https://dev.to/michaeljolley/why-did-the-angry-function-exceed-the-callstack-size-1fdk > > Why did the angry function exceed the callstack size? - Attachment (The DEV Community): Why did the angry function exceed the callstack size?
Aaron Lun (16:22:47): > ho ho ho
Kevin Rue-Albrecht (16:22:52) (in thread): > > - It got into an Argument with itself
Kevin Rue-Albrecht (16:23:26) (in thread): > I also like someone’s follow up comment : > > It cursed and recursed…
Aaron Lun (23:44:21): > So, did that fix get committed, or what?
Aaron Lun (23:55:01): > Executive decision: heatmap cannot zoom anymore.
Aaron Lun (23:55:19): > In fact, we have to overhaul the heatmap completely to clean up the code, which is now a slobbering heap.
2019-12-10
Aaron Lun (00:15:01): > OMG the heatmap plotting code is disgusting.
Aaron Lun (00:15:20): > That goes firmly into “someone else’s problem”.
Aaron Lun (00:29:46): > Added a little aesthetic touch to keep everyone motivated.
Aaron Lun (01:41:09): > (Reorging panels is known to make it do funny things, though.)
Aaron Lun (01:43:27): > I’ll probably take it out at some point. But in the meantime, enjoy.
Aaron Lun (01:54:49): > And that’s fixed as well, so now we can keep it in.
Aaron Lun (01:55:14): > Code tracker and graph maker are back online.
Kevin Rue-Albrecht (02:15:59) (in thread): > What’s “that”?
Kevin Rue-Albrecht (02:18:24) (in thread): > I was hoping to see a few heatmap implementations in iSEEu once the refactoring is done. Then we’ll be able to decide if one of them can be promoted as an iSEE builtin to replace the current one
Kevin Rue-Albrecht (02:19:05) (in thread): > I’m totally in favor of that. I never used the zoom feature in heat maps so far.
Kevin Rue-Albrecht (04:24:15): > Note to self:In the new world framework, it might be possible to create a family of “startup panels”, which would exclusively be available for the GUI when iSEE is launched without theseargument, e.g.iSEE(). > Those would allow import of anseobject (and possibly preprocessing) into the interface from various sources. > The first of those panels would beRdsUploadPanelthat includes a single shiny file upload button doing exactly the current behaviour ofiSEE(). > Other panels could includeExperimentHubPanel, to download and cache datasets.
Kevin Rue-Albrecht (04:27:29) (in thread): > Yeah it did. I’ll open a PR to mergerefactor-plottoday. It’s stable enough, that I actually have to pause and reflect on what’s my next move. There are clearly things to polish, I just need to make a list before diving in.
Kevin Rue-Albrecht (04:42:04) (in thread): > @Federico Mariniwill be thrilled:grin:
Federico Marini (07:45:49): > aaah the spinner
Federico Marini (07:45:51): > big fan
Federico Marini (07:46:01): > big big fan
Federico Marini (07:46:43) (in thread): > I am indeed
Kevin Rue-Albrecht (08:13:38) (in thread): > @Federico Marinido me a favour and try out branchkra-dna
Federico Marini (08:40:02): - File (PNG): image.png
Kevin Rue-Albrecht (08:54:09): > I’m still having issue adding a custom GIF. I’ve put one in ourwwwfolder, as indicatedhttps://github.com/emitanaka/shinycustomloader, but it’s not found by the app:confused:
Kevin Rue-Albrecht (08:55:07): > Nevermind. Lunch helped. We’re makingwww/available asiSEE/in the app
Kevin Rue-Albrecht (09:45:37): > @Aaron Lunas far as I’m concernedrefactor-plotcan be merged torefactorseehttps://github.com/csoneson/iSEE/pull/331I’m just waiting in case you want to do something onrefactorbefore this merge.
Aaron Lun (11:38:52): > Do it.
Aaron Lun (11:38:59): > Also your spinner had better still work after re-rendering.
Kevin Rue-Albrecht (11:57:34): > It rendered on initialization, and when resizing the window. The spinner is on a separate branch. I’m preparing a better one though
Kevin Rue-Albrecht (12:13:37): > How are people’s preferences on those two? - File (GIF): F.gif - File (GIF): FR.gif
Aaron Lun (12:13:50): > there’s a glitchy bit for me.
Aaron Lun (12:13:58): > bottom left green squares
Kevin Rue-Albrecht (12:14:35): > I see them on the miniatures too, but not if you open each file separately
Aaron Lun (12:15:26): > hm, okay.
Kevin Rue-Albrecht (12:15:57): > Now I just need to find a software to make them without the watermark (I usedhttps://imgflip.com/gif-makerfor this one) - Attachment (imgflip.com): GIF Maker - Imgflip > Easily create animated GIFs online from videos, Youtube, video websites, or images. Simple yet powerful customization options.
Kevin Rue-Albrecht (12:17:22): > So… FR or just F? I prefer forward-only, it gives better a sense of “loading”
Aaron Lun (12:17:38): > the left one.
Kevin Rue-Albrecht (12:46:24): > Pushing the ‘Bioc’ loader. Occasionally, it doesn’t show up during initialization - it might be if the GIF is loaded too late (?) > In any case, if it bothers you, just switch totype = "html", loader = "dnaspin", it’s a standard one which doesn’t cause the issue
Kevin Rue-Albrecht (12:54:16): > Merged.
Kevin Rue-Albrecht (12:55:57): > FYI, I amnotdiving into the HTML+CSS+webkit way of writing a custom loader: > * https://github.com/emitanaka/shinycustomloader/blob/master/inst/css-loaders/html/dnaspin.html > * https://github.com/emitanaka/shinycustomloader/blob/master/inst/css-loaders/css/dnaspin.css > Ouch
Kevin Rue-Albrecht (12:56:40): > One thing I could do is to make the GIF smaller. It’s a bit of a punch in the eye right now.
Aaron Lun (12:57:39): > Or transparent.
Kevin Rue-Albrecht (12:58:27): > Damn. Didn’t check that. Must be the GIF website that added the background. There’s wasn’t any in illustrator
Kevin Rue-Albrecht (13:04:25): > I solved the transparency, but now I run into a pixelation issue at the borders. I definitely need to find a proper software or website for that
Kevin Rue-Albrecht (13:04:49): > I can share the individual images in Dropbox if someone has better skills at putting them together
Kevin Rue-Albrecht (13:05:26) (in thread): > https://www.dropbox.com/sh/4qetcsu915bvzq7/AACxhyAbj6uT0yZp0ovezAtYa?dl=1
Kevin Rue-Albrecht (13:18:44): > There you go, new PR.
Kevin Rue-Albrecht (13:20:21): > Resizing the GIF (width and height) will take me more time (need to resave each image). I’ll revisit it later if we stick with the idea
Kevin Rue-Albrecht (16:08:56): > Mmmh. I just noticed that one annoying thing about any kind of loader is the interaction/interference with the lasso selection
Kevin Rue-Albrecht (16:09:57): > That might be resolved ifshinyitself someday implements a client-side lasso. But as long as each click causes a rerendering on the server side, there’s no way around it
Aaron Lun (16:20:17): > In any case, I won’t be touching the code base until the weekend, at which point I will do the final things: > * All panels can receive row/column selections. > * Architecture to exclude parameters from the UI.
Aaron Lun (16:20:46): > If you want to keep busy, the most orthogonal thing is to write up how you made a subclass.
Kevin Rue-Albrecht (16:21:37): > I was thinking of drafting a chapter about that in the book. We’ll probably need a new vignette for that in the package
Aaron Lun (16:26:21): > Well, the book should be almost all about subclassing.
Kevin Rue-Albrecht (16:32:23): > Yeah. But I’d like to take a teaching by example approach, walking users over consecutive chapter sections through the creation of a subclass, adding a UI, adding an observer, adding the XY data logic, adding the plot logic. I
Kevin Rue-Albrecht (16:33:08): > If it’s done nicely, it should flow well for all levels of expertise
Aaron Lun (16:34:50): > Yes. That’s what I mean.
Aaron Lun (16:35:21): > Consider “basic”, “advanced” and “extreme” sections, increasing the sophistication of panel design at each step.
Kevin Rue-Albrecht (16:36:48): > Yeah well even I only wrote a basic custom panel at this point:sweat_smile:
2019-12-11
Kevin Rue-Albrecht (06:04:01): > New version, without the hexsticker border, more similar to typical spinners. Comes with an emoji. - File (GIF): Bioconductor-loader-800.gif
Christine Choirat (12:09:39): > @Christine Choirat has joined the channel
Kevin Rue-Albrecht (14:46:43): > @Aaron LunI’ve updated the loader to use the note without the hexsticker border (above). – Much nicer and less punch-in-the-eye. Directly pushed torefactor. You’ll have to pull another time before working on that branch. > I won’t have any time to touch the code until Xmas. Stuff came up. I’ll probably have enough postprandial time over Xmas to get the docs going, in the package and/or the book
2019-12-12
Ruizhu HUANG (05:09:35): > @Ruizhu HUANG has joined the channel
2019-12-13
Kevin Rue-Albrecht (05:03:53): > Anyone feels like trying out branchkra-addin?:innocent:
Kevin Rue-Albrecht (05:04:45): > I’m looking at you@Federico Marini:wink:
Kevin Rue-Albrecht (05:26:52): > Motto of the day: “1 addin is worth a thousand keystrokes”
Federico Marini (05:31:41): > ehehehe
Federico Marini (06:02:43): > Wanna talk keystrokes Kev?
Federico Marini (06:03:13): > https://twitter.com/DooleyMurphy/status/1191744575980941312?ref_src=twsrc%5Etfw - Attachment (twitter): Attachment > Accidentally duplicated every item in my @zotero library. > > A sane person would’ve written a script to automate the merging process. > > I took the analogue approach.
Kevin Rue-Albrecht (06:06:05): > What in the name of nerd ?!?:rolling_on_the_floor_laughing:
Federico Marini (06:07:07): > Cannot oversee the craftmanship
Kevin Rue-Albrecht (06:07:45): > I mean that is both genius and nuts ^^
Federico Marini (06:09:49): > oui
Kevin Rue-Albrecht (06:11:29): > Btw, new iSEE mode idea:mode_WGCNA - File (PNG): image.png
Kevin Rue-Albrecht (06:16:30): > Not sure exactly all that would go in that mode. It would definitely require a few new panels too, but having just written a project-specific app to help me cross reference the results, I can see a plan there
Tim Triche (10:38:08): > this looks super handy
Kevin Rue-Albrecht (10:41:42): > Yeah - also, I didn’t explain: the small table in the side bar is a “tracked genes” thingy. I’ve got a list of some famous landmark genes, and when I select a module, it tells me if any of them is in the module I’m looking at. Giving me a quick idea of the module theme, on top of the GO
Kevin Rue-Albrecht (10:42:29): > Can’t do that in iSEE though (yet:stuck_out_tongue:)
Kevin Rue-Albrecht (10:48:28): > Anyway - I can’t promise anything now, but I’ll see if I can generalize the app once I’m through with the project itself. Definitely handy to curate the modules.
Aaron Lun (11:31:08): > Why do we have an add-in thing in iSEE?
Kevin Rue-Albrecht (11:31:45): > Because it pops up a template to make a new panel. Have you tried it yet?
Aaron Lun (11:32:26): > Seems pretty rstudio specific.
Kevin Rue-Albrecht (11:33:06): > Oh yeah it is, but it will make life so much easier for both us (I’m already enjoying it) and (especially) other panel developers working in RStudio
Aaron Lun (11:33:36): > Hmph.
Kevin Rue-Albrecht (11:34:54): > Try it. The template that’s generated is already a working panel. It just does exactly what it’s parent class does, but at that point users are presented with the full API in the template R script, and all they have to do it substitute whatevercallNextMethod()is relevant.
Kevin Rue-Albrecht (11:35:05): > Oh wait. It’s actuallynotRStudio-specific
Kevin Rue-Albrecht (11:36:07): > You can use the addin by callingnew_panel_file("MyFancyNewPlot", "Awesome new panel", parent="DotPlot")
Kevin Rue-Albrecht (11:36:26): > I almost forgot how awesome this is
Kevin Rue-Albrecht (11:37:48): > Hmm.. wait again. Damn of course it is RStudio specific, I’m usingrstudioapi::documentNew(template_content)to open an Untitled file
Aaron Lun (11:38:41): > Shouldn’t this go into iSEEu?
Aaron Lun (11:39:01): > Seems like exactly the sort of thing that would go there.
Aaron Lun (11:39:23): > especially if people don’t need to use it to run iSEE itself.
Kevin Rue-Albrecht (11:39:28): > sure
Kevin Rue-Albrecht (11:41:21): > Anyway. Point is that we have a template script to define new panels. Whether users use the addin in RStudio or copy the file themselves frominst/templatesand edit it themselves is not really the issue at that point.
Kevin Rue-Albrecht (11:41:56): > One bonus thing the addin does is the substitution of placeholders with the encoded, decoded, and parent class name.
Kevin Rue-Albrecht (11:42:20): > I’ll swap that to iSEEu when I get home. Finishing something else now.
Kevin Rue-Albrecht (14:04:43): > Aaron, pullrefactor, I’ve moved the addin toiSEEu
Aaron Lun (14:17:49): > Just make sure the plotting panels are operational for my hacking tomorrow.
Kevin Rue-Albrecht (14:26:18): > All looks fine to me
2019-12-14
Aaron Lun (17:51:16): > This change was more epic than I had thought.
Aaron Lun (20:56:37): > Oh this is brutal.
Aaron Lun (20:56:45): > It’s been 5 hours and I haven’t made a single commit.
Aaron Lun (21:55:48): > Finally got the UI to load.
Aaron Lun (21:55:53): > Now I can see the Gif for the first time.
Aaron Lun (22:21:27): > This PR is up to 7 kLocs.
Aaron Lun (22:21:43): > Almost 2 klocs added.
Aaron Lun (22:23:16): > On an unrelated note, I have a mohawk again.
2019-12-15
Aaron Lun (02:23:03): > We’re going to hit 300 commits on this PR before it’s over.
Kevin Rue-Albrecht (08:18:49): > It’s all worth it:slightly_smiling_face:Hopefully I’ll get that vignette done for panel developers in the meantime.
Aaron Lun (14:14:08): > Yeah, the mohawk si amazing.
Aaron Lun (14:42:44): > Transmission to self is now forbidden, which simplifies code a lot and the UI a little.
Aaron Lun (15:11:44): > Generic’d the generation of commands inprocess_selectby_choice. This enables completely arbitrary transmissions from non-DotPlot or Table panels.
Aaron Lun (15:13:46): > For example: you can have aPanelthat holds a table of gene sets, one per row. You can set it up to transmit along rows via.transmittedDimension, and then set up.processTransmissionso that it transmits the identities of thegenes in a selected gene setto another panel.
Aaron Lun (15:14:00): > Incidentally,@Kevin Rue-Albrecht, your GIF is still sticking around after re-rendering for me.
Kevin Rue-Albrecht (15:22:44): > Uh. Never saw that GIF issue myself.
Kevin Rue-Albrecht (15:23:58): > That select-by update sounds amazing.
Aaron Lun (18:00:58): > Was at safeway buying stuff
Aaron Lun (18:01:25): > this staffer was stocking the shelves. She must have been in high school or barely out of it.
Aaron Lun (18:01:31): > then she said, “excuse me sir”
Aaron Lun (18:01:46): > and after a few seconds I realized she was talking to me.
Aaron Lun (18:02:00): > I DON’T LOOK OLD ENOUGH TO BE A “SIR”
Aaron Lun (18:02:42): > So I’m going to ease my existential anguish by working on this code.
Aaron Lun (18:49:31): > Right, we’re back.
Aaron Lun (18:51:27): > So, to sum up the situation, all panels can receive from selections on both rows and columns. The UI elements for the other dimension are disabled for the default panels but it is trivial to re-enable them by specializing.hideInterfaceElementfor a new panel class. This enables stuff like dynamically performing a PCA on a subset of cells based on a subset of genes.
Aaron Lun (18:53:08): > Right. Off to buy a secret santa gift.
Aaron Lun (18:54:50): > @Kevin Rue-Albrechtnote the TODO elements for you in the PR. Almost all architectural changes have been made on my end so I will spend the next week cleaning up the documentation.
Aaron Lun (21:15:28): > Persistence GIF after reorg’ing the panels. Another problem is that the gif pops up halfway through me trying to make a brush… killing the brush. - File (PNG): Screenshot from 2019-12-15 18-13-28.png
Aaron Lun (22:42:14): > Keep up the fight! The end is near!
2019-12-16
Kevin Rue-Albrecht (03:27:01): > Which web browser are you using ? I’ve been testing with Chrome mostly and never had that GIF issue so far.
Aaron Lun (11:33:08): > chrome.
Kevin Rue-Albrecht (14:23:15): > What do you do to get that persistent GIF exactly? anything?
Kevin Rue-Albrecht (14:23:45): > Oh right got it now
Kevin Rue-Albrecht (14:24:12): > Reorganizing indeed causes the issue. I wasn’t getting the error when only resizing the window
Kevin Rue-Albrecht (14:24:15): > owww
Kevin Rue-Albrecht (14:24:46): > Damn that’s annoying
Aaron Lun (23:09:01): > Suggest reverting back to the thing I had before, which at least avoids this AFAICT.
2019-12-17
Kevin Rue-Albrecht (02:51:59): > Agreed.
2019-12-19
Mike Smith (08:05:29): > @Mike Smith has joined the channel
Mike Smith (08:10:40): > For this afternoon devel call we’re going to do a brief rundown of what was achieved in the developers day at EuroBioc.@Charlotte Sonesonhas volunteered to sum up the iSEE session, but we were wondering if either@Kevin Rue-Albrechtor@Aaron Lunhad any words of wisdom on iSEE2 and whether that changes things significantly. Feel free to say no with < 3 hours notice, but this doesn’t need to be anything super prepared, maybe just some key points that have changed. Alternatively, if you think it warrants a longer session, maybe we can schedule something for next year. Cheers
Kevin Rue-Albrecht (09:48:43): > sure i’ll do my best to join in
Aaron Lun (15:49:50): > I realized we need a new set of generics to distinguish between multiple selections and single selections. Ugh.
Kevin Rue-Albrecht (15:52:27): - File (PNG): image.png
Kevin Rue-Albrecht (16:44:37): > incoming commits
Kevin Rue-Albrecht (16:44:56): > That takes care of item n. 1
Aaron Lun (17:14:24): > Ran into a bizarro world bug when DataTable wasn’t working.
Aaron Lun (17:14:36): > THen I realized I was callingshiny::dataTableOutputinstead ofDT::dataTableOutput.
Aaron Lun (17:24:44): > @Kevin Rue-AlbrechtWhat about theenv?
2019-12-20
Aaron Lun (00:36:19): > Anyway, the single/multi split is done. Note a whole bunch of things got renamed, so be sure to pull before making new commits.
Aaron Lun (00:37:07): > holy craphttps://github.com/csoneson/iSEE/pull/327/commits: > > This pull request is big! We’re only showing the most recent 250 commits.
Aaron Lun (00:37:10): > Never gotten that before.
Kevin Rue-Albrecht (05:04:52) (in thread): > Been looking at the code and planning, but I can’t give you an estimate on when I’ll get it done. It’ll happen. Enjoy Xmas in the meantime:slightly_smiling_face:
Kevin Rue-Albrecht (05:05:58): > Yeah - I had to bookmark the comment with the TODO list that you left for me, because otherwise I have to expand the list of commits 4-5 times before the comment shows up …
Kevin Rue-Albrecht (05:14:49): > Well, it’s calledrefactorfor a reason too. I just hope that after that we’ll earn a well-deserved period of gratifying rest watching the community turning into panel developers themselves.
Kevin Rue-Albrecht (10:29:09) (in thread): > I’m sure you’re aware of it, but just a reminder that the app crashes when I try to save a selection. > Error message is > > Warning: Error in .multiSelectionToSave: could not find function ".multiSelectionToSave" > 73: [[ > 72: observeEventHandler [/Users/kralbrecht/git/iSEE/R/observers_brush.R#158] > 1: shiny::runApp > > Because.multiSelectionToSaveis not defined yet.
Aaron Lun (11:19:31) (in thread): > huh.
Aaron Lun (11:33:58) (in thread): > Fixed.
Kevin Rue-Albrecht (13:22:35) (in thread): > I fixed a typo. Don’t forget to pull.
2019-12-21
Aaron Lun (21:29:42): > 10 kLOCs reached.
2019-12-22
Aaron Lun (01:58:25): > link information back online.
Aaron Lun (20:15:11): > Generics online for colors.
Aaron Lun (20:16:44): > Right, I’ve had enough for the day.
Aaron Lun (21:51:04): > of course
2019-12-23
Aaron Lun (22:24:15): > man, if you forget toforcebefore using a variable in an observer, you really get this weird twilight zone debugging experiences.
2019-12-24
Aaron Lun (00:18:32): > blew past the 300 mark
Aaron Lun (17:43:31): > had enough. Time to clean the toilet.
Aaron Lun (18:41:59): > Phew.
Aaron Lun (18:56:28): > after watching some kiki’s delivery service, my heart is back in the game. Documenting the observers now.
2019-12-25
Aaron Lun (01:15:45): > just realized I don’t have any lunch to eat tomorrow.
Aaron Lun (01:15:49): > Well, guess it’s subway again.
2019-12-26
Aaron Lun (19:28:48): > Hello.
Aaron Lun (19:28:51): > ?
Aaron Lun (19:28:56): > Is anyone back yet?
2019-12-27
Kevin Rue-Albrecht (04:35:39): > I pushed last night my WIP new vignette for (basic) panel development. Branchrefactor-vignette. Poorly chosen name as the the “configure” vignette is the one that will need refactoring; what I’m doing is describing how to developnewpanel classes.
Kevin Rue-Albrecht (04:38:19): > FYI:https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btz877/5640500
Kevin Rue-Albrecht (04:41:36): > > Cerebro aims to overcome the technical hurdles and allow direct and interactive exploration of pre-processed scRNA-seq results. > That’s harsh for iSEE, it does the same.
Aaron Lun (13:52:36): > there is literally a milliion of these apps, I wouldn’t worry about it.
Aaron Lun (13:53:06): > Also, just shaft panel devel to the book, don’t bother putting it in vignettes.
Aaron Lun (14:21:36): > Dealing with very difficult potential bug related to the fact that reactive evaluation order should not be predictable.
Aaron Lun (15:42:10): > Phew. That’s done. NEed to have a think about whether that was the best way to do it, but it’s done.
Kevin Rue-Albrecht (16:35:03): > > reactive evaluation order should not be predictable > Not even sure what you mean.
Kevin Rue-Albrecht (16:49:16): > > Also, just shaft panel devel to the book, don’t bother putting it in vignettes. > Aargh. I ended up trying to have a lightweight version as a vignette, while shafting the advanced details into the book. > But all in the book would definitely give greater freedom for content, chapters, cross-references, and thoroughness
Aaron Lun (17:19:27): > See my latest chapter 07.
Aaron Lun (17:19:47): > I think book content should be all about case studies and worked examples. Leave the technical stuff for the man pages and just link to them.
Aaron Lun (18:52:30): > augh
Aaron Lun (18:52:32): > auwahg
Aaron Lun (18:52:41): > this reactive nonsense hurts so much
Aaron Lun (19:58:00): > :sob:
Aaron Lun (19:58:10): > I’ve been hitting my head against this for 3 hours
Aaron Lun (21:07:57): > I DID IT
2019-12-28
Aaron Lun (00:56:05): > phwa. Okay. Docs are all fixed up except for heatmap and voice control.
Kevin Rue-Albrecht (02:20:03): > Well voice control is my responsibility. Also because i haven’t checked yet if it still works in the new world
Kevin Rue-Albrecht (02:21:29) (in thread): > Fait enough. Chapter 1 was indeed a pain : listing all parameters of all panel classes. If you’ve done that in man pages, no point duplicating the effort
Aaron Lun (02:22:02): > O
Aaron Lun (02:22:02): > M
Aaron Lun (02:22:03): > G
Aaron Lun (02:22:09): > Check out 08 in the book.
Kevin Rue-Albrecht (11:40:55): > Someone gave themselves a Christmas toy for endless fun:yum:
Aaron Lun (18:51:09): > https://ltla.github.io/darth-circle/ - Attachment (ltla.github.io): Continuous integration in a galaxy far, far away > If you don’t get this, you’re not nerd enough:
2019-12-29
Aaron Lun (01:44:57): > Approaching 400 commits.
Kevin Rue-Albrecht (13:13:49): > Quick fix pushed torefactor: selectize shows panel full names again
Aaron Lun (13:18:39): > Current priority is to get the heatmaps operational again. This is an opportunity to go nuts with ComplexHeatmap.
Aaron Lun (13:19:06): > I am literally grinding throughtest_plotting.R, so don’t touch that.
Kevin Rue-Albrecht (13:20:39): > Also: I’ve manually recompiled and pushed the updated book for previewing. That said, I’ll do it again to move the references “chapter” at the end again
Kevin Rue-Albrecht (13:26:10): > The voice feature needs an update too, because available panels are not just listed as a character list anymore (panelTypes). > I’ve written some code for the RStudio addin in iSEEu, to detect “Panel”-derived classes, but that seems to work only for those defined in packages: > > x <- unique(c(getClasses("package:iSEE"), getClasses("package:iSEEu"))) >
Aaron Lun (13:39:27): > just trawl thereservoir.
Kevin Rue-Albrecht (13:44:00): > fair enough - that’s all I need at runtime indeed
Aaron Lun (16:15:18): > god the plotting tests are fixed.
Kevin Rue-Albrecht (17:04:32): > glad you went through those
Kevin Rue-Albrecht (18:17:24): > I’m pausing here for today, but tomorrow I’ll merge the two chapters about the API (“panel API” and “plotting API”).
Aaron Lun (18:23:39): > The book isn’t really a priority right now
Aaron Lun (18:29:10): > I was only adding stuff to it because I was moving stuff out of the vignettes
2019-12-30
Aaron Lun (03:45:54): > All tests back online except for heatmap and voice.
Aaron Lun (03:46:25): > I’m done. Over to you@Kevin Rue-Albrecht.
Federico Marini (04:18:26) (in thread): > they recently added some commands to git
Federico Marini (04:18:35) (in thread): > maybe for the next round you can ping them
Federico Marini (04:18:41) (in thread): > as an easter egg
Federico Marini (04:18:52) (in thread): > that command works only on may 4th
Kevin Rue-Albrecht (04:32:01): > :ok_hand:
Kevin Rue-Albrecht (04:50:37): > I wanna look at voice today, see how far I get. > For the heatmap, I say we open an iSEE contest, with prizes like: > * One iSEE T-shirt > * All hex/iSEE stickers we can find > * Special & persistent recognition by iSEE in the form of an iSEE panel, and a badge that’ll be publicly visible in a custom iSEE mode > * Half-an-hour one-on-one with a representative from the iSEE team for Q&A and feedback > Y’know, something inspired from:https://blog.rstudio.com/2019/01/07/first-shiny-contest/ - Attachment (blog.rstudio.com): Announcing the 1st Shiny Contest | RStudio Blog
Kevin Rue-Albrecht (11:43:44): > Damn adding a new panel to the GUI got a lot more complicated. I duplicated some code for the proof of of concept, but panels can be added again by voice using “create panel “. > Branchrefactor-voice
Kevin Rue-Albrecht (12:04:20): > “Remove panel” command restored.
Aaron Lun (12:50:52): > I don’t care either way about the voice, but you’re going to have to do the heatmap yourself. Too critical to entrust to randoms, and we need to get this done.
Kevin Rue-Albrecht (12:57:58): > I know:slightly_smiling_face:
Kevin Rue-Albrecht (13:01:20): > I just got the “receive from” and “send to” vocal commands left now. Looks like “create” was the only tricky one.
Aaron Lun (18:06:20): > @Charlotte Soneson@Federico Marinithe new API is more or less ready for testing, see Chapters 7-9 of the book for a few worked examples.
2019-12-31
Aaron Lun (01:11:04): > Hm. Just spent two hours watching a halo 4 playthrough.
Aaron Lun (01:11:15): > Decided I don’t regret not playing that one.
Kevin Rue-Albrecht (06:11:04): > I just recompiled and deployed the book (kevinrue.github.io/iSEE-book) > I also merged the refactored voice control (including unit tests and docs) torefactorNext up: (Complex?)Heatmap. But that’ll be after lunch if not next year (ho-ho-ho).
Aaron Lun (14:16:34): > Exposed a few things that touchedheatmap.R.
Aaron Lun (14:32:08): > Note that internal non-slot variables that are added toinput,rObjectsoroutputshould be prefixed withINTERNAL, see some examples inconstants.R.
Aaron Lun (14:32:37): > This allows people to add their own stuff without worrying about overwriting our objects.
Aaron Lun (14:32:45): > Slot-related things are fine because they’re exposed and part of the official API annyway.
Aaron Lun (19:27:10): > Yep, definitely not missing not playing halo 4.
Aaron Lun (20:46:26): > Example gene set table created. BEHOLD!
2020-01-01
Aaron Lun (02:27:23): > Just doing my expenses. My disposable income (after tax + rent + food) is 60 k a year.
Aaron Lun (02:27:56): > That’s more than my gross salary in my postdoc!:moneybag:
Aaron Lun (02:28:02): > :konata:
Kevin Rue-Albrecht (11:24:56): > Progress on the ComplexHeatmap class on branchrefactor-heatmapBest served with > > app <- iSEE(sce, initial = list(ComplexHeatmapPlot(SelectRowSource="RowDataPlot1", SelectColSource="ColDataPlot1"), ColDataPlot(), RowDataPlot())) > > As we’re reimplementing it anyway, I would very much like to stay away from the clunky “add/clear features in heatmap by hundreds” feature that required a bunch of extra buttons and observers and not really knowing which features got added anyway in the end. > Bottom line is that the new heatmap behaves exactly like a regular panel: it receives selections (possibly unions) from other panels.
Kevin Rue-Albrecht (11:27:35): > Only limitation is that - for its own protection - the heatmaprequiresincoming selections on both rows and columns. Otherwise it shows a blank plot. That is because it would be a really bad idea to show the entire assay indicating incoming selections by colors or transparency (I’ve tried)
Kevin Rue-Albrecht (11:28:35): - File (PNG): image.png
Kevin Rue-Albrecht (11:29:09): - File (PNG): image.png
Kevin Rue-Albrecht (11:31:45): > Next slots-UI-observers-commands needed: > * show/hide row/column names > * cluster yes/no column/rows > * legend direction vertical/horizontal (right now it isn’t pretty when position==right) > * assay selectize (prototype uses the first available assay) > Taking a break for today though.
Kevin Rue-Albrecht (11:35:48): > I gotta say, the panel color isn’t that bad for a completely arbitrary choice “[color content]”. A bit pale, but not bad.
Aaron Lun (12:54:36): > The thing is, someone is going to want to upload their own features.
Aaron Lun (13:26:12): > That actually seems to be the no 1 use case.
Kevin Rue-Albrecht (13:28:45): > Perhaps a separate panel dedicated to uploading features? Eg « FeatureSetPanel »Could be a simple text box.Bonus would be that multiple panels could listen on it to receive the same gene list
Aaron Lun (13:29:16): > That’s far too complicated for the average user.
Aaron Lun (13:31:42): > What’s wrong with having just a modal that opens up an aceEditor to edit/upload stuff? That way it’s hidden most of the time unless people need it.
Aaron Lun (13:32:08): > You can even store the editor’s text in a slot so that it persists and can be dynamically edited.
Aaron Lun (13:32:34): > E.g., adding hashes to the front of invalid gene names to indicate that they are not being used.
Kevin Rue-Albrecht (13:45:10): > Still. That makes it the only built in panel that has an asynchronous relationship with the panels that it receives a selection from
Aaron Lun (13:59:44): > Tough.
Aaron Lun (14:00:52): > Do you actually see a use case where synchronized transfer of gene names matters? I don’t.
Aaron Lun (14:01:27): > If it bothers you so much, just have a checkbox button that switches between custom selection and row selections.
Aaron Lun (14:02:40): > But insertion of custom features is a must-have for a heatmap, I don’t see any other use case.
Kevin Rue-Albrecht (17:30:23): > I don’t have a use case for synchronised transfer, no.Thinking more about the « necessity » for custom feature list, I think it looks nice on a wish list , but my earlier thought was that if users have custom lists of features to visualise, as a heatmap or any other panel, then I don’t think they absolutely need to use iSEE for that. Otherwise where do we draw the line? Any row panel could then ask to support custom features.Let’s not forget that iSEE « core» is here to help interactive visualisation with linked selections between panels, not to solve every possible plot people can dream of. That’s the job of downstream panel developers and iSEEu
Aaron Lun (17:33:38): > Well, the heatmap is sort of useless without it.
Kevin Rue-Albrecht (17:33:42): > For me, the job of a built in heatmap panel is to play nice with other built in panels. A way to quickly visualise a selection of cells and genes (yes, columns and rows), selected in other panels.
Kevin Rue-Albrecht (17:35:50): > Well if I have a gene list, I do my own heatmap r script. Which is what iSEE users are likely to do, copy pasting from the code tracker after their first session
Aaron Lun (17:36:23): > The heatmap feature list is a key parameter. Like the gene choice in FeatAssayPlot.
Aaron Lun (17:36:46): > I don’t see how people are going to easily select the genes they want to show from transmitted row selections.
Kevin Rue-Albrecht (17:38:59): > sélections from plots are easy, we’re just stuck with tables because of single selectionsThough we could enable multiple selection and add a button « clear selection » maybe
Aaron Lun (17:40:33): > That’s not what I mean. Look, the use case is this. The end user has a gene list in a CSV file. They want to show thisin the app to see how their different samples express this gene list, possibly interacting with column selections.
Kevin Rue-Albrecht (17:40:52): > Also, do I remember well that the heatmap feature selectize widget grows ridiculously large, listing every single feature shown in the heatmap?
Aaron Lun (17:41:11): > Just shove it in an aceEditor in a modal somewhere.
Aaron Lun (17:42:04): > Anyway, continuing on from my point; we cannot rely on the true end users having any knowledge of how to use R.
Aaron Lun (17:42:17): > People who set up iSEE instances are rarely the true end-users of the app.
Kevin Rue-Albrecht (17:50:12): > I know. But then again, we can’t solve everyone’s problems with a single GUI panel. A hundred other expression browsers are here to prove that. Let’s finish building the core features of the new heatmap panel first. Then we can figure out how to plug in advanced features and get feedback from external users.
Kevin Rue-Albrecht (17:54:59): > I also suspect that the most recent heatmap related GH issue (shiny bookmarks) has to do with our selectize-modal-pObjects combo complicating session snapshots. I never got to the bottom of the issue, but that’s worth a thought too
Aaron Lun (18:35:02): > I would say that a custom gene listisa core feature.
Aaron Lun (18:35:31): > I do not see how a heatmap could be practically useful without it.
2020-01-02
Kevin Rue-Albrecht (05:41:57): > I know what you mean. I just want us to carefully think about features before rushing into implementations that require clunky hacks. > Heatmaps are by far the most complex/painful type of plot to make interactive. > Also, thinking about it (again) further, I’ll also point out that another key difference between heatmap and other builtin panels is that - in the absence of incoming selection - other panels show all possible data points. If heatmaps were to do that, they’d have to show the entire assay matrix. Which is clearly not realistic without some kind of downsampling. Which would have a whole different meaning for heatmaps if it’s even possible.
Kevin Rue-Albrecht (05:43:15): > Anyway. A bit of progress on the prototype this morning: row and column names can be toggled visible/not.
Kevin Rue-Albrecht (05:48:42): > Also, I’ve taken dev notes which turned into a nice checklist on how to add a new panel setting: from the creation of a slot to store the setting value to the UI, observers, and plot-generating code. > Seehttps://github.com/kevinrue/iSEE-book/blob/develop-advanced/06-add-panel-setting.Rmd
Kevin Rue-Albrecht (05:52:59): > Ultimately, a bit of reformatting could make it a nice little chapter for panel developers - starting with helping Fede and Charlotte catch up with the new implementation.
Kevin Rue-Albrecht (06:30:46): > Added a selectize to switch heatmap between assays. Shamelessly stolen/refactored fromFeatAssayPlot. > It is wayyyy too easy to add new settings now. Process: > * create slot > * create constant choices (optional) > * initialize default value > * create ui (helper functions are on the rise for that) > * create observers (helper functions exist, just give the setting name) > * update plotting code to use settings > … and … you’re done
Kevin Rue-Albrecht (06:33:54): > To add a selectize widget for choosing an assay, there is now.create_assay_selectize. Save yourselves a bit of coding but mostly just less thinking.
Kevin Rue-Albrecht (08:52:14): > colDatafields can be used as column annotations for the heatmap (they don’t use ECM colormaps yet)
Kevin Rue-Albrecht (09:01:18): > Same forrowData
Kevin Rue-Albrecht (09:03:13): - File (PNG): image.png
Aaron Lun (11:46:39): > Note that exposed functions should be camelCase, see.processMultiSelectionsand that function I converted to add selection box for examples.
Aaron Lun (11:47:08) (in thread): > Well, that’s why we have a custom selection to start users off with. That’s what we had before.
Kevin Rue-Albrecht (13:41:46): > Noted. If this point was about.create_assay_selectize, just forget it. Turns out the function was a wrapper around a one liner
Kevin Rue-Albrecht (13:42:35): > The code’s still ugly (i.e. prototype level) but I’ve restored coloring of column and row metadata respecting the ECM colormaps
2020-01-03
Kevin Rue-Albrecht (06:15:41): > OK. Refactored the processing of heatmap row and column annotation. Incidentally, I also generalized the functions that coerce dataframe columns to factor or numeric, so that they can be applied to any dataframe in the plot environment (heatmaps work with at least two dataframes to handle row and column annotations, none of which are calledplot.data). See.coerce_dataframe_columns
Kevin Rue-Albrecht (06:27:22): > Summary so far: > * Heatmap shows up only when receiving both row and column incoming selections (i.e., the panel always behaves with a “restrict” effect, because it’s not reasonable to show the full assay matrix in the absence of selection, in contrast to other builtin panels) > * There are two selectize inputs to add any number of row and column metadata as left and top annotations, respectively. The ExperimentColorMap object is used. > * There is a selectize to switch between available assays. (ExperimentColorMap object is not used yet.) > * There are checkboxes to show/hide column/row names. > * There are radiobuttons to switch the legend position (bottom/right) and direction (horizontal/vertical)
Kevin Rue-Albrecht (06:38:36): > I’ve been thinking about the behaviour in the absence of incoming selection, as discussed with Aaron. > I think that instead of selectize inputs (for both rows and columns) which would literally grow out of control in screen size with hundreds of items, the “Data parameters” box could simply have two buttons > * Edit Rows > * Edit Columns > Each of which would open an aceEditor populated with all possible row and column names - perhaps alphabetically sorted for convenient search - with those currently not visible being commented out. > Users could manually (un)comment ad libitum, and then click a button to apply the changes. > Separately, incoming selections could be applied with effect: > * add to heatmap = uncomment in aceEditor > * remove from heatmap = comment in aceEditor > * show in heatmap = set aceEditor, ignoring previous set of features. > In that framework, the heatmap would need to watch the aceEditor - showing the uncommented items - as it used to watch the selectize. > The benefit here is that the aceEditor will not take any screen estate when not used, and its size will be independent of the number of features in the heatmap (the biggest drawback of selectize widgets in this case)
Aaron Lun (17:11:50): > Yes, that’s what I’ve been saying.
Aaron Lun (17:12:19): > Each button would also have a checkbox above it, indicating whether you want to enslave it to the selection (in which case the custom stuff is ignored).
Aaron Lun (17:12:33): > Inside the modal, you could have “append current row selection” buttons.
Aaron Lun (17:12:45): > You only need to write the modal code once and recycle it for the rows/columns.
Aaron Lun (17:13:17): > Note that in this context, the heatmap’s memory will contain the literal text in the aceeditor, not the character vector like we currently have.
Kevin Rue-Albrecht (17:17:38): > :+1:good - i think we’re getting to a common view then. Agreed about recycling for row/column. > a couple of points though: > 1. A button in the modal is one way. I was more thinking about one of our patented “selection effect”: setting it to “Add” would append the selectionas it is brushed > 2. why the aceeditor text rather than the character vector of row/col names (or integer vector of indices)?
Aaron Lun (17:20:45): > (1) Sounds like it could get chaotic. You’d need to distinguish between features that are present because you appended an incoming selection and features that were already there, otherwise if I brushed multiple times, I’d just append loads of stuff. And people are guaranteed to brush multiple times even if they don’t want to do anything with that brush. It’s like how you click on text on a webpage even if there isn’t a link. You’d have to remember what you appended the last time in order to wipe it out when someone changes the brush.
Aaron Lun (17:21:24): > (2) Otherwise if someone closes the modal and opens it, the commented out features will be gone. I want to keep those because people might want to comment something out, have a look at the result, decide they liked it better the old way and then uncomment it. Given that the custom gene list is effectively manually curated, it would be a user win if they can do that instead of repeatedly copy-pasting from a file outside the app.
Kevin Rue-Albrecht (17:36:33): > (1): there would be a selection effect « do nothing » where users could brush ad libitum without any effect on the heatmap
Kevin Rue-Albrecht (17:41:40): > (2) maybe it got lost a bit in my earlier messages but I was thinking of listing ALL the features in the modal, commenting out the inactive ones. That means that if users paste a custom list of eg 10 (with and without comments) I’d complete the list with the rest of the existing features commented out
Kevin Rue-Albrecht (17:41:58): > Kind of a heavy handed selectize
Kevin Rue-Albrecht (17:42:34): > (Plus, aceEditors come with a search bar)
Kevin Rue-Albrecht (17:43:49): > Anyway, easier to just implement something that you can play with than talk about it in the air
Aaron Lun (17:45:45): > In general, if we are giving people an editor box, there shouldn’t be any unprompted changes to that box. Stuff disappearing, getting added or commented out quietly and automatically is pretty disconcerting. The modal should have plenty of space for explicit buttons to perform complex actions (e.g., “validate names”, which comments out names that aren’t present; “cluster names”, which rearranges them by clustering; “import names”, and so on).
Kevin Rue-Albrecht (17:46:14): > I see
Kevin Rue-Albrecht (17:48:16): > I’ll sleep on that and keep you posted when I have something. Flying back tomorrow and back to school on Monday.
2020-01-04
Kevin Rue-Albrecht (10:49:41): > Branchrefactor-heatmapwith: > > app <- iSEE(sce, initial = list( > ComplexHeatmapPlot(SelectRowSource="SampAssayPlot1", SelectColSource="FeatAssayPlot1", > Colnames=colnames(sce), ShowDimNames=c("Row")), > FeatAssayPlot(), SampAssayPlot())) > > Stopping here for now.
2020-01-05
Kevin Rue-Albrecht (17:44:44): > @Aaron Lundid you fancy trying out the modal lately?
Aaron Lun (17:46:53): > not yet
Aaron Lun (17:53:38): > Banging my head on other things
Aaron Lun (17:53:42): > OTL
Kevin Rue-Albrecht (17:55:21): > No rush. I’ll leave it at that for today and I want to have a think to consider next steps. Happy to hear thoughts or to see someone pick up the code if they have an idea. > Current state: > * the main UI has two buttons: “Edit rownames” and “Edit colnames” > * clicking those open a modal > * modal contains an aceEditor prepopulated with the current set of features in the heatmap (this is different from your suggestion to remember items commented out, at least for now). > * modal also contains a second aceEditorread-onlypopulated with the incoming selection, if any > * Users can edit the first aceEditor manually > * a button is available to add the incoming selection to the “main” aceEditor > * a button is available tosetthe incoming selection in the “main” aceEditor (i.e, replace everything by the incoming selection) > * a master button is there to apply the new set to the heatmap
2020-01-06
Aaron Lun (16:18:35): > second aceeditor spends a lot of real estate not doing anything. Why not just have a single ace editor and a button for mass import from a row selection? It’s pretty clunky to have to select things one at a time.
Aaron Lun (16:20:36): > Especially if I want to add stuff rather than “setting”.
Aaron Lun (16:26:20): > It seems to me the ideal approach would be to have a single aceeditor showing the current “workspace”; and then on the right, we have: > * Reorganize by cluster (clustering mode) > * Reorganize by sorting (alphanumeric) > * Validate names (comments out names that are not valid or duplicates). > * Import current selection (pulls in the current multiple selection) > Those buttons will update the aceeditor as required, and then you have another button to accept changes. This would not even need to use transient memory.
Kevin Rue-Albrecht (16:52:41): > Sounds good. I think I agree with everything, but as always, we’ll see what we think when we get to toy with the next version. Might not be before the weekend though.
2020-01-07
Aaron Lun (01:14:24): > You can see what I meant inrefactor-more-heatmap.
Aaron Lun (01:15:06): > jesus is your enter key broken or something?
Federico Marini (05:49:55): > come on Aaron we save on the message number:smile:
Aaron Lun (19:02:47): > nudge
2020-01-08
Kevin Rue-Albrecht (14:05:24): > Nudge received.
Kevin Rue-Albrecht (14:05:56): > I just had a look at therefactor-more-heatmap. Looks good.
Kevin Rue-Albrecht (14:07:10): > I definitely agree that the second “incoming” aceEditor was too much, and that a single button “Import selection” is enough.
Kevin Rue-Albrecht (14:07:54): > Looks like the observer needs to be updated though
Kevin Rue-Albrecht (14:09:05): > I don’t think the modal is the right place for a “Clustering” button. I’d rather have that in the “Visual parameters” box
Kevin Rue-Albrecht (14:11:59): > A few reasons for that last one: > * It is a “visual” parameter, not an “edit rownames” functionality > * It would be easier to toggle and visualize the result without having to open&close the modal every time > * I want to add the other clustering option of ComplexHeatmap: distance and clustering methods
Kevin Rue-Albrecht (14:13:36): > That said, I think the modal could have buttons or aselectInputto display names alphabetically or in the actual order of the heatmap.
Aaron Lun (14:13:37): > ¯*(ツ)*/¯
Aaron Lun (14:13:45): > Well it’s a data parameter moret han anything else.
Kevin Rue-Albrecht (14:14:28): > Clustering? Sounds more visual to me. Same data, different order.
Aaron Lun (14:14:48): > The whole app is visual.
Aaron Lun (14:14:59): > And order matters.
Kevin Rue-Albrecht (14:17:48): > Anyway. One thing that will also need attention are the observers. The heatmap rerenders a lot more than it should, right now. But I guess that will be handled as we put the new observers in place.
Kevin Rue-Albrecht (14:20:26): > Oh wait - another reason why I want “Clustering” outside of the modal - in fact as acheckbox- is that I want to be able to switch between the order of features as I gave them and the order clustered according to various parameters (hence the distance and method choices, asselectInput).
Kevin Rue-Albrecht (14:21:17): > In summary > * cluster:checkbox > * distance method:selectInput > * clustering method:selectInput
Aaron Lun (14:21:18): > Okay, but this all sounds like data parameters to me.
Kevin Rue-Albrecht (14:21:45): > Meh. Fine. As long as it’s possible.
Kevin Rue-Albrecht (14:22:52): > I just want to keep track of the order given in the modal somewhere (i.e., in thememory), and apply the clustering as a “visual” layer, without changing thememory.
Aaron Lun (14:23:22): > I don’t see why that isn’t compatible with having them in the data parameters.
Aaron Lun (14:23:29): > Or in fact why it isn’t compatible with what we have now.
Kevin Rue-Albrecht (14:26:01): > It’s not incompatible. But I saw the modal only as a tool to quickly paste a list of features without crowding the Data Parameters box, not as an extra box of options.
Kevin Rue-Albrecht (14:26:21): > I’d rather keep as much of the “visual” options in the Data, Visual and other boxes
Aaron Lun (14:26:26): > Yes, but you can still have the clustering parameters in the data parameters box.
Aaron Lun (14:26:39): > To me, that involves some computation and it’s not purely aesthetic.
Kevin Rue-Albrecht (14:26:42): > Yes - exactly, I’ve agreed with you on that one
Kevin Rue-Albrecht (14:27:12): > i.e., we’ve agreed that clustering options will go in the Data parameters box. Just not in the modal
Aaron Lun (14:27:35): > good
Kevin Rue-Albrecht (14:30:06): > so what’s the plan now, shall we already merge torefactor-heatmapor keep fixing the observers onmore-heatbefore we merge? That’s a lot of branches right now
Kevin Rue-Albrecht (14:30:28): > Also, to avoid conflicts and parallel work
Aaron Lun (14:30:33): > FINISH HIM
Aaron Lun (14:30:34): > :-1:
Aaron Lun (15:32:18): > that means merge it, BTW.
Kevin Rue-Albrecht (15:33:08): > Yup. I was literally just getting at it now. Had dinner
Kevin Rue-Albrecht (15:38:33): > Alright. Done (merged)
Kevin Rue-Albrecht (15:39:04): > I just finally took the time to read your comments on the PR. Would have saved us a bit of conversation earlier. My bad.
Kevin Rue-Albrecht (15:39:53): > set the channel topic: Interactive SummarizedExperiment Explorer
Kevin Rue-Albrecht (15:40:29): > Interesting. Topic and Description are different things for a channel.
Kevin Rue-Albrecht (16:14:20): > “Import selection” button back online.
Kevin Rue-Albrecht (16:28:34): > Philosophical question: clustering checkbox,TRUEorFALSEdefault setting?
Aaron Lun (16:28:58): > Hm.
Kevin Rue-Albrecht (16:28:58): > I lean toward FALSE. User data first.
Aaron Lun (16:29:04): > k
Kevin Rue-Albrecht (16:31:14): > I gotta say, the format might change, but chapter06-add-panel-setting.Rmddefinitely needs to feature in the iSEE-book one way or another. I keep using it as a roadmap to remember all the essentials.
Kevin Rue-Albrecht (16:32:55): > That said, I’m aware that it still misses a few things that I haven’t taught myself properly either. In particular guideline forsetValidity2.
Kevin Rue-Albrecht (16:37:39): > Clustering of rows is now controlled by a checkbox.
Kevin Rue-Albrecht (16:40:32): > Argh. Does it still make sense if clustering of samples isTRUEby default?
Kevin Rue-Albrecht (16:41:03): > I guess, given that features and samples do not follow the same rules for this panel anyway
Aaron Lun (16:42:01): > The previous default was to order samples by the selected coldata variables.
Kevin Rue-Albrecht (16:43:25): > And if there are none? Do you remember?
Aaron Lun (16:43:49): > Just used whatever the input order was.
Aaron Lun (16:43:51): > Which is fine.
Aaron Lun (16:43:56): > Sometimes that makes sense.
Aaron Lun (16:44:13): > You can even have an “orderBy” radio button, “None”, “ColData”, “Cluster”.
Kevin Rue-Albrecht (16:45:56): > OK. Well that sounds like a combo job for another day. I’ll focus on polishing up observers for what’s already there for tonight
Kevin Rue-Albrecht (17:21:51): > Different clustering distances can be selected for features
Kevin Rue-Albrecht (17:38:10): > Different clustering methods can be selected for features
Kevin Rue-Albrecht (17:39:32): > ShouldselectInputs for distance and method be hidden when clustering is unticked/inactive ?
Kevin Rue-Albrecht (17:40:04): > Same question for the “Edit rownames” button: should it be hidden when “use custom features” is unticked?
Aaron Lun (17:40:05): > Probably, yes.
Aaron Lun (17:40:29): > Wherever we can, we should hide shit that isn’t relevant.
Kevin Rue-Albrecht (17:45:28): > I was also thinking that it makes sense, and simplifies not worrying about observers rerendering a plot when an irrelevant input is changed (e.g., clustering method when clustering is off). > That said, a minor drawback there is that users need to turn on clustering with whatever values the hidden parameters had, before they can actually change those values, causing a rerendering for each individual change (turning on the clustering, changing each parameter). > Leaving those conditional parameters visible would allow users to set clustering methods and distance before turning on the clustering, thereby rendering the whole thing only once at the end. > > Having said that, I agree that it’s just a whole easier for us and less confusing for users to hide things until they’re meant to be used.
Aaron Lun (17:46:01): > I seriously doubt they’re going to fiddle with these much. If we have sensible defaultsit hosuld be oaka
Aaron Lun (17:46:04): > okay.
Kevin Rue-Albrecht (17:49:23): > Anyway. I’ll leave it here for tonight, you can play a bit with it and refine where need be (often faster than telling me). I’ll have another look at what I can polish tomorrow.
Aaron Lun (22:55:47): > Looks good, but thiere’s a double-rendering at some point that creates a plot on the graphics device. Probably because the draw commands are executed by.generateOutput. Suggest dumping everything used in thedrawinto some value in the output of.generateOutputand then retrieving it in.renderOutputto actually do the drawing.
2020-01-09
Kevin Rue-Albrecht (17:12:10): > Good point.
Kevin Rue-Albrecht (17:12:41): > I’ve done prettier fixes, but have a look at my commit, and see if you can think of a better way to polish it
Aaron Lun (17:13:13): > how ugly can it be? Just save the args as a list and then do.call(draw, args) on it.
Kevin Rue-Albrecht (17:14:22): > Right. I never think of that syntax.
Kevin Rue-Albrecht (17:27:17): > Hmpf.. not sure which syntax I prefer. I don’t know if it’s just an impression butdo.callmight render a bit faster
2020-01-11
Kevin Rue-Albrecht (11:05:19): > Hey team. > I was thinking that it would be nice to group all iSEE packages in one place using a GitHub organization:https://github.com/isee-developersIdeally, that would mean transfering ownership of all our repositories to that organization account (to maintain repo URLs mentioned in the f1000 article, we could always fork it back to our respective accounts).
Kevin Rue-Albrecht (11:09:55): > In any case, the overall idea being to facilitate discovery (and maintenance) of the various iSEE-related packages.
Kevin Rue-Albrecht (11:10:54): > Re: maintenance, an organization allows multiple admins, among other things, useful for settings in particular.
Kevin Rue-Albrecht (11:26:50): > Relevant reading: > * https://help.github.com/en/github/setting-up-and-managing-organizations-and-teams/managing-team-access-to-an-organization-repository#giving-a-team-access-to-a-repository > * https://help.github.com/en/github/setting-up-and-managing-organizations-and-teams/repository-permission-levels-for-an-organization
Kevin Rue-Albrecht (11:52:56): > Anyway, there’s no rush. Happy to hear thoughts. > In the meantime, I’ve moved the repos that were under my account to the organization and forked them back to my account for safekeeping (iSEE_custom, iSEEWorkshop2019, isee-shiny-contest). It’s easy to revert if we end up rejecting the idea of an organization.
Charlotte Soneson (14:30:36): > I like the idea of an organization, which has iSEE stuff and nothing but iSEE stuff:slightly_smiling_face:
Kevin Rue-Albrecht (14:42:30): > Fresh from the oven: Proof of concept that a bookdown can be automatically built on Travis and deployed on GitHub Pages:https://github.com/isee-developers/bookdown-demo
Kevin Rue-Albrecht (14:43:19): > (as long as it fits in the Travis time limit for jobs)
Federico Marini (17:16:15): > Also liking the orga idea
Federico Marini (17:16:25): > Makes a little more “universe” thing
Federico Marini (17:16:27): > :slightly_smiling_face:
Kevin Rue-Albrecht (18:09:49): > Thanks for the feedback Charlote and Fede. Curious to hear Aaron’s thoughts, but then we could start moving more things to the organization.
Kevin Rue-Albrecht (18:11:13): > Another little perk is that organizations allow the creation of cross-repositories “Projects”. I’ve started one athttps://github.com/orgs/isee-developers/projects/1to track remaining things needed for the transition to iSEE 2.0.
Kevin Rue-Albrecht (18:13:28): > At the moment, I’ve only listed a few Heatmap-related fixes that are still needed, but it’ll also be useful to track changes in downstream packages.
Kevin Rue-Albrecht (18:15:42): > Transferring packages from my account, I realized thatiSEEuwill basically deprecateiSEE_custom. I guess we’ll just archive that one when the time comes.
Aaron Lun (18:44:15): > Do it.
Aaron Lun (18:44:22): > Why not just iSEE/iSEE.
Aaron Lun (18:44:34): > e.g.https://github.com/RcppCore
Aaron Lun (21:36:03): > or iSEEall
Aaron Lun (21:36:11): > iSEEeverything
Aaron Lun (22:08:56): > Finish the heatmap, merge the PR, this other stuff can wait.
Aaron Lun (22:16:32): > dammit. Should have really bought some painkillers when I was at target. These cough drops aren’t going to cut it.
2020-01-12
Charlotte Soneson (03:15:48): > Ok,csoneson/iSEEandcsoneson/iSEEuare transferred. I don’t think we should need to fork them back, the original repo is redirected to the new one (and we don’t have to bother keeping the actual repo and the fork synced).
Federico Marini (03:24:58): > iSEEdata would be also nice
Kevin Rue-Albrecht (04:02:26) (in thread): > I wanted to, but you’ll see that there’s an account called « isee » inactive since 2011 with a single repo. I’ve messaged github about the possibility of removing this kind of old useless accounts. Waiting on an answer
Kevin Rue-Albrecht (04:03:40): > It’s possible to rename the organisation. Let’s see if we can settle on a name, given the various ideas above
Federico Marini (04:29:06): > btw, the orga thing is quite handy
Federico Marini (04:29:09): > good catch:wink:
Kevin Rue-Albrecht (04:41:44): > I’ve switched all the isee-developers to “Maintain” for the isee-core team (https://github.com/orgs/isee-developers/teams/isee-core/repositories). > It’s one level below “Admin”, but we should all be able to change that setting on a per-repository basis.
Kevin Rue-Albrecht (04:47:29): > Aaron I’ll get to the heatmap when I can volunteer some again, I’ll keep you posted. I have a plan for all items inhttps://github.com/orgs/isee-developers/projects/1except one: I don’t know if ComplexHeatmap can support transparency for side annotations, like ggplot2 did.
Kevin Rue-Albrecht (04:48:40): > Fede and Charlotte, it’d be helpful if you could start looking at the vignettes, to see what needs updating in 2.0. We need to divide and conquer the remaining tasks if we want to make it for the April release.
Kevin Rue-Albrecht (04:51:49): > Given the sporadic availability that we all have, I’m tempted to track the remaining items and progress using issues and/orhttps://github.com/orgs/isee-developers/projects/1, or do you guys find it “too much” ?
Kevin Rue-Albrecht (13:28:58): > “Edit rownames” doesn’t fit with our “features x samples” naming. Are there any favorite between “Edit features”, “Edit feature names”, “Edit plot features”, “Edit custom features” … ? - File (PNG): image.png
Aaron Lun (13:43:23): > feature names.
Kevin Rue-Albrecht (17:07:34): > I’ve just implemented the coloring effect of incoming column selections in the heatmap. I ticked off all the to-do list that I came up at the start of the weekend.
Kevin Rue-Albrecht (17:09:08): > The only thing I can think of - from the current iSEE - is that columns should be sorted by selected ColData, if any (otherwise, take the asce order as is).
Kevin Rue-Albrecht (17:39:01): > alright - I’m just finishing off this sorting and pushing before going to sleep
Kevin Rue-Albrecht (17:40:12): > It just leaves the transparency effect, which I think we can’t reimplement in ComplexHeatmap.
Aaron Lun (17:43:28): > Screw it.
Kevin Rue-Albrecht (17:44:47): > Philosophically, I was also wondering whether mixing alpha and color was an efficient mix of information to display
Kevin Rue-Albrecht (17:45:26): > Anyway
Kevin Rue-Albrecht (17:46:19): > At the moment, complexheatmap is using.create_dotplot_selection_param_box, so I guess that has to be overidden to delete that “transparency” option for complexheatmap, right?
Kevin Rue-Albrecht (17:47:03): > Eh I don’t mean overridden, I mean a.create_heatmap_selection_param_boxneeds to be implemented instead, right?
Kevin Rue-Albrecht (17:48:50): > In any case, I’ll leave that up to you. Shouldn’t take long, and you’ve got a clearer idea than me what you want this panel to look like.
Kevin Rue-Albrecht (17:49:37): > At some point we should also trash the oldheatmap.Rfile and unit tests. They’re the only thing breaking Travis lately.
Aaron Lun (18:04:01): > That’s correct.
Aaron Lun (18:04:26): > create_heatmap_selection_param_box can be written off create_selection_param_box by oputting stuff in the ellipsis.
2020-01-13
Kevin Rue-Albrecht (09:13:51): > The DockerHub organizationiseedevelopers(building the Docker image of iSEE) is now linked to the newisee-botaccount. > This separates the DockerHub from my personalkevinrueaccount. (Until now, DockerHub required access to all my GitHub repositories - or in the case ofcsoneson/iSEEall of Charlotte’s repositories ) - File (PNG): image.png
Kevin Rue-Albrecht (09:15:48): - File (PNG): image.png
Kevin Rue-Albrecht (09:29:19): > For the record, I followed the instructions here:https://docs.docker.com/docker-hub/builds/#autobuild-for-teams - Attachment (Docker Documentation): Set up automated builds > How Automated Builds work Docker Hub can automatically build images from source code in an external repository and automatically push the built image to your Docker repositories. When you set…
2020-01-14
Kevin Rue-Albrecht (04:41:47): > Market study for@Charlotte Sonesonand@Federico Marini: can you guys foresee some time between now and March/April to update the vignettes and proof-read the documentation, while Aaron and I finish up the code side? > I can start seeing the finish line of the ComplexHeatmap panel on the horizon, but time is scarce. > From what I can see, the vignettes do not run any heatmap-related code, so technically you don’t need to wait for the finished new heatmap panel. > The only mention of heatmap is about the selectize configuration, but any other panel could be used for that example:https://github.com/isee-developers/iSEE/blob/master/vignettes/configure.Rmd#L452Updated to-do list for 2.0:https://github.com/orgs/isee-developers/projects/1
Federico Marini (06:17:03): > Likely. I am trying to close up open end, especially before April, so it would make sense to put in some efforts in this direction. Sorry again for missing out the refactoring “hands-on” phase
Kevin Rue-Albrecht (06:22:08): > To be honest the code refactoring was messy enough with only 2 pairs of hands. What I’m more worried about are glitches in the user experience: bugs, loading time, documentation/usability/discoverability of key information, superfluous/unnecessary rerenderings, etc.
Kevin Rue-Albrecht (06:24:50): > It’s actually a good thing to have a couple of usnotbeing involved in the code refactoring: this gives you a user perspective on things. If you struggle to understand something, then external users definitely won’t, and that means the feature will need more/better documentation
Federico Marini (06:53:46): > Ok, I’ll pretend I have stayed out of iton purpose :smile:
Federico Marini (06:54:07): > joke apart: I am happy to involve some new persons in give a check at the whole documentation
Federico Marini (06:54:23): > i.e., some new iSEE users we “convert” or win to our side
Charlotte Soneson (07:55:37): > Yeah, I should also have time to do this.
Kevin Rue-Albrecht (13:13:53): > yolooooo I snatched the “isee” account away from that user inactive since 2011
Kevin Rue-Albrecht (13:39:39): > before anyone calls the cops: I “snatched” the accountlegally :wink:Message from GitHub support below - File (PNG): image.png
Kevin Rue-Albrecht (13:39:52): > update your remotes accordingly:slightly_smiling_face:
Federico Marini (14:39:14): > great move:slightly_smiling_face:
2020-01-15
Aaron Lun (11:43:55): > NEED ISEE GIFS STAT
Kevin Rue-Albrecht (12:08:34): > Eh… same thing with more words and less capitalization please?
Aaron Lun (12:10:54): > Never mind, I found Fed’s youtube video.
Aaron Lun (12:11:06): > Got ambushed with a meeting today so I’m just going to talk about iSEE.
Kevin Rue-Albrecht (12:22:33): > While you’re around, I had a question: your “Organize by order” button in the modal… was that supposed to say “Order alphabetically”, or something else?
Kevin Rue-Albrecht (12:23:07): > I’m not a native speaker, but still, that’s a bit cryptic to me.
Aaron Lun (12:23:25): > probably
Kevin Rue-Albrecht (12:23:56): > Well you put that button there…:sweat_smile:
Aaron Lun (12:24:53): > Hey that’s past aaron
Kevin Rue-Albrecht (12:27:20): > Actually, similar question for “Validate name”. Originally you said something about commenting out invalid names. Not sure if we’re still on that page, or perhaps discarding invalid names altogether. > I’m just thinking about what is supposed to happen when users click: “validate names” then “apply”, in that order.
Kevin Rue-Albrecht (12:27:44): > I guess putting#should be enough to make invalid names truly invalid, names which would then be ignored and discarded when “Apply” is clicked. Sounds good?
Aaron Lun (12:29:17): > There’s no need to discard them explicitly. Just leave them in and let the heatmap rendering code take care of removing anything with a hash in front.
Kevin Rue-Albrecht (12:30:21): > Good. That’s exactly what I meant with my last message.
Aaron Lun (12:35:59): > k
Aaron Lun (12:36:22): > I can’t think of any case where a gene would actually have a hash in it as part of its name, so let’s not worry about that for now.
Aaron Lun (12:38:35): > where’s the book live now?
Kevin Rue-Albrecht (12:40:42): > Still under my account. I didn’t want to confuse users with something that talks about a version of iSEE that isn’t available yet.
Aaron Lun (12:40:55): > remind me of the link, I’m going to put it in my slides
Kevin Rue-Albrecht (12:41:43): > Repo:https://github.com/kevinrue/iSEE-bookBook:https://kevinrue.github.io/iSEE-book/ - Attachment (kevinrue.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Kevin Rue-Albrecht (12:50:41): > Any idea why the heatmap is rerendered even without clicking “Apply”, i.e. when typing in the aceEditor ? > I can’t find the observer that does that
Kevin Rue-Albrecht (12:51:11): > Anyway. I’ll push stuff later this evening. I’ve dealt with most of the remaining heatmap-related critical to-do’s
Aaron Lun (12:51:35): > I think the heatmap rendering itself might use the contents of the ace editor rather than pulling it out from the memory.
Aaron Lun (12:51:57): > Can’t entirely remember.
Kevin Rue-Albrecht (12:52:43): > OK. We’ll figure that one out later.
Kevin Rue-Albrecht (13:39:29): > When do you have your talk? Planning a live demo too?
Aaron Lun (13:39:52): > This afternoon.
Kevin Rue-Albrecht (13:41:31): > Ok then I should be able to clean up and push the latest changes by then if you fancy a live demo at the bleeding edge
Aaron Lun (13:42:29): > No, I’m not going to do that.
Federico Marini (14:16:29): > can we call the iSEE bot AI-see to follow the newest wave?
Federico Marini (14:16:30): > :smile:
Kevin Rue-Albrecht (14:16:46): > :face_palm_star_trek:
Kevin Rue-Albrecht (16:49:05): > Oh boy - to cover all bases, we need to precompute which assays are continuous/discrete, so that we can apply the right type of colormap to the main matrix:sob:
Aaron Lun (16:49:34): > Should be a case of just taking the [1, 1] value and checking it.
Aaron Lun (16:49:49): > They’re matrices, after all.
Kevin Rue-Albrecht (16:49:50): > couldstorage.modeon the matrix work too?
Aaron Lun (16:50:00): > Not guaranteed if it’s not an atomic matrix.
Aaron Lun (16:50:13): > e.g., HDF5Matrix or sparse matricx.
Kevin Rue-Albrecht (16:50:26): > thought there might be a case like that:sleepy:
Aaron Lun (16:50:45): > just chuck it in cacheCommonInfo with the other checks.
Kevin Rue-Albrecht (16:51:08): > that part I figured, t’was the [1,1] that I hadn’t got to yet
Kevin Rue-Albrecht (16:51:54): > That all intensifies my love/hate relationship to ComplexHeatmap
Kevin Rue-Albrecht (16:52:31): > So powerful, but so much simpler to hack in an R script than a GUI
Kevin Rue-Albrecht (16:53:45): > Wait until we allow scaling by gene.. we’ll need a different set of colormaps then.
Aaron Lun (16:53:57): > why?
Kevin Rue-Albrecht (16:54:01): > divergent colors
Aaron Lun (16:54:16): > huh?
Kevin Rue-Albrecht (16:54:29): > e.g.:
Kevin Rue-Albrecht (16:55:08): > you’ve got logcounts, nicely ranging [0, +oo)
Kevin Rue-Albrecht (16:55:30): > you’ve made a nice 2-color colormap, say from white to red, i.e. “sequential”
Kevin Rue-Albrecht (16:56:31): > now, you tick the box “scale rows”, you’ve got a distribution that’s centered and scaled, so a 3-color scale (i.e., “divergent”) would be more appropriate
Kevin Rue-Albrecht (16:59:06): > The way I see it, ExperimentColorMap needs an additional argument likediscrete=FALSE, for instancescaled=TRUE, which would switch between the two types (sequential, divergent) of colormaps
Kevin Rue-Albrecht (16:59:28): > I’d say that’s a post-April-release feature.
Kevin Rue-Albrecht (17:50:58): > :sob:I just realized that the matrix is not ordered according to the column metadata, only the metadata itself
Kevin Rue-Albrecht (17:51:15): > I’ll fix that and go to sleep. No work on the assay colormap tonight
Kevin Rue-Albrecht (18:12:06): > Ugh. Done.
Kevin Rue-Albrecht (18:25:51): > It looks like a lot of thosecmds <- c(cmds, "print(1:10)")should be replaced by our command store system. I spotted some.text_eval(cmd, envir)which seem to be reevaluating the same code multiple times. Or more precisely, it’s called with the first few commands collected, then it’s called again later with all the commands collected.
Aaron Lun (19:11:02): > crushed the presentation
Aaron Lun (19:57:30): > I think we should make everythign classier
Aaron Lun (19:57:36): > by translating the panel names into french
Aaron Lun (19:57:45): > “le tableau de genomique”
Aaron Lun (20:05:20): > > tracé de dimension réduite >
2020-01-16
Federico Marini (04:07:01): > Ouiiiii+
Federico Marini (04:07:15): > Le nouvelle feature de lacarte de chaleur
Federico Marini (04:08:31): > Get your script from thetraquer de code logiciel:smile:
Federico Marini (04:09:10): > You can paste it in yourR marque-vers-le-bas document
Kevin Rue-Albrecht (06:02:51) (in thread): > traqueur de code:wink:
Kevin Rue-Albrecht (06:03:28): > I’ll do the translation if someone picks up the coding:stuck_out_tongue:
Federico Marini (09:23:00): > Funny thing to re-discover
Federico Marini (09:23:14): > gource, with the latest changes
Kevin Rue-Albrecht (15:48:54): > the book’s moved to the iSEE org. The address is now:https://isee.github.io/iSEE-book/ - Attachment (isee.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Aaron Lun (15:49:48): > What happened to my tableau des ensemble de gènes
Kevin Rue-Albrecht (16:25:14): > just pushed the switching between discrete and continuous colormaps for the assay
Kevin Rue-Albrecht (16:25:40): > Best enjoyed with > > assay(sce, "letters") <- matrix(sample(letters[1:3], prod(dim(sce)), TRUE), nrow = nrow(sce), ncol = ncol(sce)) >
Aaron Lun (16:26:14): > I don’tt hink this is ever going gto be used.
Aaron Lun (16:26:21): > Maybe SNPs. Maybe.
Kevin Rue-Albrecht (16:26:26): > or > > assay(sce, "letters") <- matrix(sample(c("0|0", "0|1", "1|1"), prod(dim(sce)), TRUE), nrow = nrow(sce), ncol = ncol(sce)) > > for the purists
Kevin Rue-Albrecht (16:26:40): > stop reading my mind, it’s embarassing
Kevin Rue-Albrecht (16:33:49): > Anything keeping us from throwing the old heatmap code out?
Kevin Rue-Albrecht (16:34:25): > it’d be nice to have Travis pass again, to see real issues, and where we stand in terms of code coverage
Kevin Rue-Albrecht (16:47:18): > hmpf… I was happily deleting files and stuff, until I ran intoheatMapPlotDefaults(sce, n=1)indefaults.Rwhich reminded me that we need to follow the deprecation cycle for the heatmap too…:sleepy:
Aaron Lun (18:17:04): > Yes.
2020-01-18
Aaron Lun (14:27:33): > ETA
Kevin Rue-Albrecht (16:14:01): > Depends what you’re asking about. > I’m hoping tomorrow for the deprecation code.
Kevin Rue-Albrecht (16:20:55): > I’ve got a feeling that it would take you a fraction of the time that it would take me. I only realised today that you’ve also added code to convert the old DataFrames to the new classes - so ComplexHeatmapPlot needs to be implemented there too for the deprecation of the old heatmap
Kevin Rue-Albrecht (16:23:38): > Anyway if it’s still to do tomorrow I’ll try. Once it’s done (meaning unit tests should pass again), i guess the next logical step would be to see what code needs coverage and documenting the new internal functions.
Aaron Lun (22:39:44): > I have a paper to review tomorrow, so it’s all yours.
2020-01-19
Kevin Rue-Albrecht (13:43:45): > Finally about to pass R CMD check locally again:tada:
Kevin Rue-Albrecht (13:48:44): > I’ve been thinking about the centering and scaling features that we had in the old heatmap panel (seehttp://shiny.imbei.uni-mainz.de:3838/iSEE/#if you need a reminder). > I’m not in a rush to reimplement this, as it finally reminded me of the GH issue about log-transformation of XY axes on the fly; users could actually precompute themselves row-scaled values for each feature. That would then allow them to explicitly define an ECM colormap specifically for that scaled assay.
Kevin Rue-Albrecht (13:50:56): > Only caveat is that the scaled/centered assay would be computed overallsamples, even if the interactive heatmap is restricted to a subset of samples. Depends how frequent the use case is. One of those things that is probably better left aside until users explicitly expose a relevant use case/issue.
Aaron Lun (14:05:48): > I don’t know anyone with the foresight to center before they use the app.
Aaron Lun (14:06:33): > And in fact, having pondered the entire issue a bit more, we should probably add a log x / log y button to the colData/rowData/featAssay/sampAssay plots.
Kevin Rue-Albrecht (14:06:40): > Well, hopefully hindsight will remind them when the assay isn’t available
Kevin Rue-Albrecht (14:07:06): > log(x… +1?)
Aaron Lun (14:07:18): > Well, that’s the thing. It’s a common use case that is going to be a pain point, so we might as well do it for them.
Aaron Lun (14:07:30): > The log would be justscale_x_log10. Nothing more than that.
Aaron Lun (14:08:07): > In any case, centering/scaling was present in the old world, so it should also be present in the new world.
Kevin Rue-Albrecht (14:08:09): > > The log would be justscale_x_log10. Nothing more than that. > Just to point out that this one’s the equivalent oflog10(x), which will generate NAs for 0s
Aaron Lun (14:08:53): > That’s fine. We did our job.
Kevin Rue-Albrecht (14:09:34): > > Well, that’s the thing. It’s a common use case that is going to be a pain point, so we might as well do it for them. > Just to point out that this just creates an additional complexity when the heatmap is restricted to a subet of samples. Users would then probably like the choice of centering/scaling either on the visible samples, or all samples (including those not visible)
Aaron Lun (14:10:07): > What’s the use case for the latter? SEems a bit werid
Aaron Lun (14:10:42): > I would say the latter defeats the purpose of doing centering/scaling.
Kevin Rue-Albrecht (14:11:17): > OK. I was wondering if values should change for the same feature, depending on which samples are shown
Kevin Rue-Albrecht (14:11:40): > None of the other panel do that kind of thing
Aaron Lun (14:11:48): > yes, that’s fine, I think you can think of centering/scaling as a visual effect.
Kevin Rue-Albrecht (14:12:34): > Different subject: any idea why magick wouldn’t be installed on Travis if it’s a “Suggest” dependency ofknitr? > > --- re-building 'basic.Rmd' using rmarkdown > Quitting from lines 59-60 (basic.Rmd) > Error: processing vignette 'basic.Rmd' failed with diagnostics: > there is no package called 'magick' > > Seehttps://travis-ci.org/iSEE/iSEE/builds/639198923#L1420https://github.com/yihui/knitr/blob/master/DESCRIPTION#L128 - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Aaron Lun (14:13:13): > this just happened a few days ago.
Aaron Lun (14:13:28): > I also had problems installing magick, it requires system libs
Aaron Lun (14:13:38): > have to break out the ol’sudo apt-get.
Kevin Rue-Albrecht (14:13:46): > Git blame says 11 months ago:https://github.com/yihui/knitr/blame/cce76d9a9afe6ed1e060feba507b82531bd0f9e3/DESCRIPTION#L128
Aaron Lun (14:14:11): > Well, magick itself must have switched to requiring system libs that it didn’t previously use.
Kevin Rue-Albrecht (14:14:35): > I might just try the workaround of transiently addingmagickto our own “Suggests” DESCRIPTION. Thoughts?
Kevin Rue-Albrecht (14:14:57): > i.e., just to get it installed in the Travis package cache
Kevin Rue-Albrecht (14:15:27): > whether it works or not, I can remove it from our DESCRIPTION in the next commit
Aaron Lun (14:15:54): > Don’t think that’ll work, but you can try.
Kevin Rue-Albrecht (14:19:39): > Ohh right > > begin installing package 'knitr' > * installing **source** package 'knitr' ... > **** package 'knitr' successfully unpacked and MD5 sums checked > **** using staged installation > **** R > **** demo > **** inst > **** tests > **** byte-compile and prepare package for lazy loading > **** help > ***** installing help indices > **** building package indices > **** installing vignettes > **** testing if installed package can be loaded from temporary location > **** testing if installed package can be loaded from final location > **** testing if installed package keeps a record of temporary installation path > * DONE (knitr) > begin installing package 'magick' > Makefile:15: recipe for target 'magick.ts' failed > make: ***** [magick.ts] Error 1 >
Aaron Lun (14:20:20): > Basically, whoever is managing that docker image needs to include the magick system libs.
Aaron Lun (14:20:40): > or however travis does it, I don’t know.
Kevin Rue-Albrecht (14:20:50): > Damn. And here I thought we could finally use Travis as a shared TODO list
Kevin Rue-Albrecht (14:21:52): > Basically,R CMD checkpassed on my laptop, but now there are the remaining documentation bits and unit tests that Travis can help us track down in a central way
Kevin Rue-Albrecht (14:23:01): > > have to break out the ol’sudo apt-get. > Does that mean you’ve faced and solved the same issue on another repo?
Aaron Lun (14:24:24): > Locally. I don’t build on travis.
Aaron Lun (14:24:57): > File an issue with whoever manages the base R travis setup.
Aaron Lun (14:25:04): > I don’t know who this is.
Kevin Rue-Albrecht (14:25:13): > Jim Hester, no?
Aaron Lun (14:25:43): > ¯*(ツ)*/¯
Aaron Lun (14:25:45): > If you say so
Kevin Rue-Albrecht (14:33:20): > trying this out:https://travis-ci.community/t/error-configuration-failed-for-package-magick/1674/2 - Attachment (Travis CI Community): ERROR: configuration failed for package ‘magick’ > Did you try the magick ropensci Travis.yml addons: apt: update: true sources: - sourceline: ‘ppa:opencpu/imagemagick’ - sourceline: ‘ppa:ubuntugis/ppa’ packages: - libmagick++-dev - librsvg2-dev - libwebp-dev - libpoppler-cpp-dev - libtesseract-dev - libleptonica-dev - tesseract-ocr-eng - r-cran-rgdal - libfftw3-dev - cargo For me this did the trick
Federico Marini (14:40:13): > There’S something going on on the knitr side
Federico Marini (14:40:21): > Herve already opened a couple of issues
Federico Marini (14:40:53): > https://github.com/yihui/knitr/issues/1797
Federico Marini (14:41:19): > Feel free to:thumbsup:it
Kevin Rue-Albrecht (14:43:36): > Done. Thanks
Kevin Rue-Albrecht (14:44:58): > Ugh. Same issue after the Travis CI fix above:https://travis-ci.org/iSEE/iSEE/builds/639210859?utm_medium=notification&utm_source=github_status - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (14:52:33): > It’s not pretty … but the combo of the Travis fix aboveandforcing the installation ofmagick(adding it to our DESCRIPTION) got it through the installation ofmagick, but not it fails with a missing file error > > [...] > * DONE (magick) > [...] > > --- re-building 'basic.Rmd' using rmarkdown > Quitting from lines 59-60 (basic.Rmd) > Error: processing vignette 'basic.Rmd' failed with diagnostics: > Magick: unable to open image `/tmp/RtmpkuPa81/Rbuild407a3ca423c0/iSEE/vignettes/basic_files/figure-html/unnamed-chunk-1-1.png': No such file or directory @ error/blob.c/OpenBlob/2874 > --- failed re-building 'basic.Rmd' > > https://travis-ci.org/iSEE/iSEE/builds/639213616?utm_source=github_status&utm_medium=notification{EDIT} I just realized that this is similar to the issue reported by Hervé. Hurray.
Kevin Rue-Albrecht (14:58:07): > Well… that’s all folks for today. Out of steam, and it won’t hurt to see if knitr gets fixed in the next few days.
Kevin Rue-Albrecht (14:58:47): > I imagine that a fair number of packages are affected byknitr::include_graphicsbugs
2020-01-20
Mike Smith (07:04:56): > @Mike Smith has left the channel
2020-01-21
Aaron Lun (02:42:08): > YES! Another review down. 3 for 3 this long weekend.
Aaron Lun (11:47:22): > KEEP PUSHING.
Federico Marini (12:19:59): - File (PNG): image.png
Federico Marini (12:21:20): > This means one thing and one thing only:
Federico Marini (12:21:51): > iSEE 2.0 will get metallic look holo stickers
Aaron Lun (12:27:08): > OH YEAH
Kevin Rue-Albrecht (13:04:27): > :rocket:
Federico Marini (13:05:37): > 50 for 19 bucks is quite good
2020-01-22
Federico Marini (06:45:50): - File (PNG): image.png
Federico Marini (06:46:05): > Something does not figure here, but hey:smile:
Kevin Rue-Albrecht (06:53:41): > I also love their page:https://www.stickermule.com/uk/products/holographic-stickersMoving the mouse of the photo of stickers previews the effect - Attachment (Sticker Mule): Holographic stickers | Free shipping > Custom holographic stickers are an eye-catching way to promote your business or event. Holographic vinyl has a rainbow effect which changes with light & perspective.
2020-01-23
Kevin Rue-Albrecht (03:04:25): > Why feature comparison always have to be so negative about other software? So many “No” in that table.https://academic.oup.com/nar/advance-article/doi/10.1093/nar/gkz1209/5709708
Aaron Lun (03:05:25): > Otherwise it doesn’t get published, obviously.
Aaron Lun (03:05:28): > I saw that one too.
Aaron Lun (03:06:00): > If you want to publish something, you first have to put everyone else down. You should know that by now.
Aaron Lun (03:07:09): > Some people say that we stand on the shoulders of giants. But actually, I like to think of it as climbing a mountain of corpses. And dragging down and/or clambering over your fellow climbers along the way.
Kevin Rue-Albrecht (03:08:57): > I know all that. Am I naive for keeping hoping that things could get better? …:sweat:
Aaron Lun (03:13:16): > I think my mindset is more fun.
Aaron Lun (03:13:26): > Why do you think I review so many papers?
Kevin Rue-Albrecht (03:14:28): > ? - File (PNG): image.png
Federico Marini (03:29:06): > On my side of the world, I tried to go against that trend
Federico Marini (03:29:33): > I hate it when people “invent” features or call things as such just to have an extra green tick on their tool
Federico Marini (03:30:09): > This is my suppl table forideal
Federico Marini (03:30:09): > https://federicomarini.github.io/ideal_supplement/
Federico Marini (03:30:51): > They’ll be coming:smile: - File (PNG): image.png
2020-01-24
Aaron Lun (03:45:34): > FINISH HIM
Aaron Lun (03:45:36): > Going to sleep.
Federico Marini (04:21:47): > It’s ordered:wink:
Aaron Lun (11:19:12): > Oh. I was talking about the heatmap PR.
Kevin Rue-Albrecht (11:25:36): > Yeah.. that what I thought but it was too fun to see@Federico Marinitake the bullet for me:innocent:
Kevin Rue-Albrecht (11:36:40): > As far as I can see I’ve got the documentation of the ComplexHeatmapPlot class and methods sorted (locally). I was waiting to see thatmagickissue resolved before pushing
Kevin Rue-Albrecht (11:36:55): > Anyway - next up are unit tests.
Kevin Rue-Albrecht (11:37:48): > Only once those are back to a good level for the new code will I bother considering amending or adding any new feature to the heatmap panel
Kevin Rue-Albrecht (11:38:35): > I’d rather have a minimal panel working by the next release than a fancy(ier) one that would require an extra 6 month
Aaron Lun (11:41:24): > As long as we can map user’s old choices to the new heatmap, that is fine.
Kevin Rue-Albrecht (12:00:41): > You can be the judge of that. It’s my latest commit (aside from magick business):https://github.com/iSEE/iSEE/commit/a5175aad6697b2061ebaeb58d5bae66eabe3ef74#diff-f624098ec4d23f8cbc0468fcaef383fe
Kevin Rue-Albrecht (12:01:51): > Considering that we’ve changed the underlying heatmap function (ggplot -> ComplexHeatmap) I’ve mapped what could be mapped. If you wanna change stuff, be my guest
Aaron Lun (12:03:10): > centering, lower and upper don’t seem to have a mapping.
Kevin Rue-Albrecht (12:03:33): > Because it’s not reimplemented yet
Aaron Lun (12:04:08): > To be clear, when I said “mapping”, I was referring to functionality, not just the literal mapping of the constructor.
Aaron Lun (12:04:32): > So as long as we can say to people that no functionality was lost, I’m happy with that.
Kevin Rue-Albrecht (12:07:52): > Oh ok. Well yeah well the idea of mapping old to new functionality itches me too, but unfortunately time is scarce lately. I’m hoping to spare a bit of time this weekend. I’ll definitely finalize and push documentation for the panel later this evening.
Kevin Rue-Albrecht (12:12:32): > To clarify: when you say PR, do we agree that the plan is, in that order > * refactor-heatmap-> merge ->refactor > * refactor-> merge ->master > (Whenrefactoris ready, obviously)
Aaron Lun (12:12:44): > Yes
Kevin Rue-Albrecht (17:14:05): > Finally a miraclehttps://travis-ci.org/iSEE/iSEE/builds/641546598 - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Aaron Lun (17:14:54): > oh yeah
Kevin Rue-Albrecht (17:15:28): > thatknitrthing was driving me nuts
Aaron Lun (17:18:39): > I’ll pass that onto Yihui when I meet him next week
Kevin Rue-Albrecht (17:22:59): > well you can tell him that it’s one of his comment that ultimately fixed it for mehttps://github.com/yihui/knitr/issues/1796#issuecomment-575371248
Kevin Rue-Albrecht (17:24:12): > oh wait, he must have got a notification already as I put the link above in the commit message. I can see it in the GH conversation
2020-01-25
Aaron Lun (06:16:34): > Good, good. Your hate makes you strong.
Kevin Rue-Albrecht (13:20:00): > … and we are back to < 10 branches after pushing some stale ideas to my fork (kevinrue/iSEE)
Kevin Rue-Albrecht (14:39:31): > Woot woot. Just covered the last line (0.5%) ofExperimentColorMap.R
Kevin Rue-Albrecht (17:41:40): > Ahhh the good ol’days of unit testing
Kevin Rue-Albrecht (17:42:15): > Already brought up a (tiny) glitch in the deprecation code
Aaron Lun (17:42:27): > :chuuni:
Kevin Rue-Albrecht (18:25:52): > Hanging up for today.
Kevin Rue-Albrecht (18:26:34): > (Re?) discovered an issue involving open lasso and size/shape aesthetics:https://github.com/iSEE/iSEE/issues/344
Kevin Rue-Albrecht (18:27:54): > Thankfully, it doesn’t crash the app. If just affects the panel. So “medium-priority” bug
Aaron Lun (18:29:52): > I barely understand that ggplot code as it is.
Kevin Rue-Albrecht (18:32:51): > Long story, but the result of a lot of back and forth between us about how the lasso should look depending on the active aesthetics
Kevin Rue-Albrecht (18:33:05): > See the comment in the issue for the overall logic
Kevin Rue-Albrecht (18:42:55): > Therefactorbranch is now ~25.4% of themasterbranch (in commits)… when does it becomes worth starting a new repo instead?:kissing::musical_note:
Kevin Rue-Albrecht (18:48:56): > Anyway, really leaving it here for tonight. I’ll do some more tomorrow, let’s see how far I can go
2020-01-26
Kevin Rue-Albrecht (09:58:27): > @Aaron LunSanity check: how canxever beNULLin signatures wherexis explicitly something that’s not NULL, e.g.,ColumnDotPlotCan I just remove thoseif(is.null(x))statements and see if anything breaks? - File (PNG): image.png
Kevin Rue-Albrecht (10:05:42): > Nevermind, I just rediscovered the logic of howcallNextMethodcan turn that incomingxinto aNULL. Tracking the code of parent classes, it just seems that there is currently no scenario where it is madeNULLat this point
Kevin Rue-Albrecht (12:13:36): > Forget about the above. It’s long fixed now. I’ve got another one:
Kevin Rue-Albrecht (12:15:18): > See the “Transmitting ColorBySampName …” on the right. “ColorBySampName” should probably be replaced by something user-friendlier, though I don’t know how yet. > Also, when coloring RedDimPlot by “Sample name” coming from Column Data Plot, only one point is colored, even when selecting multiple data points. - File (PNG): image.png
Aaron Lun (13:41:15): > The latter is historical and reflects the fact that the selection goes through the selectize.
Aaron Lun (13:41:58): > Otherwise, if they want to color multiple points, they should use the multiple selection options.
Kevin Rue-Albrecht (13:49:11): > OK, well I won’t get to that tonight
Kevin Rue-Albrecht (13:49:36): > Having way too much fun going “zero to hero” with the unit test coverage
Kevin Rue-Albrecht (17:10:43): > Btw, for that story about using stuff inplot_envto pass information to downstream methods, make sure you useinherits=FALSE, otherwise it picks up variables that exist in.GlobalEnvwhich can lead to pretty odd behaviors.. > > exists("col_selected", envir=plot_env, inherits=FALSE) >
Aaron Lun (17:11:08): > Yeah.
Kevin Rue-Albrecht (17:11:47): > I haven’t checked all of them yet in the code. I just came across one while writing the test.
Kevin Rue-Albrecht (17:12:13): > those tests are are painful and frustrating as they are useful, to be fair
Kevin Rue-Albrecht (17:53:11): > I’m leaving the tests at that for tonight.
2020-01-27
Aaron Lun (16:27:46): > I think we only have two more things to do in the short term.
Aaron Lun (16:28:06): > The first is the log/centering of the heatmap, which@Kevin Rue-Albrecht.
Aaron Lun (16:28:21): > I will also restore the custom panels, which are pretty useful for people who don’t want to set up a class.
Kevin Rue-Albrecht (16:29:42): > Oh. I really hoped that was the end of the custom panels. I know setting up a new class is a learning curve. But custom panels were so… clunky
Aaron Lun (16:30:19): > Well. Someway to EXTREMELY EASILY set up a new class. Maybe a function that generates a new panel instance, sort of like asetPanel()thing that creates a class in the local session and fills it in.
Kevin Rue-Albrecht (16:33:41): > Mmh. Perhaps. But I’d say that higher priority would be to update the vignettes. I’d really like to see the package in a shape ready for the next release before considering adding more features.
Aaron Lun (16:34:59): > The vignettes are fine, they wouldn’t build if they’re not up to date.
Aaron Lun (16:35:08): > And the heatmap thing is not a new feature, it’s an old one.
Aaron Lun (16:35:51): > Same for the custom panels, really.
Kevin Rue-Albrecht (16:36:16): > > The vignettes are fine, they wouldn’t build if they’re not up to date. > Uh. Sorry. Didn’t see that happening
Kevin Rue-Albrecht (16:38:17): > OK, so .. my plan is to have those unit tests back to 100% next weekend.
Kevin Rue-Albrecht (16:38:34): > For the centering, famous last words are “it shouldn’t be that long/bad”
Kevin Rue-Albrecht (16:40:53): > log is also not complicated syntactically but raises questions about disabling it for non-numeric and negative values
Aaron Lun (16:41:08): > Oh wait, log doesn’t matter, I got confused.
Aaron Lun (16:41:15): > Centering and scaling is what I meant.
Kevin Rue-Albrecht (16:42:06): > Right. One less worry then. Good.
Kevin Rue-Albrecht (16:44:28): > If you’re bored yourself, did you seehttps://github.com/orgs/iSEE/projects/1about > > do not rerender heatmap when incoming row selection changes, if using the custom feature list
Kevin Rue-Albrecht (16:45:35): > It’s because we set up an observer to allow directly updating the heatmap using the incoming selection instead of passing through the modal.
Kevin Rue-Albrecht (16:47:01): > The consequence of that is that even when using the modal input, the heatmap is redrawn when the incoming selection changes (even though the heatmap features hasn’t changed)
Kevin Rue-Albrecht (16:47:29): > it’s like this observer should be conditionally disabled if using the modal content
Aaron Lun (16:50:15): > Might be possible to just delete the edge in the selection_links graph (or whatever it’s called now) when we are using the modal for the rows.
Aaron Lun (16:51:04): > That should be possible and not have any problems. I think.
Aaron Lun (16:51:21): > As long as you redraw the edge when you check the box to use the incoming selection for the rows.
Kevin Rue-Albrecht (16:53:38): > OK, well, I’ll finish the tests first and give this some thought if you don’t get there before I do
2020-01-28
Kevin Rue-Albrecht (06:42:17): > I think we’re covered (see the 237x on the left) …:rolling_on_the_floor_laughing: - File (PNG): image.png
Aaron Lun (11:29:59): > oh yeah
Kevin Rue-Albrecht (16:12:16): > No one touches anything onrefactor. I’m wrapping up the commits to ramp up the coverage
Kevin Rue-Albrecht (16:56:21): > https://www.youtube.com/watch?v=V92OBNsQgxU - Attachment (YouTube): Wagner - RIDE OF THE VALKYRIES - Furtwangler
Kevin Rue-Albrecht (16:56:39): - File (PNG): image.png
Kevin Rue-Albrecht (17:14:42): > “Around the codebase in 1 PR” - Jules Verne > > cat R/* | wc -l > 14328 > - File (PNG): image.png
Kevin Rue-Albrecht (17:16:51): > Also known as “There and Back Again” - Bilbo Baggins
2020-01-29
Kevin Rue-Albrecht (05:38:09): > Inspiration for users out there: Application of the iSEE heatmap to curate clusters at various resolutions :) - File (PNG): image.png
Kevin Rue-Albrecht (05:40:00): > FWIW: I’m so looking forward to 2.0 so that we could also throw aclustreepanel iniSEEu
Kevin Rue-Albrecht (14:16:12): > Happy to point out that the automated docker build for the iSEE organization is up and running:https://hub.docker.com/r/iseedevelopers/isee
Kevin Rue-Albrecht (16:57:11): > While I’m thinking about the center/scale feature: does it really ever happen that one does one without the other? does it really make sense to center without scaling when all features are ultimately on the same color scale?
Aaron Lun (16:57:39): > I often center without scaling.
Aaron Lun (16:57:45): > those are log-fold changes from the average.
Kevin Rue-Albrecht (16:57:54): > OK, just checking
Aaron Lun (16:58:04): > I never scale without centering, though. Now that doesn’t really make much sense.
Kevin Rue-Albrecht (16:58:57): > OK, it seemed to me that two independent checkboxes had something odd to it
Kevin Rue-Albrecht (16:59:04): > I can hide one conditionally.
Kevin Rue-Albrecht (16:59:55): > That said in the thinking process I also got to think that it could (?) be nice to have another type of scaling, namely “% of max” (per feature)
Kevin Rue-Albrecht (17:00:41): > Though that usually also benefits from using a quantile of the max, which would be yet another UI.
Aaron Lun (17:00:59): > ¯*(ツ)*/¯
Aaron Lun (17:01:00): > Sure.
Kevin Rue-Albrecht (17:01:14): > Anyway - let’s stick to making the old implementation live again before adding new stuff
Kevin Rue-Albrecht (17:51:53): > hmpf.. I guesscheckboxGroupInputexists for a reason: independent checkboxes show on separate lines (i.e. not inline) - File (PNG): image.png
Kevin Rue-Albrecht (17:52:12): - File (PNG): image.png
Aaron Lun (17:52:41): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (17:53:01): > but they need to be implemented as separate boxes if “scale” is to be shown only when “center” is active
Aaron Lun (17:53:14): > Yes, I know.
Kevin Rue-Albrecht (17:53:28): > sanity check
Kevin Rue-Albrecht (17:54:17): > oh well. it probably won’t look that bad anymore once we have the usual master checkboxInput at the top to show selected “sets” of visual parameters
Kevin Rue-Albrecht (18:14:23): > @Federico Mariniif you don’t know it yet:https://github.com/rstudio/bootstraplib/
Kevin Rue-Albrecht (18:16:48): > you should be able to Bioc’ize apps in no time with that
Kevin Rue-Albrecht (19:07:16): > There ya go, branchrefactor-transformsfor a preview using radiobuttons: inline and elegant enough for a mere 3 options
Kevin Rue-Albrecht (19:07:23): > the plot doesn’t react to the choice yet, I was just playing with the UI
Kevin Rue-Albrecht (19:08:56): > It’s not bad either, considering that checkboxes can leave the order of transformations open for interpretation anyway
Kevin Rue-Albrecht (19:10:10): > (in case we wanted to offer other transforms, e.g. log)
2020-01-30
Federico Marini (01:59:58) (in thread): > reminds me of the other opkg, Fresh
Kevin Rue-Albrecht (05:12:02): > Ugh. Looks like the unit tests are down on RELEASE_3_10 following the scater updates:sad_file: > > 1. Error: .make_customDataPlot responds to a transmitting column brush receiver (@test_custom.R#116) > 2. Error: .make_customDataPlot responds to a transmitting column brush receiver (@test_custom.R#153) > 3. Error: .showPanel adds a valid row to the table of active panels (@test_voice.R#121) > > Anyway, doesn’t affect the fix to the functionality of the package that we just pushed as 1.6.1
Kevin Rue-Albrecht (05:22:24): > Uh…. I just realize that the heatmap centering and scaling was applied to the whole matrix, as opposed to centering and scaling each row (=feature) separately.https://github.com/iSEE/iSEE/blob/master/R/heatmap.R#L117Thoughts?
Federico Marini (05:24:26): > … that it should be done row wise indeed?
Kevin Rue-Albrecht (05:24:43): > yup, that’s my thinking
Federico Marini (05:24:43): > Otherwise the whole point of scaling/centering is gone, I gues
Federico Marini (05:24:44): > s
Kevin Rue-Albrecht (05:25:30): > Centering across all features… I’m trying to imagine if there’s any scenarion when that’s even desirable
Kevin Rue-Albrecht (05:26:07): > given that features can be in wildly different ranges
Federico Marini (05:28:11): > The way I see it (ah ah, not intended pun) is that centering (or Z scaling) is done to have an immediate glimpse across samples,by feature, odf what is up or down
Federico Marini (05:28:51): > It would make sense if all are on the same scale, but even in that case, I’d do it row wise, and then in the end it would have anyway the same scale:stuck_out_tongue:
Kevin Rue-Albrecht (05:29:30): > yup
Kevin Rue-Albrecht (05:32:16): > Oh no wait - I’m just not used to theavefunction. the…is a grouping factor
Kevin Rue-Albrecht (05:33:00): > Not completely awake yet. That would have been a surprisingly massive oversight:sweat_smile:
Kevin Rue-Albrecht (06:07:07): > :sob:centering and scaling UI need to be hidden when a discrete assay is selected:sob::sob::sob::sob::sob:
Stephanie Hicks (06:22:48): > Tweets from@Leonardo Collado Torreshttps://twitter.com/fellgernon/status/1222653061413261312:tada: - Attachment (twitter): Attachment > Bioconductor @Bioconductor #rstats community members @KevinRUE67 @CSoneson @FedeBioinfo Aaron Lun get a shoutout due to their award winning #shiny app #iSEE at #rstudioconf :raised_hands::skin-tone-4::raised_hands::skin-tone-4::raised_hands::skin-tone-4::raised_hands::skin-tone-4: > > It builds upon #shiny and also #SummarizedExperiment, both packages I :heart: ^^ > > #fixedTheTypos https://pbs.twimg.com/media/EPe8ei8UcAA5Kw9.jpg
Federico Marini (06:39:28): > As I said to Charlotte, I wonder if the long wave of the shiny contest will ever end:smile:
Federico Marini (06:39:37): > Great job Leo in capturing that!
Federico Marini (06:39:59): > Next time they do so, they should also put the sticker
Federico Marini (06:40:00): > :smile:
Kevin Rue-Albrecht (06:49:28): > Hallelujah the almight Shiny god be praised - File (PNG): image.png
Kevin Rue-Albrecht (10:51:42): > https://blog.rstudio.com/2019/12/05/emails-from-r-blastula-0-3/No Kevin, don’t think about it …:innocent: - Attachment (blog.rstudio.com): Emails from R: Blastula 0.3 | RStudio Blog
2020-01-31
Kevin Rue-Albrecht (10:12:46): > Mmmh - tempted to mess with the audience in future presentations ^^ - File (PNG): image.png
Kevin Rue-Albrecht (10:13:58): - File (PNG): image.png
Federico Marini (10:33:43): > Tsk. Think of having impaired people feeling the data
Federico Marini (10:33:47): > from the “rug of points”
Federico Marini (10:34:09): > i’m sure that as we speak someone is thinking of monetizing this
Kevin Rue-Albrecht (10:42:46): > I know. This is one of the worst use cases for 3D plotting. I just wanted to try out the package (rayshaderpresented at RStudio conference yestreday). At least it’s really easy to use, as advertised.
Kevin Rue-Albrecht (10:43:03): > You just pass it the ggplot object:plot_gg(<ggplot_object>)
Federico Marini (10:47:52): > I know, it is crazy easy now
Federico Marini (10:48:03): > that guy Tyler is the bomb
Aaron Lun (11:29:23): > oh that’s funny
Federico Marini (12:04:47): > Feeling a tSNE plot like a rugged carpet is also something that somehow gives me some cheap thrills
Federico Marini (12:05:19): > I do have such carpets in the bathroom, btw (my turn on updating on the bathroom status, Aaron:smile:)
Federico Marini (13:16:04): - File (JPEG): Photo on 31.01.20 at 19.15 #2.jpg
Federico Marini (13:16:09): > Taking the sticker game to the next level:slightly_smiling_face:
Kevin Rue-Albrecht (13:18:22): > I see (oh oh oh)
Federico Marini (13:18:34): > 2.0 sticker, 2.0 version
Federico Marini (13:19:00): > as usual, happy to send around each one the fair share of needed merchandising:smile:
Kevin Rue-Albrecht (14:14:22): > Fede’s back to back “meetings” earlier today: running from office to office around campus “can I interest you in a new iSEE sticker”:stuck_out_tongue_winking_eye::stuck_out_tongue_winking_eye::stuck_out_tongue_winking_eye::stuck_out_tongue_winking_eye::stuck_out_tongue_winking_eye:
Federico Marini (14:18:33): > Too bad I got them delivered at home
Federico Marini (14:18:59): > I could have been with a permanent dumb smile through all the day
Kevin Rue-Albrecht (14:22:10): > Right there with you. I know I’d be too:sweat_smile:
Kevin Rue-Albrecht (16:33:12): > Alright, let’s get that heatmap done
Kevin Rue-Albrecht (17:43:02): > Have a play.
Kevin Rue-Albrecht (17:43:08): > Branchrefactor
Kevin Rue-Albrecht (17:44:07): > There are still a few kinks to iron out, but it’s getting there
Kevin Rue-Albrecht (17:45:16): > e.g. I would like the UI controlling the custom lower and upper bounds of the assay colorscale to be updated to match the range ofplot.datawhen the transformation (centering, scaling) is changed
Kevin Rue-Albrecht (17:48:39): > To illustrate the point I ran into the following scenario: > * the logcounts assay is active > * I activate the custom lower/upper bounds and set them to something consistent with the range of the logcounts [0, 20] > * I toggle centering for the assay: the values are now centered [-5, 10] while I’m still applying the “natural logcounts” bounds
Kevin Rue-Albrecht (17:51:30): > anyway - logging off, I’ll pick up the code tomorrow, feedback (and help finishing that heatmap) are welcome
Aaron Lun (17:52:29): > I have a separate iSEE-related weekend project required to prepare for its deployment inside the company, so you’re on your own.
Aaron Lun (17:52:35): > Also, I have a bunch of papers to review.
Federico Marini (18:03:11): > I’d be really interested in hearing some details, provided you can share them,@Aaron Lun
Federico Marini (18:03:40): > I assume you’ll have some shiny server pro-like solution?
Aaron Lun (18:03:50): > Rsconnect, but yes.
Aaron Lun (18:04:09): > The tricky bit is allowing other people to click on a link and generate a non-default iSEE instance based on the link parameters.
Aaron Lun (18:04:35): > This is equivalent to the bookmarking problem but because we instantiate from classes, the Shiny bookmarks are not sufficient.
Aaron Lun (18:05:18): > So just like we have special behavior foriSEE()w/o any arguments, I need to add a special mode that takes a base64-encoded set of S4classes and launches the app from that.
Federico Marini (18:06:13): > Interesting
Federico Marini (18:07:15): > What about the hardware behind it? I am asking because indeed that project I mentioned back then got funded, so once the IT will unite on some infrastructural decisions, we might have a green light to go get the things we need
Aaron Lun (18:07:40): > I can ask, but basically Rs connect running off a cluster node with SLURM.
Aaron Lun (18:07:51): > As in, with access to SLURM.
Federico Marini (18:12:20): > Ok, kinda out of the league of what the current stand is
Federico Marini (18:18:14): > thanks tho. We should owe a beer or two to the person that pushed for having the feature of uploading implemented
Federico Marini (18:18:29): > plus, well, the internal beer round to the implementer:slightly_smiling_face:
2020-02-01
Aaron Lun (14:00:16): > @Kevin Rue-Albrechtyou shouhld consider having a separateheatmap.Rfile to handle all of the non-S4-related utilities.
Kevin Rue-Albrecht (14:00:37): > sane advice
Kevin Rue-Albrecht (14:01:10): > i’m just in the middle of YAF (yet another fix), but I’ll clean after
Kevin Rue-Albrecht (14:01:50): > aka “forbid negative upper bounds and positive lower bounds for the heatmap colorscale when rows are centered”
Aaron Lun (14:21:13): > Ugh! I’m sick of reviewing a paper in the morning. Going to have lunch now.
Aaron Lun (14:21:26): > On the plus side, I got my annual pay summary and I can see that I’ve hit the big 2.
Kevin Rue-Albrecht (14:57:59): > I’m OD’ing on heatmap. Leaving it at that for today
Kevin Rue-Albrecht (14:58:07): > It’s mostly there though
Kevin Rue-Albrecht (15:14:10) (in thread): > That point reminds me that one of the barriers to adoption for iSEE around me tends to be the gap between (1) wannabe users who end up running small instances on their laptop, (2) PIs and institute IT departments who are scared/worried/discouraged by the work needed to setup a web server for access by either internal staff or the outside world (including external collaborators). > Point is, we should also have some documentation for the various use cases related to hosting apps.
Kevin Rue-Albrecht (15:14:49) (in thread): > The book could have a chapter about that, if there isn’t a better place
Federico Marini (15:21:54) (in thread): > definitely. the current shiny server at my institute is a war remnant:stuck_out_tongue:
Aaron Lun (17:21:23): > I wonder whether we can get the modal to close upon pressing “apply”.
Kevin Rue-Albrecht (17:21:45): > pretty sure there’s a function for that
Aaron Lun (17:24:47): > Clicking on “Primary.Type” as a column annotation brings up > > Warning: Error in parse: <text>:2:103: unexpected ']' > 1: # Keep all samples to compute the full range of continuous annotations > 2: column_data <- colData(se)[, c("NALIGNED", "TOTAL_DUP", "PCT_CODING_BASES", "Primary.Type", drop=FALSE] > ^ > 84: parse > 82: .text_eval > 81: .process_heatmap_column_annotations_colorscale > 80: .local > 79: .generateOutput > 77: observeEventHandler > 6: shiny::runApp >
Aaron Lun (17:25:12): > Also consider having one of those checkboxes on by default, probably the “Annotation” one.
Aaron Lun (17:26:21): > It would also be nice to show the name of the panel that we’re importing features from.
Aaron Lun (17:27:10): > I have more comments, but let’s start with the above. Most of it seems to work, so I think we’re pretty close.
Kevin Rue-Albrecht (17:38:53) (in thread): > uh. missing a bracket
Kevin Rue-Albrecht (17:39:51) (in thread): > that’s weird
Aaron Lun (17:40:26) (in thread): > That’s also probably why I noticed that all cells had the same metadata data if I just click in “NALIGNED”.
Kevin Rue-Albrecht (17:41:21) (in thread): > I’m tracking the issue now. I must have introduced it recently, that was like the first feature in put in place
Kevin Rue-Albrecht (17:42:36) (in thread): > oh wait,deparsereached the max character length before it wraps to a new line. I hate that thing
Aaron Lun (17:43:27) (in thread): > we have a safe_deparse function somewhere.
Kevin Rue-Albrecht (17:44:58) (in thread): > gotcha.deparse_for_viewing
Kevin Rue-Albrecht (18:08:44): > Alright. I’ve dealt with the above. Thanks for being specific, helps a lot. Really logging off now. Summary:
Kevin Rue-Albrecht (18:10:40): > > I wonder whether we can get the modal to close upon pressing “apply”. > Done > > Clicking on “Primary.Type” as a column annotation brings up > Done. I’ve also applied the fix to rowData annotations. There might be other deparse that could be converted in other places. > > Also consider having one of those checkboxes on by default, probably the “Annotation” one. > Annotations are selected by default > > It would also be nice to show the name of the panel that we’re importing features from. > I’m showing it next to the “import” button. Feel free to relocate. At least the code is in place already.
Aaron Lun (18:11:33): > Most excellent.
Aaron Lun (18:11:55): > Well, while I’m complaining, I’ll note that you might not show “Selected points” if no selection has been made.
Kevin Rue-Albrecht (18:12:04): > One thing I’ve temporarily given up on, if you wanna have a go: > hiding the UI related to centering, scaling, and lower/upper bounds if the assay is discrete
Kevin Rue-Albrecht (18:12:29): > take a moment to think of the implications and you’ll understand why that last one made me cry
Aaron Lun (18:13:06): > I don’t think you can hide them at the conditional JS level. But you should be able to disable them via shinyjs.
Kevin Rue-Albrecht (18:13:11): > > I don’t think you can hide them at the conditional JS level. > exactly
Kevin Rue-Albrecht (18:13:40): > > But you should be able to disable them via shinyjs. > Forgot about that one.
Kevin Rue-Albrecht (18:14:34): > Indeed, we only know the continuous/discrete assays at the R level, while conditional hiding is only at the JS level. Though indeed shinyjs bridges that.
Aaron Lun (18:15:15): > You should also be able to hide them with shiny JS, check outhidden.
Kevin Rue-Albrecht (18:16:01): > Right. Let me stash that in the GH project. Too hard to keep track of things here, and I really don’t wanna do that tonight. It’ll give me an easy warm up exercise tomorrow
Kevin Rue-Albrecht (18:20:06): > G’night. The code base is all yours til my tomorrow
Aaron Lun (20:31:35): > BEHOLD:https://gist.github.com/LTLA/f523f28e014c09b3083865832a3224ce
Aaron Lun (20:33:28): > If you’re running an enterprise operation and you can provide paths/URLs to a SE object and iSEE parameter list, your users can instantly launch a fully parameterized instance by just clicking on a link.
Aaron Lun (20:33:47): > Oh, this requires theinitializationbranch BTW.
Aaron Lun (23:27:11): > You have a missing@importFrom shiny updateNumericInputsomewhere.
2020-02-02
Aaron Lun (02:35:54): > oh gawd finally got that review done.
Aaron Lun (02:35:57): > ANother one to go tomorrow.
Aaron Lun (02:36:02): > And then I will be free!
Kevin Rue-Albrecht (03:54:48): > > You have a missing@importFrom shiny updateNumericInputsomewhere. (edited) > Done
Kevin Rue-Albrecht (13:37:41): > Looks like we might be able to properly mimic shiny sessions at some point, for unit testing:https://github.com/rstudio/shiny/blob/89be4bdce9dce6f8042087ce6ece0763eb8bb10c/R/mock-session.R#L80(not released yet)
Kevin Rue-Albrecht (13:41:01): > I’ve been able to simulate (input, output, session) for a lot of the basic cases recently, but calls toshowNotificationrequire more of thesession$<functions>than I can simulate myself.
Aaron Lun (14:20:29): > Let’s merge this in today.
Aaron Lun (14:24:26): > Finish off the heatmap. I’ll restore the custom panels later.
Aaron Lun (14:27:25): > Get this bad boy on BioC.
Kevin Rue-Albrecht (14:34:53): > Will do in the next few hours. I’ll let you know
Kevin Rue-Albrecht (18:12:27): > Done as far as I can see.
Kevin Rue-Albrecht (18:13:48): > I found a more elegant and I think less confusing way of handling discrete assays: I simply update thecheckboxGroupInputof visual parameter choices to include/exclude options that toggle parameters specific to continuous assays
Kevin Rue-Albrecht (18:14:53): > Excluding the options automatically unselects them, which in turn toggles.conditional_on_check_groupto hide the corresponding inputs
Aaron Lun (18:15:35): > That could interact unpredictably with the app’s memory.
Kevin Rue-Albrecht (18:15:38): > I find this more elegant as it doesn’t leave disabled UI clutter the screen and tempt users into clicking things that would be disabled anyway
Aaron Lun (18:16:18): > If you go back to a continuous assay, the memory will have forgotten that the checkbox for those options were previously selected.
Kevin Rue-Albrecht (18:16:54): > indeed, it will forget, but I’m not sure that’s a bad thing
Aaron Lun (18:17:20): > well, it shouldn’t do that.
Aaron Lun (18:18:06): > if I go to a discrete assay and then go back, I should get the same view of the app.
Kevin Rue-Albrecht (18:19:47): > right
Aaron Lun (18:20:16): > What’s wrong with justdisableing the relevant parameters upon changes to theassaychoice?
Kevin Rue-Albrecht (18:20:49): > it’s just weird leaving stuff that isn’t active, visible to users
Aaron Lun (18:20:57): > Happens all the time.
Aaron Lun (18:21:32): > You couldhiddenit instead.
Kevin Rue-Albrecht (18:22:48): > I started with that one, as you suggested. It was even a stranger effect: I suddenly had 2 visual param choices that were showing/hiding…. nothing
Kevin Rue-Albrecht (18:23:23): > because the checkboxes were still there but their “child UI” were all hidden
Kevin Rue-Albrecht (18:23:58): > I think I’ll settle with disabling/enabling child UIs while leaving them visible
Kevin Rue-Albrecht (18:24:56): > It’s the only thing that is visually and practically coherent
Aaron Lun (18:25:32): > k
Kevin Rue-Albrecht (18:32:35): > Damn… it appears R cannot center or scale character matrices. Odd:stuck_out_tongue:
Kevin Rue-Albrecht (18:33:25): > I look forward to crowd sourcing the bug hunting
Kevin Rue-Albrecht (18:40:29): > I think I broke the heatmap metadata annotations when I got rid of the “Selected points”:sob:
Kevin Rue-Albrecht (18:41:18): > Oh. Maybe not.
Kevin Rue-Albrecht (18:41:46): > I made it crash somehow when selecting column metadata but I can’t reproduce it:confused:
Kevin Rue-Albrecht (18:46:10): > Ouch. More importantly, the limits of continuous metadata colorscales are wrong. All colorscales seem to be driven by the range ofplot.data. A mishap when I refactored the code in the last few days
Kevin Rue-Albrecht (18:48:39): > Fixed.
Kevin Rue-Albrecht (18:48:46): > I think we’re good now.
Kevin Rue-Albrecht (18:54:09): > Yup. I pushed all the buttons and didn’t break anything. Haven’t checked the code tracker though. (I can’t tell the difference anymore - same feeling as reading a draft manuscript for the 1,000th time)
Aaron Lun (20:27:38): > Notes: > * Make sure enabling/disabling works when the panel is instantiated with a character matrix. > * Splitheatmap.Rintooutputs_heatmap.Randobservers_heatmap.R. > * Pick a better name for “DIvergent colormap”, I don’t even know what this means. “Alternative”, or even “Centered”, would be better. > * A better box color for the heatmap class, I can barely see the white text on light-blue.
Aaron Lun (20:30:44): > Also “Transforms” -> “Transform”
Aaron Lun (20:31:57): > Also you have a variable named.assayCenterRowsTitleand.assayScaleRowsTitle, these don’t seem like they should haveTitle. The Title designation indicates that it is a string that is visible in the UI.
Aaron Lun (20:37:38): > Why are we depending oncirclize::colorRamp2? Isn’t there something in base orRColorBrewer?
Aaron Lun (20:38:23): > If those fixes are made, I’m happy for you to do the honors.
2020-02-03
Kevin Rue-Albrecht (03:48:47): > Those do not look terribly difficult to deal with and I agree with all of them, but I won’t be able to do any work at all today. I’ll let you know when I get to it. > A couple of notes: > > A better box color for the heatmap class, I can barely see the white text on light-blue. > Note the “expert”#ABCDEFcolor code indicating that I don’t have the spirit of a color designer yet. > > Why are we depending oncirclize::colorRamp2? Isn’t there something in base orRColorBrewer? > That’s what ComplexHeatmap documents best?Heatmapargumentcol. I imagine it’s possible to provide a wrapper ourselves, supporting base or RColorBrewer, but I’ve never bothered myself, and with limited time it made sense to start with the documented way. Keep also in mind that we’re tracking and reporting the code to users: I’d rather give them a script that is as readable and “best practice” as possible. Diverging from the documentation is more of a “power user” kind of thing.
Federico Marini (04:20:08): > colorRampPalettefromgrDevicesshould also work
2020-02-04
Federico Marini (15:24:36): > Naive question after the new world will be deployed
Federico Marini (15:24:42): > do we have any breaking changes?
Federico Marini (15:24:48): > I already have to think ofhttps://xkcd.com/1172/ - Attachment: Workflow > [Title text] “There are probably children out there holding down spacebar to stay warm in the winter! YOUR UPDATE MURDERS CHILDREN.”
Kevin Rue-Albrecht (15:28:05): > Depends what you call a breaking change:iSEE(sce)still works:slightly_smiling_face:
Kevin Rue-Albrecht (15:29:07): > and we have deprecation-related converters for the*Paramsarguments
Kevin Rue-Albrecht (15:30:39): > In fact, I was hoping that you and Charlotte could be the early testers of the new world, to answer that kind of question.
Federico Marini (15:34:26): > This is indeed on my list
Federico Marini (15:34:30): > I did not forget it
Federico Marini (15:34:45): > When should we start to dig up everything?
Kevin Rue-Albrecht (15:36:45): > I’m about to push commits to deal with Aaron’s latest suggestions
Kevin Rue-Albrecht (15:37:22): > just running my traditionalcovr::package_coverage()first
Kevin Rue-Albrecht (15:43:01): > Ugh. Two tests failed. What did I dooooo:stuck_out_tongue:
Kevin Rue-Albrecht (15:46:17): > damn renaming of constants. forgot to rename them in the tests:face_palm_star_trek:
Kevin Rue-Albrecht (16:57:56): > There’s a funky incoming commit@Aaron Lun: thanks to the latest round of unit test fixes, I noticed thatsynchronizeAssayswas working under the assumption that all assays were continuous, when it had to assign a default colormap to orphan assays. Mostly because we didn’t check the nature of each assay inse. > I borrowed the.cacheCommonInfo,ComplexHeatmapPlotwhich does exactly that, to assign a default colormap matching the nature of each assay
Kevin Rue-Albrecht (16:58:41): > there might be more elegant, but I’m honing my “code reuse” skills
Kevin Rue-Albrecht (17:00:02): > (though tbh I hardly see anything more concise than.cacheCommonInfo,ComplexHeatmapPlotto identify the nature of each assay)
Kevin Rue-Albrecht (17:00:36): > bottom line is that there was an issue and it’s fixed (as demonstrated per updated unit tests)
Aaron Lun (17:03:56): > I would prefer you not use the.cacheCommonInfoin other contexts; strip out the functionality in a separate function that happens to be called by bothsyncandcache
Kevin Rue-Albrecht (17:03:58): > Thinking about mergingrefactormakes me realize that I’ve got no clue how to write aNEWSentry for 2.0 > “Hold on to your hats. The world has changed”
Kevin Rue-Albrecht (17:05:39): > Good point. Then comes the hardest part: deciding in which R file that functionality goes. Some kind of utils-* I guess
Kevin Rue-Albrecht (17:43:36): > partially named assays must die
Kevin Rue-Albrecht (17:46:50): > Done. I just pushed my last one of the day.
Kevin Rue-Albrecht (17:48:32): > Btw, I got one for you@Aaron Lun: (https://github.com/orgs/iSEE/projects/1) > > DataBoxOpen=TRUEdoes not open the data param box on startup > I’ve had a brief look but didn’t get why (the other boxes work just fine)
Kevin Rue-Albrecht (17:56:34): > Woooot Nevermind I got it:smile:
2020-02-05
Kevin Rue-Albrecht (15:58:10): > Alright. I spotted a couple more minor things which I just pushed torefactor
Aaron Lun (15:59:44): > DO IT
Kevin Rue-Albrecht (16:00:37): > Yup. I’ve gone as far as I could.
Kevin Rue-Albrecht (16:02:57): > There’s only one thing which I’ve been looking into but I haven’t cracked yet: > when the heatmap is set to use custom features (not the directly incoming row selection), brushing in the transmitter panel triggers a rerendering of the heatmap which is effectively a no-op.
Kevin Rue-Albrecht (16:03:43): > I don’t know if it’s possible toreturn(NULL)somewhere, to interrupt the observer before it reaches the corresponding.requestUpdate
Kevin Rue-Albrecht (16:19:59): - File (PNG): image.png
Kevin Rue-Albrecht (16:20:06): - File (PNG): image.png
Aaron Lun (16:20:11): > YEAH
Kevin Rue-Albrecht (16:54:57): > We’re having aRtsnemissing issue on Travis
Kevin Rue-Albrecht (16:54:59): > https://travis-ci.org/iSEE/iSEE/builds/646602628?utm_source=github_status&utm_medium=notification - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (16:55:12): > shouldn’t it be brought in byscater?
Kevin Rue-Albrecht (16:56:47): > Y’know what, I’ll just clear the package cache on Travis and run a build from scratch. Doesn’t cost anything but time
Aaron Lun (17:14:26): > scatersuggests it.
Kevin Rue-Albrecht (17:15:19): > I know. I don’t get why it doesn’t get installed. Re-run of Travis is here:https://travis-ci.org/iSEE/iSEE/builds/646602628?utm_medium=notification&utm_source=github_status - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (17:15:36): > From the looks of it, Rtsne isn’t in the list of packages being installed, again.
Kevin Rue-Albrecht (17:16:14): > I just checked?devtools::installwhich I’m sure you know says > > dependencies``TRUEis shorthand for “Depends”, “Imports”, “LinkingTo” and “Suggests”.NAis shorthand for “Depends”, “Imports” and “LinkingTo” and is the default.FALSEis shorthand for no dependencies (i.e. just check this package, not its dependencies).
Kevin Rue-Albrecht (17:16:34): > But it doesn’t really say anything about Suggests from Suggest’ed dependencies
Aaron Lun (17:17:05): > Well, it’s a Suggests of a Suggests, which might be why.
Kevin Rue-Albrecht (17:17:45): > I’m too lazy to browsescater’s history: was Rtsne an Imports before?
Kevin Rue-Albrecht (17:17:58): > I’m curious what changed that causes the issue now
Aaron Lun (17:24:57): > It was a Suggests before. But we only ever called it viascater.
Aaron Lun (17:25:21): > It seems like this is a bug of Travis to not pull down Suggests of Suggests.
Kevin Rue-Albrecht (17:26:03): > Anyway. I’m making another branch in the meantime, to Suggest Rtsne directly ourselves. Feel free to open a Travis-R issue if you like
Kevin Rue-Albrecht (17:28:28): > Uh. That’s odd, the latest build ofmasterbefore that was detecting Rtsne as a dependency: > > Installing 75 packages: SummarizedExperiment, SingleCellExperiment, BiocGenerics, S4Vectors, shinydashboard, shinyAce, shinyjs, rintrojs, colourpicker, igraph, vipor, rentrez, AnnotationDbi, cowplot, dplyr, BiocStyle, knitr, rmarkdown, scRNAseq, scater, DelayedArray, Rtsne, irlba, viridis, org.Mm.eg.db, GenomicRanges, Biobase, IRanges, GenomeInfoDb, miniUI, XML, DBI, RSQLite, tidyselect, plogr, bookdown, highr, markdown, xfun, base64enc, tinytex, ExperimentHub, ggbeeswarm, DelayedMatrixStats, BiocNeighbors, BiocSingular, BiocParallel, beachmat, matrixStats, gridExtra, XVector, RCurl, GenomeInfoDbData, zlibbioc, bitops, futile.logger, snow, lambda.r, futile.options, formatR, bit64, blob, bit, AnnotationHub, BiocFileCache, rappdirs, interactiveDisplayBase, dbplyr, beeswarm, HDF5Array, rhdf5, Rhdf5lib, RcppAnnoy, RcppHNSW, rsvd > https://travis-ci.org/iSEE/iSEE/builds/646133158
Kevin Rue-Albrecht (17:49:02): > Found the answer:https://travis-ci.community/t/is-there-a-way-not-to-install-r-packages-suggested-by-dependencies/3870/5 > > Only directed suggested dependencies of your packages are installed, not all suggested dependencies of all dependencies. - Attachment (Travis CI Community): Is there a way not to install R packages suggested by dependencies? > To be honest, I don’t really understand what happens behind the scene either; I just tinkered around until it worked. The prepare cache step installs the R dependencies and caches them, not running R CMD check until the test stage. Then in the test stage, R CMD check actually doesn’t start immediately. It will take a while to get the cached packages ready, but that still saves a lot of time compared to installing from scratch so it does not time out. I haven’t tried the prepare cache state with …
Kevin Rue-Albrecht (17:49:32): > Which begs the question how did we get Rtsne automatically installed before. Meh.
Aaron Lun (17:49:46): > Can’t you put that requirement in the travis and not in theDESCRIPTION?
Aaron Lun (17:49:51): > Seems like a Travis problem.
Kevin Rue-Albrecht (17:51:03): > > Can’t you put that requirement in the travis and not in theDESCRIPTION? > I didn’t bother saying it here, but in the end that’s exactly what I did
Kevin Rue-Albrecht (17:51:52): > > Seems like a Travis problem. > I thought almost exactly that. Though not a “problem” per se, just a design choice
Kevin Rue-Albrecht (18:24:36): > all done, g’night
Aaron Lun (18:24:51): > pushed to BioC/
2020-02-06
Kevin Rue-Albrecht (02:22:44): > did you? you can?
Kevin Rue-Albrecht (02:24:07): > Apparently not, so I guess that was “push” to Bioc.
Kevin Rue-Albrecht (02:28:11): > Done.
Aaron Lun (02:29:57): > I don’t think I have push access to BioC anyway.
Kevin Rue-Albrecht (02:47:04): > I had the new travis settings backwards:https://blog.travis-ci.com/2019-08-27-deployment-tooling-dpl-v2-preview-release > > cleanup: false > - Attachment (blog.travis-ci.com): Announcing dpl v2 developer preview release > Over the last few months, we have been rewriting the current codebase of dpl, our deployment tooling, and the result is a new major version: dpl v2. Almost every line of code has been touched, cod…
Aaron Lun (02:47:47): > Only +15000? WEAK.
Federico Marini (09:49:52): > https://community.rstudio.com/t/shiny-contest-2020-is-here/51002 - Attachment (RStudio Community): Shiny Contest 2020 is here! > Shiny apps are a great way to communicate your data science insights with striking, dynamic, interactive visualizations and reports. Over the years, we have loved interacting with the Shiny community and loved seeing and sharing all the exciting apps, dashboards, and interactive documents Shiny developers have produced. So last year we launched the Shiny contest and we were overwhelmed (in the best way possible!) by the 136 submissions! Reviewing all these submissions was incredibly inspiring an…
Kevin Rue-Albrecht (09:52:47): > Finally we’re in the galleryhttps://shiny.rstudio.com/gallery/I thought that would never happen
Aaron Lun (11:36:51): > you’re spawning a lot of tokens. Is this deliberate?
Kevin Rue-Albrecht (12:41:39): > Yup
Kevin Rue-Albrecht (12:42:04): > I can’t get Travis to deploy pkgdown on GitHub pages
Kevin Rue-Albrecht (12:44:03): > I’ve finally managed to set up the keys on Travis and GitHub using the Travis package, but Travis doesn’t detect the key at the point of deploying
Kevin Rue-Albrecht (12:50:24): > Between the repo settings, my GH account, the organisation account, my Travis account, the isee-bot machine user account on GH and Travis. I just don’t know where it’s going wrong
Kevin Rue-Albrecht (14:13:05): > In particular, I don’t get why for some reason the new key doesn’t seem to be detected (end ofhttps://travis-ci.org/iSEE/iSEE/builds/646762083) or used (https://github.com/iSEE/iSEE/settings/keys). - Attachment (travis-ci.org): Travis CI - Test and Deploy Your Code with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (14:14:12): > Though at the start of the build it seems to detect it > > $ export TRAVIS_DEPLOY_KEY=[secure] >
Kevin Rue-Albrecht (17:07:42): > Thanks to@Leonardo Collado TorresI found a workaround:https://github.com/r-lib/pkgdown/issues/1206
Kevin Rue-Albrecht (17:57:09): > wooohooohohohoohttps://isee.github.io/iSEE/:tada::tada::tada::tada::tada: - Attachment (isee.github.io): Interactive SummarizedExperiment Explorer > Create an interactive Shiny-based graphical user interface for exploring data stored in SummarizedExperiment objects, including row- and column-level metadata. The interface supports transmission of selections between plots and tables, code tracking, interactive tours, interactive or programmatic initialization, preservation of app state, and extensibility to new panel types via S4 classes. Special attention is given to single-cell data in a SingleCellExperiment object with visualization of dimensionality reduction results.
Aaron Lun (17:58:30): > EXECLELTN
Peter Hickey (18:06:35): > something odd when clicking the “Click to expand the list of features available in iSEE applications.” button
Peter Hickey (18:06:44): > looks like somethings not getting marked up
Aaron Lun (18:08:02): > But there’s all that beautiful raw markdown
Kevin Rue-Albrecht (18:08:34): > uh oh - i updated the travis badge, perhaps i touched something else
Kevin Rue-Albrecht (18:09:18): > ? it all looks normal to me - File (PNG): image.png
Kevin Rue-Albrecht (18:09:22): > screenshot or didn’t happen
Kevin Rue-Albrecht (18:09:45): > oh wait
Kevin Rue-Albrecht (18:09:54): > in the pkgdown pages, sorry
Peter Hickey (18:10:00): - File (PNG): image.png
Peter Hickey (18:10:10): > yah, online i mean
Peter Hickey (18:10:20): > chrome on ubuntu FWIW
Kevin Rue-Albrecht (18:10:48): > oh well - the markdown is also messed up in the “articles” navbar menu
2020-02-07
Kevin Rue-Albrecht (03:29:25): > https://github.com/iSEE/iSEE/issues/350just gave me the motivation for finally writing a GitHub issue template:sleepy:
Kevin Rue-Albrecht (03:30:12): > Session info, web browser, and steps to reproduce the issue
Kevin Rue-Albrecht (10:17:52): > Funny new functionality in RStudio - File (PNG): image.png
Charlotte Soneson (10:36:06): > Starting to go through the documentation - the following line (and similar) appear sometimes underDefining the interfaceand sometimes underDefining the panel name(sometimes both). Any strong feelings for where it should go? > > #' \item \code{\link{.fullName}(x)} will return the full name of the panel class. >
Kevin Rue-Albrecht (10:36:42): > panel name
2020-02-08
GitHub (Legacy) (01:18:59): > was added to this conversation
Aaron Lun (01:21:03): > What happens if I typehttps://github.com/iSEE/iSEE/pull/353?
Aaron Lun (01:21:09): > Hm. Nothing.
Aaron Lun (01:23:03): > Hmmm….
Aaron Lun (01:24:10): > oh well.
Aaron Lun (04:01:56): > @Kevin Rue-Albrechtif you’re going to do these microPRs then rebase rather than doing a merge commit.
Kevin Rue-Albrecht (04:02:31): > yeah sorry - i was doing it on the GH website itself
Aaron Lun (04:03:25): > Also, a few of your tests will break with the upcoming SCE update, it seems you hard-coded things (e.g., the final tests intest_table.R). Probably want to be less stringent here.
Kevin Rue-Albrecht (04:05:54): > FWIW, I never figured out to userebaseproperly. If you have a short explanation or a good resource, I’m all ears. The git docs aren’t really clear to me on that subject
Kevin Rue-Albrecht (04:07:18): > I generallymergeinstead
Aaron Lun (04:08:55): > rebase is as if you had committed to master all along, effectively transplanting your commits from your PR branch to master. This avoids an extra merge commit at the cost of trashing your branch history - but the latter is not an issue for short-lived branches.
Aaron Lun (04:09:38): > So it mimics a linear flow of commits, while merge will preserve the two streams of commits becoming one.
Kevin Rue-Albrecht (04:12:33): > > Another side effect ofrebasingwith remote branches is that you need to force push at some point. > So where does therebasehappen? What’s the sequence of commands? because we clearly don’t want to force push tomaster
Kevin Rue-Albrecht (04:13:06): > I’m a little worried whenhttps://git-scm.com/book/en/v2/Git-Branching-Rebasingsays > > $ git checkout experiment > $ git rebase master > First, rewinding head to replay your work on top of it... > Applying: added staged command >
Aaron Lun (04:13:12): > Just look at the drop down choices on the green button in the PR.
Kevin Rue-Albrecht (04:13:26): > ow
Kevin Rue-Albrecht (04:13:31): > never paid attention to that one
Kevin Rue-Albrecht (04:14:09): > so “rebase and merge” it is?
Aaron Lun (04:14:58): > Yes. And fix your tests on 353, I find it hard to test things because you didn’t putiSEE:::in front of your internal functions.
Kevin Rue-Albrecht (04:18:02): > Well I find it hard to test code that I haven’t written. You know better than me how you want your own code to be tested. Come on.
Aaron Lun (04:20:32): > Well, what environment are you using? Are you runningdevtools::load_all()before every test? Seems a bit dangerous.
Kevin Rue-Albrecht (04:22:09): > Yes I use that. What do you find dangerous about it? > I know it used to cause me to forget theiSEE:::but I’m putting that one everywhere now. I just haven’t fixed all the old tests yet
Aaron Lun (04:22:27): > okay.
Aaron Lun (04:22:52): > Well anyway, something is breaking travis, so if you can fix it, merge the PR and close rob’s issue.
Aaron Lun (04:33:49): > As for your previous question, I’m pretty sureload_allchanges some kind of namespace search path, causing weird problems like, e.g.,test_checkrefusing to seeBiocGenerics::cbindin top-level calls withintest_thatblocks.testthatitself runs inload_allmode, which triggers these problems and requires me to prefacecbindand related implicit generics with their package IDs. Who knows what else is going on there, so I always make sure that the tests can be run outside of aload_allcontext.
Kevin Rue-Albrecht (04:35:39): > ok - well if you have some tips/tricks/guidelines for us to remember, you might want to list them out in a wiki page, like I did for Docker
Aaron Lun (04:36:44): > Anyway, that fail should be fixed, but you’ll need to fix those fragile table tests when the SCE PR is merged.
Kevin Rue-Albrecht (04:37:38): > I’ll let you know when I get a chance
Aaron Lun (04:44:14): > We actually have bigger problems. Check out this craziness: - File (PNG): image.png
Aaron Lun (04:44:52): > Our points are getting scrambled in terms of their assignments.
Kevin Rue-Albrecht (04:45:10): > Ugh. What does the code tracker say?
Aaron Lun (04:47:32): > Looking at the console gives a clue: > > Warning in width_x * runif(nrow(current), -1, 1) : > longer object length is not a multiple of shorter object length > Warning in width_y * runif(nrow(current), -1, 1) : > longer object length is not a multiple of shorter object length >
Aaron Lun (04:49:07): > This is ajitterPointsfailure but that code hasn’t changed in two years!
Aaron Lun (04:50:57): > And running the code locally from the code tracker is fine.
Kevin Rue-Albrecht (04:53:25): > perhaps there’s something about commands stores not being evaluated, or evaluated twice even maybe?
Aaron Lun (04:57:27): > It seemsmergeis now tossing outNAs in the keys. I wonder when this happened.
Kevin Rue-Albrecht (04:58:28): > no clue
Aaron Lun (05:00:43): > Maybe it always happened.
Aaron Lun (05:07:27): > fixed.
Kevin Rue-Albrecht (05:08:46): > nice. thanks
Charlotte Soneson (11:03:59): > I’m going through the new version. In addition to the issue and PR in the repo, a couple of comments: > * The default for the heatmap isClusterDistanceFeatures = "spearman"andClusterMethodFeatures = "ward.D2". However, I think the Ward linkage is to be used together with the Euclidean distance (see e.g.https://www.ncbi.nlm.nih.gov/pubmed/28085891), so I would suggest to change the default combination. > * The panel widths (and some other things) really have to be integers (e.g.,RedDimPlot(PanelWidth = 2)will give an error). Maybe this is fine, but I just wanted to bring it up anyway since it may be a source of confusion.
Charlotte Soneson (11:05:57): > I also started converting the modes iniSEEu-modeEmptyanmodeGatingare done, Michael said he’d be happy to take a look at hismodeReducedDim
Aaron Lun (13:40:29): > I guess the panel dims could be a bit more relaxed if we made themnumericand coerced them to integer later, but that would be a bit of a pain on our side.
Aaron Lun (13:41:12): > Or, auto-coerce them ininitialize,Panel-method.
Aaron Lun (13:45:37): > Why is codecov being such a pain?
Kevin Rue-Albrecht (13:47:53): > what’s wrong? looks like to me it’s doing its job
Aaron Lun (13:48:14): > It’s blocking 357.
Kevin Rue-Albrecht (13:50:03): > well, 80% of the new code isn’t covered, so …
Kevin Rue-Albrecht (13:50:10): > it’s complaining
Aaron Lun (13:50:20): > yeah, but it shouldn’t block as long as travis passes.
Aaron Lun (13:50:44): > in the past it just said, ok, you can merge now.
Kevin Rue-Albrecht (13:50:52): > that’s how one ends up with decreasing code coverage
Aaron Lun (13:51:28): > But these are docfixes.
Kevin Rue-Albrecht (13:51:47): > let me check, there has to be some new code in there
Aaron Lun (13:52:58): > also, see if you can reproduce the hinton plot issue.
Kevin Rue-Albrecht (13:57:51): > well i don’t get why 357, but the last commit onmastersays - File (PNG): image.png
Kevin Rue-Albrecht (13:58:27): > anyway, I’m runningpackage_coverage()now and I’ll see what needs fixing
Kevin Rue-Albrecht (14:02:15): > alright I’ve got the report of uncovered lines
Kevin Rue-Albrecht (14:02:41): > that said, it’s bereallygreat if future PR could cover new code:https://codecov.io/gh/iSEE/iSEE/tree/outputs/R - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Aaron Lun (14:11:45): > Just renewed my lease.
Aaron Lun (14:11:51): > 3166/mo.
Aaron Lun (14:11:58): > Pretty cheap, no rent increase.
Kevin Rue-Albrecht (14:21:05): > alright - running coverage locally to check that i’ve got it all
Kevin Rue-Albrecht (14:31:06): > Hm. Not yet. > In the meantime, two related points: > * We should really agree (and probably document in the wiki) recommended testing guidelines. In the rush of writing tests whenever I get the chance, I’m writing stuff that “works”, but not really consolidating repeated routines into helpers likemimic_live_app > e.g. setting a minimalmemoryoften looks like that: > > ComplexHeatmapPlot1 <- ComplexHeatmapPlot(PanelId=1L) > sce <- .cacheCommonInfo(ComplexHeatmapPlot1, sce) > ComplexHeatmapPlot1 <- .refineParameters(ComplexHeatmapPlot1, sce) > memory <- list(ComplexHeatmapPlot1=ComplexHeatmapPlot1) > pObjects <- mimic_live_app(sce, memory) > > > * The act of writing unit tests for code I haven’t written is useful in that it forces me to read and understand the relevant code to design the tests. That said, it’s not really the most time efficient way, given that I don’t necessarily know all the thinking behind the code. I’m repeating myself, but we should really try and write the unit tests alongside the code, not after.
Kevin Rue-Albrecht (14:36:00): > lolhttps://github.com/iSEE/iSEE/pull/359/checks?check_run_id=433776270 - File (PNG): image.png
Kevin Rue-Albrecht (14:36:43): > not sure if failed -> error is improvement or deterioration
Kevin Rue-Albrecht (14:50:22): > alright, 357 passes again, it’s funny (=weird) that it was blocked by a decrease of coverage introduced by an earlier PR though
Aaron Lun (16:39:28): > Does the hinton plot reproduce on your end?
Kevin Rue-Albrecht (17:47:51): > on themasterpulled right now, it is OK - File (PNG): image.png
Aaron Lun (17:49:33): > I was referring to the brushing issue with my MRE.
Aaron Lun (17:49:46): > If you remember who our contact was, maybe you can post it on theshinyrepo.
Kevin Rue-Albrecht (17:51:25): > oh right - indeed I still can’t draw a brush
Aaron Lun (18:58:42): > Awww yeah. Check out the example in?createLandingPagefor 360.
Aaron Lun (19:02:54): > Though on that note, CHMP is giving some errors like: > > Error in : If input is not a matrix, it should be a simple vector. >
Aaron Lun (19:41:28): > Aw gawd.
Aaron Lun (19:41:33): > I don’t want to review any more papers.
2020-02-09
Kevin Rue-Albrecht (06:14:19): > Proofreading the book makes me think that in iSEEu we could have more Table panels subclasses dedicated to internal metadata. That would ultimately avoid the need for .sanitize_SE_input() to flatten nested DataFrames. Instead, external package developers could add/maintain custom panels accessing their own nested/internal DataFrame. > Caveat: information would be split over multiple table panels > Positive: it would help each table panel to keep a reasonable number of columns and avoid long column names generated by the flattening
Kevin Rue-Albrecht (06:14:38): > https://isee.github.io/iSEE-book/server.html - Attachment (isee.github.io): Chapter 3 The app server | Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Aaron Lun (14:23:31): > Huh?
Aaron Lun (14:23:43): > Internal metadata?
Kevin Rue-Albrecht (14:25:51): > I know you’re planning to move size factors. But I’m thinking about those tables that are parallel to coldata and rowdata
Kevin Rue-Albrecht (14:27:12): > At the moment, sanitize_se is pasting them to the colData and rowData. Perhaps they could stand out better as separate panel sub classes
Aaron Lun (14:30:36): > If they’re internal, it is highly unlikely that people should be touching them directly.
Aaron Lun (14:30:49): > So yes, I would say that they should not be exposed.
Aaron Lun (14:32:14): > Incidentally, the heatmap-command generating code needs some fragmentation in the same vein that the plot-generating code is fragmented.
Kevin Rue-Albrecht (14:35:03): > Command store I imagine?
Aaron Lun (14:35:28): > I purged the command store, because it wasn’t actually getting used if we fragment the function into smaller functions.
Aaron Lun (14:35:54): > Also, why is there > > # If there is a matrix to work with at all > if (all(dim(plot_env[["plot.data"]]) > 0)) { >
Aaron Lun (14:35:55): > ?
Kevin Rue-Albrecht (14:38:15): > Something was crashing if the matrix didn’t have rows or column. Possibly the side annotations
Kevin Rue-Albrecht (14:42:41): > Or the scaling. Well you get the picture. Some aspect of the heat map couldn’t be drawn so there is no point generating or evaluating code beyond that point.
Kevin Rue-Albrecht (14:43:23): > Anyway, I’m not sentimental with the code if you want to tinker with it to try other behaviors
Aaron Lun (15:29:42): > I did some reorganization and noticed a few things.
Aaron Lun (15:30:05): > I’m going to work on the custom panels today, so the heatmap code is all yours again.
Kevin Rue-Albrecht (15:30:55): > > the heatmap code is all yours again. > meaning? I didn’t have immediate plans for it
Aaron Lun (16:42:30): > cehck out some of the comments
Aaron Lun (21:52:03): > I’ve fixed most of the issues but the heatmap tests are now broken. I need to get onto restoring this custom thing so I probably can’t get to fixing the tests.
2020-02-10
Aaron Lun (02:46:34): > O
Aaron Lun (02:46:34): > M
Aaron Lun (02:46:35): > G
Aaron Lun (02:46:36): > https://github.com/iSEE/iSEE/pull/362
Aaron Lun (02:46:53): > You’re probably thinking “how?”
Aaron Lun (02:47:01): > but you won’t be able to help yourself saying “wow”
Federico Marini (04:17:28): > I’mma say wowzer indeed
Federico Marini (04:17:39): > can’t keep up with all the cool stuff, damn
Kevin Rue-Albrecht (13:33:24): > you can leave me theheatmapbranch - I’ll clean it up
Federico Marini (14:54:04): - File (PNG): image.png
Federico Marini (14:54:23): > wut?
Charlotte Soneson (15:19:24): > It should apparently be back up soon
Charlotte Soneson (15:21:08) (in thread): > According tohttps://twitter.com/sinabooeshaghi/status/1226913124403441665 - Attachment (twitter): Attachment > Excited to say that Seurat should be back on CRAN imminently, per @satijalab. We will update our notebooks once it is.
Federico Marini (16:00:57): > aha, ok
Federico Marini (16:01:09): > wondering if it is because of their 460 dependencies
Kevin Rue-Albrecht (16:02:20): > that’s exactly what I was wondering
Kevin Rue-Albrecht (16:03:21): > Where did I recently read that CRAN is planning to put an upper limit on the number of (non-base) dependencies?
Aaron Lun (16:19:43): > Merge yours. Also, you need to fix your name/email at some point, we’re getting mystery commits from “Your Name”.
Kevin Rue-Albrecht (16:21:51): > .gitconfig fixed
Kevin Rue-Albrecht (16:23:15): - File (PNG): image.png
Aaron Lun (16:42:00): > If you want to play around with the new functions, can you resurrect the custom panel vignette with the new fns?
Aaron Lun (16:42:07): > Also need to add a section pointing to the book.
Kevin Rue-Albrecht (16:45:46): > I’m rather looking into the mechanism to set default panel options. I’ve just had a vision myself, and started looking into theknitr::opt_chunk$get/setinfrastructure. See where I’m going with that?
Kevin Rue-Albrecht (18:01:40): > Enjoy branchdefaults > > opts_panel$set(downsample=TRUE, downsample.res=50, selected.color="orange") > iSEE(sce) >
Kevin Rue-Albrecht (18:01:48): > > opts_panel$restore() > iSEE(sce) >
Kevin Rue-Albrecht (18:02:02): > > opts_panel$get() >
Kevin Rue-Albrecht (18:02:55): > Refinements and unit tests will come after feedback
2020-02-11
Aaron Lun (02:27:19): > @Kevin Rue-Albrechtlooks good but this should be a camelCase thing.
Aaron Lun (02:29:13): > Anyway, the custom panel creator will now do a limited AUTODETECTION of the appropriate UI for a custom function’s arguments.
Aaron Lun (02:31:40) (in thread): > e.g.iSEEOptions.
Aaron Lun (02:32:13): > That’s right. You now only need to pass in a function, and at least for simple flags and options, the function will figure out the appropriate UI element.
Kevin Rue-Albrecht (17:07:46): > Alright. Among the misc fanciful things, we have now automated DockerHub builds fromRELEASE_x_xbranches (≥3.10) in addition to themasterbranch
Kevin Rue-Albrecht (17:08:06): > More info athttps://github.com/iSEE/iSEE/wiki/Docker-image
2020-02-12
Kevin Rue-Albrecht (15:38:18): > FYI I’m ondefaults.
Aaron Lun (15:39:00): > FINISH HIM.
Aaron Lun (15:39:05): > :-1:
Kevin Rue-Albrecht (15:40:19): > I also added custom-related unit tests covering most of what was left, but I’m stuck at figuring out.execute_custom_function()
Aaron Lun (15:40:48): > That’s the stuff inside the observer.
Aaron Lun (15:41:06): > It’s the layer between the auto-generated UI and the user-supplied FUN.
Kevin Rue-Albrecht (15:41:28): > we just need to cover the case where there are incoming row/column selections
Kevin Rue-Albrecht (15:42:41): > Let me see if I can get a decent test in the next 30min. Otherwise I might let you slam dunk that one
Kevin Rue-Albrecht (15:56:29): > Hmm. > I imagine we can expect scenarios wherefn_argsischaracter(0), leading to the following call, right? > > "gg <- fn_name(se, NULL, NULL, )" >
Kevin Rue-Albrecht (15:56:49): > Just checking, so that I update the function to optionally trim that last,
Aaron Lun (15:57:56): > Probably, yes.
Kevin Rue-Albrecht (16:13:42): > FYI, we’re good: 100%.
Kevin Rue-Albrecht (16:13:46): > Just running through CMD check
Aaron Lun (16:13:57): > sweet
Aaron Lun (16:14:05): > Did you enjoy the awesomeness of the auto-UI generator?
Aaron Lun (16:14:09): > It’s pretty damn awesome.
Aaron Lun (16:14:30): > I wonder whether we could do that for more stuff.
Kevin Rue-Albrecht (16:15:53): > Ehm ehm. I… uh… just focused on getting the PR ready for merge. Haven’t actually launched a GUI yet:sweat_smile:
Kevin Rue-Albrecht (16:16:30): > Speaking of which. One-phrase description for the NEWS file entry 1.99.0 ?
Kevin Rue-Albrecht (16:16:42): > Aside from “hold on to your hats”
Kevin Rue-Albrecht (16:18:38): > Looking at closed PRs, “Refactor to use S4 classes” would do. Or elaborating a bit “Refactor to panel implementation as S4 classes”
Aaron Lun (16:18:51): > Yeah, that would be fine.
Kevin Rue-Albrecht (16:22:25): > Another one: “Allow customization of landing pages (from callingiSEE()in no-SE mode) for enterprise deployments.” > Care to rephrase a bit?
Aaron Lun (16:22:48): > That’ll do.
Kevin Rue-Albrecht (16:33:59): > OMG, the usage ofcreateCustomPlot()isridiculouslysimple:no_mouth:
Aaron Lun (16:38:01): > Yes. Yes it is.
Kevin Rue-Albrecht (16:38:26): > Still.. it feels like it would take all 4 of us on a 6 month sabbatical to write a book covering the current state of the features
Aaron Lun (16:39:01): > IMO the book only needs to cover the extensions.
Kevin Rue-Albrecht (16:40:23): > Indeed. That is a DRY area. Otherwise, there’ll be too many places to update information if we change anything “core”
Kevin Rue-Albrecht (16:47:40): > OK, I think I got all of the NEWS. I was so busy catching up on yours that I forgot one of mines
Kevin Rue-Albrecht (16:48:30): > btw, I’ve kicked off a section 1.99.1 but I haven’t bumped in DESCRIPTION yet
2020-02-13
Kevin Rue-Albrecht (10:33:25): > Might be worth rephrasing the following in the “basic” vignette: > > Note that all computations (e.g., dimension reduction, clustering) must be performedbeforepassing the object to theiSEEfunction. > We don’t want to confuse users too soon with the notion of custom panels, but at the same time “must” might scare off prospective users before they get to learn that custom panels can compute things on the fly.
Aaron Lun (13:13:41): > k
Aaron Lun (13:13:50): > also note to self; need to clean up custom landing page docs.
Kevin Rue-Albrecht (17:14:42): > Speaking of which - you can check out the updated man page of?iSEEOptions
Kevin Rue-Albrecht (17:15:45): > I think we’re good to merge when Travis completes
Kevin Rue-Albrecht (17:25:01): > Merged.
2020-02-14
Andrew Skelton (05:06:36): > @Andrew Skelton has joined the channel
Aaron Lun (11:49:57): > @Kevin Rue-AlbrechtProbably keep pushing on iSEEu. The book and docs will improve themselves only once we try to actually do extensions. I will take care of improving the internal docs, but we need some bodies pushing hard on iSEEu; we can reasonably aim to get it into BioC for the next release.
Aaron Lun (11:50:08): > Tagging@Charlotte Soneson@Federico Marinias well.
Aaron Lun (11:50:36): > Low hanging fruit; a nice MA and volcano plot for viewingprecomputedDE results in therowData.
Kevin Rue-Albrecht (12:33:56): > I’m cleaning up/simplifying the settings of the iSEE organization and repo. If you notice any issue pushing to it, let me know.
Kevin Rue-Albrecht (12:35:53): > As Team “isee-core” we already have Admin access to it. So there was no point having individual “Write” access (Admin includes Write anyway)
Aaron Lun (12:36:04): > uh. ok.
2020-02-15
Kevin Rue-Albrecht (12:31:38): > While we’re in a renaming frenzy, I noticed that we haveSampleAssayPlotyetColumnDataPlot. > Shouldn’t “Column” be replaced by “Sample” everywhere we can?, i.e.SampleDataPlot? > Same thing forFeatureAssayPlotandRowDataPlot, we could have insteadFeatureDataPlot
Kevin Rue-Albrecht (13:07:13): > Along the same lines,RedDimPlotgot expanded toReducedDimPlotbut notReducedDimensionPlot.
Aaron Lun (14:15:27): > That sounds reasonable.
Aaron Lun (14:15:34): > if you want to do it.
Kevin Rue-Albrecht (14:16:29): > I don’t mind doing it. Just checking before I bother with a PR.
Kevin Rue-Albrecht (14:22:57): > different topic, playing with colors for new panels, I noticed that panel color is passed as is to JavaScript, which doesn’t work well with R-specific colors e.g. “mediumpurple2” (simpler names like “darkblue” do work)
Aaron Lun (14:23:16): > Should mention that it should be the rgb code.
Aaron Lun (14:23:26): > something like “#6af61cd”
Kevin Rue-Albrecht (14:23:34): > makes me wonder whether we could throw in agplots::col2hex()call, though it’s a 2-liner
Kevin Rue-Albrecht (14:23:42): > > > gplots::col2hex > function (cname) > { > colMat <- col2rgb(cname) > rgb(red = colMat[1, ]/255, green = colMat[2, ]/255, blue = colMat[3, > ]/255) > } >
Aaron Lun (14:23:53): > I think we should just expect people to pass in the hex.
Aaron Lun (14:24:02): > Just document it in?.panelColor.
Aaron Lun (14:38:13): > Otherwise we’d need to figure out whether a string was a hex or not, which is at least an extra line.
Kevin Rue-Albrecht (15:00:34): > Incoming branchfixes-misc
Aaron Lun (16:28:29): > Huh. I was thinking you’d rename things the other way.l
Kevin Rue-Albrecht (17:06:52): > Oh. I kinda find more meaningful thinking in terms of experimental entities than dimension names.
Aaron Lun (19:07:20): > That’s fine, but we will have to sync all the argument names as well.
Aaron Lun (19:07:51): > We have a LOT of “Row” stuff, e.g., SelectRowSource, SelectRowTable…
Aaron Lun (19:46:06): > I would suggest the following rule: use “Feature” or “Sample” when we’re talking about a single row or column, use “Row” or “Column” when we’re talking about multiple rows or columns.
Aaron Lun (19:46:23): > Otherwise we end up with a “FeatureAssayPlot” that cannot transmit to a “FeatureDataPlot”.
Aaron Lun (19:49:39): > This would also solve the naming problem for some the selection-related aesthetic parameters compared to the multiple selection parameters.
Aaron Lun (19:50:06): > For example, we could now haveColorByFeatureSource(replacingColorByRowTable), which is pretty distinguishable fromSelectRowSource.
2020-02-16
Aaron Lun (04:05:15): > Also, thatsizeFactorsrelated failure needs fixing with the latest SCE update.
Kevin Rue-Albrecht (12:15:54): > Little prototype of aschexpanel inhttps://github.com/iSEE/iSEEu/blob/panel-schex/R/panel_schex.R
Kevin Rue-Albrecht (12:18:51): > Tagging@saskiawho motivated the progress on custom panel in the first place
Kevin Rue-Albrecht (12:30:15): > As always, setting up a prototype is the least amount of work. > There are quite a few corner cases that still need to be caught and handled to avoid tripping theschexfunctions > In particular, do not select categorical colData (the panel is hardcoded to compute themeanof values in each hex)
Kevin Rue-Albrecht (14:31:32): > On second thought, I’ve reverted the earlier changes and now it’s all Row* and Column* > > exportClasses(ColumnAssayPlot) > exportClasses(ColumnDataPlot) > exportClasses(ColumnDataTable) > exportClasses(ColumnDotPlot) > exportClasses(ColumnTable) > exportClasses(ComplexHeatmapPlot) > exportClasses(DotPlot) > exportClasses(ExperimentColorMap) > exportClasses(Panel) > exportClasses(ReducedDimensionPlot) > exportClasses(RowAssayPlot) > exportClasses(RowDataPlot) > exportClasses(RowDataTable) > exportClasses(RowDotPlot) > exportClasses(RowTable) > exportClasses(Table) > - Attachment: Attachment > Huh. I was thinking you’d rename things the other way.l
Aaron Lun (14:31:54): > Well, I was thinking that the old name was actually fine, see my proposed rule above.
Aaron Lun (14:32:22): > As in, we should continue calling them FeatureAssay, because the feature is on the x-axis; they’re not points.
Aaron Lun (14:32:36): > We should then align all of our constants to this distinction.
Kevin Rue-Albrecht (14:37:51): > I saw the proposed rule. I’m not sure what will be easier from a user perspective. I just think that being consistent within panel class names would be a good start irrespective of how we write the man pages.
Kevin Rue-Albrecht (14:39:21): > You’re right that argument names probably need to be reviewed too. A list of them might be nice to maintain somewhere for a condensed look, even if it’s redundant to scrolling through AllClasses.R
Kevin Rue-Albrecht (14:40:21): > Anyway - I’ll stop messing with the code base and think more about it. Happy to watch you take it one way or another.
Aaron Lun (14:41:25): > Well, I would say that “Row data plot” and “Row assay plot” are too close for comfort.
Aaron Lun (14:41:48): > Especially since their contracts and modes of operation are very distinct.
Kevin Rue-Albrecht (14:42:59): > I’ve often thought that “Row metadata plot” would be more intuitive/explicit
Aaron Lun (14:43:48): > I think we should just roll back those changes to where we were before.
Kevin Rue-Albrecht (14:58:29): > fair enough
Aaron Lun (14:59:00): > are they clean enough to just revert?
Kevin Rue-Albrecht (14:59:56): > that’s what I’m looking at right now, in theory yes
Aaron Lun (15:00:01): > need to eat, back in 1
Aaron Lun (16:06:24): > well, I thought the ReducedDimension rename was okay.
Aaron Lun (16:06:28): > It was just the others.
Kevin Rue-Albrecht (17:00:42): > Done. We need to work on communication:stuck_out_tongue:
2020-02-17
Aaron Lun (01:12:40): > If you can fix the coverage and merge and push to BioC, I will update the iSEE chapter in the book.
Aaron Lun (01:55:28): > Also, there’s a ColorByFeatNameAssay that should be ColorByFeatureNameAssay in the docs.
Aaron Lun (01:55:51): > I would guess that’s the same for any SampNameAssay that might be floating around.
Aaron Lun (01:59:39): > I also need some idea for another usage example, we only have two in the current chapter.
Kevin Rue-Albrecht (11:05:02): > I’ve got the doc updates covered. What about showcasing some SingleR functionality in iSEE ? Perhaps a barplot of predictions, with selections making part of the bars transparent of without color.
Kevin Rue-Albrecht (12:27:04): > Illustration: > > library(ggplot2) > n_cells <- 1000L > > plot.data <- data.frame( > cell.type=sample(head(LETTERS), n_cells, TRUE), > selected=sample(c(TRUE, FALSE), n_cells, TRUE) > ) > > # SelectionEffect = none > ggplot() + > geom_bar(aes(cell.type, fill=cell.type), plot.data, color="black") + > theme_minimal() > > # SelectionEffect = color > ggplot() + > geom_bar(aes(cell.type), plot.data, color="black", fill="white") + > geom_bar(aes(cell.type, fill=cell.type), subset(plot.data, selected), color="black") + > theme_minimal() > > # SelectionEffect = transparency > ggplot(plot.data) + > geom_bar(aes(cell.type, fill=cell.type), plot.data, color="black", alpha=0.2) + > geom_bar(aes(cell.type, fill=cell.type), subset(plot.data, selected), color="black") + > theme_minimal() >
Aaron Lun (12:27:22): > What am I meant to be looking at?
Kevin Rue-Albrecht (12:27:31): > #cool-vis:stuck_out_tongue: - File (PNG): image.png
Kevin Rue-Albrecht (12:29:00): > The full bar could be showing the composition of the entire dataset, while highlighting the incoming selection
Aaron Lun (12:29:24): > Is this a custom panel?
Kevin Rue-Albrecht (12:29:44): > Not sure how useful in “production”, but I’m just brainstorming ideas
Kevin Rue-Albrecht (12:30:30): > Right now it’s nothing else than a plot, but you were asking for “usage examples”
Kevin Rue-Albrecht (12:30:51): > We don’t have bar plots yet, could be something
Aaron Lun (12:31:06): > Usage examples with our current panels.
Kevin Rue-Albrecht (12:34:06): > Mmh. As demonstrated by theschexpanel, I’m a bit stuck when it comes to extending existing panels. Although in that case, it’s probably because I should have extended a virtual class instead of ReducedDimensionPlot
Kevin Rue-Albrecht (12:35:45): > Oh wait. That makes me realize that I haven’t re-reverted ReducedDimensionPlot. Ugh.
Aaron Lun (12:35:58): > Well, I thought it was fine to extend RDP for Hex.
Kevin Rue-Albrecht (12:36:00): > (i.e., putting it back to the fully extended name)
Aaron Lun (12:36:35): > It’s just that you’re intercepting the aesth calls.
Kevin Rue-Albrecht (12:36:46): > Thing is that if it extends RDP, it needs to access slot names using the triple colon. Do we really want to encourage that?
Aaron Lun (12:37:04): > Why do you need to access slot names via the triple colon?
Aaron Lun (12:37:11): > The slot names are exposed in the class definition.
Kevin Rue-Albrecht (12:39:08): > Uh.. wait. How do you suggest accessingiSEE:::.redDimTypethen, in a way that the hex panel is robust to (potential) future changes to the slot name?
Aaron Lun (12:40:04): > The slot names are already exposed to the user.
Aaron Lun (12:40:13): > You might as well expose them to devs.
Kevin Rue-Albrecht (12:40:58): > Oh. so you think they should just callx[["Type"]]then?
Aaron Lun (12:41:09): > Yes.
Aaron Lun (12:41:31): > Users are doing it anyway, so devs doing it is the least of our problems.
Aaron Lun (12:42:13): > In any case, if we rename a slot, we can just override[[to switch between names.
Kevin Rue-Albrecht (12:44:50): > Right. So if we ever change “Type” to anything else, they’ll have to update accordingly. I thought they could point to the variable name, but indeed users/developers should not have to worry about our internal variable names. Nevermind
Aaron Lun (12:45:52): > It’ll be a soft update with deprecations as long as they are accessing it via[[.
Kevin Rue-Albrecht (16:58:52): > panel colors moved toiSEEOptions
Kevin Rue-Albrecht (17:07:01): > Ugh. Luckily, commits are cheap
Kevin Rue-Albrecht (17:17:35): > I guess all we miss now is the version bump when we’re ready. NEWS look comprehensive and uptodate
Aaron Lun (23:59:40): > FINISH HIM
2020-02-18
Kevin Rue-Albrecht (05:12:52): > One leftover:SampleResshould be renamed. First “Sample” is confusing and if kept should be “Downsample“.DownsampleResolutionsounds most appropriate even though quite long (althoughColorByFeatureNameAssayseems to be our champion)
Aaron Lun (11:29:45): > fine with me.
Kevin Rue-Albrecht (11:37:45): > I’ll wrap it up tonight. Haven’t spotted anything else that needs immediate attention.
Aaron Lun (11:38:39): > Make sure you catch it in the renaming bundle indefaults.R.
Aaron Lun (11:39:16): > Also choose a better name for the options file instead ofdefaults2.R. LIkeglobals.R. I would say useiSEEOptions.R
Kevin Rue-Albrecht (11:44:58) (in thread): > Nice catch. Thanks
Aaron Lun (11:45:29): > but that’s a bad name for the autocompleter, so go with something else.
Kevin Rue-Albrecht (11:46:09): > ah, you mean users will autocomplete to “Downsample” instead of the full name?
Kevin Rue-Albrecht (11:47:02): > Closer to the original naming, it could be “SamplingResolution” (?)
Aaron Lun (11:47:50): > No, I was referring to theiSEEOption.R
Aaron Lun (11:48:01): > This is bad for auto-complete because it runs intoiSEE-main.R.
Aaron Lun (11:48:13): > so I have to press tab twice in the space ot two chraracters.
Kevin Rue-Albrecht (11:49:10) (in thread): > Might be useful to kick off threads when we’re talking about multiple things in parallel.
Kevin Rue-Albrecht (11:50:29) (in thread): > What aboutpkgOptions.R?
Kevin Rue-Albrecht (11:50:46) (in thread): > OrglobalOptions.R
Aaron Lun (11:51:19) (in thread): > Well, the general philosphy is that any end-user-visible (non-generic) function is put in the file with the same name.
Kevin Rue-Albrecht (11:52:09) (in thread): > Ok… so do we stick withiSEEOption.Rdespite the keyboard hickup ?
Aaron Lun (11:53:02) (in thread): > I guess so.
Aaron Lun (11:53:08) (in thread): > there’s an S on the end.
Aaron Lun (11:53:37) (in thread): > You could also name itiSEE-options.R, which is close enough I guess.
Kevin Rue-Albrecht (11:53:59) (in thread): > Yep. My bad. A risk of typing instead of copy pasting
Kevin Rue-Albrecht (11:55:26) (in thread): > Up to you. We’re stuck with the camelCase, otherwise I was happy with the originalknitr::opts_chunkappearance
Kevin Rue-Albrecht (11:55:47) (in thread): > I do likeiSEE-options.R, tbh
Kevin Rue-Albrecht (12:22:12): > ’nother one: I noticed that the panel full names are still “Row statistics table” when the class are “Row data table”. We should change that too, right?
Aaron Lun (12:22:47): > yes
Kevin Rue-Albrecht (13:59:48): > Almost ready to push. Had to update some unit tests that were referring to “statistics table”
Kevin Rue-Albrecht (14:19:53): > Pushed to GH. Let’s hope we can close that PR tonight. [and push upstream too]
Kevin Rue-Albrecht (14:21:16): > (It did pass CMD check on my laptop)
Kevin Rue-Albrecht (15:49:44): > alright - I’m sick of checking for typos in the package, when this Travis build completes I’ll just merge and push upstream
Kevin Rue-Albrecht (15:50:17): > at least it’ll get iSEEu and the book going while we spot other potential fixes
Kevin Rue-Albrecht (15:56:15): > alright - pushed upstream
2020-02-19
Aaron Lun (13:24:44): > I don’t seepanel.colorin?iSEEOptions.
Kevin Rue-Albrecht (14:37:32): > open an issue:stuck_out_tongue:
Kevin Rue-Albrecht (14:38:01): > PS: sorry, I forgot. too many things to keep track of lately
Kevin Rue-Albrecht (14:41:27): > I also noticed too late that I haven’t changed the default clustering distance and method following Charlotte’s suggestion. Also I’m not sure whether there is a better way to order those options in the selectize (some kind of popularity, or just alphabetically)
2020-02-20
Charlotte Soneson (02:17:04): > Seems something is up with the code tracker…it prints out the whole creation of the SE: - File (PNG): Screen Shot 2020-02-20 at 08.16.31.png
Aaron Lun (02:18:23): > Probably something is broken witht he initialdeparse(substitute(se)). How are you callingiSEE()?
Charlotte Soneson (02:20:17): > I was calling themodeGating()fromiSEEu. It looks fine withmodeEmpty()though. Let me see if I can find where it breaks.
Charlotte Soneson (02:29:35): > Hm, ok.iSEE(se = se)works, butdo.call(iSEE, list(se = se))doesn’t
Aaron Lun (02:32:28): > Yes, that is well known.
Aaron Lun (02:32:36): > do.callbreaks a lot of the back-calling machinery.
Aaron Lun (02:32:56): > And there’s no way for a function to know whether it was called bydo.callor normally, so we can’t protect against it internally.
Charlotte Soneson (03:04:41): > What do you think about having the default ofextrabeing all panel types rather than only the ones ininitial?
Kevin Rue-Albrecht (07:39:20) (in thread): > I did think that too at some point.
Charlotte Soneson (08:24:31) (in thread): > I think it would be beneficial for multiple reasons: > * you don’t need to remember which are the available panel types > * it is consistent with the behaviour of previous versions > * I don’t see many occasions when you want to limit the set of possible panel types > * themodeEmpty()becomes pretty pointless ifextrais not specified…
Kevin Rue-Albrecht (08:36:05) (in thread): > I haven’t tested the feature recently, I’m not sure howextradeals with multiple instances of the same panel class, or if a preconfigured instance is given asextra(do all new instances get initialized as preconfigured, or does only the class matter)
Kevin Rue-Albrecht (08:38:56) (in thread): > But it’d be nice if at least all the builtin panels were included as extra with 0 preconfiguration (minus panel classes that are not available for a givense). Then any “real” extra would be concatenated to that “default” set (minus those not available again). And i guessextrawould have to contain a single “prototype” of each class
Charlotte Soneson (08:41:09) (in thread): > Yes. I think it would be fine if the default was a list with the eight built-in panels. If people want to configure it or explicitly exclude some panels, that’s fine, and they’ll anyway have to change the defaults. I just felt that if you didn’t specify anything in particular, you should have access to the standard panels by default.
Aaron Lun (11:34:01) (in thread): > I suppose. Should be an easy fix.
2020-02-21
Aaron Lun (23:54:06) (in thread): > Okay, I now agree with this, because it was a pain to actually have to fill this out.
Aaron Lun (23:54:47) (in thread): > FYI iSEE will always clone the “First” instance of a panel. Panels ininitialare given priority (perhaps they shouldn’t be) and then the first panel inextrais used.
2020-02-23
Kevin Rue-Albrecht (14:04:03): > shinyapps.iousage in the last 3 months - File (PNG): image.png
Kevin Rue-Albrecht (15:38:30): > Automated Travis build and deploy is set up for iSEE-book.masteris now protected, so that applies when sides branches are merged
Aaron Lun (15:41:04): > k
Aaron Lun (15:42:43): > Who is working on iSEEu right now?
Aaron Lun (15:42:58): > Probably want to pick that up if we want to get in before the next release.
Kevin Rue-Albrecht (15:47:14): > I haven’t yet, and March is going to be tough
Aaron Lun (15:48:58): > can you just copy and paste the current examples from the book?
Aaron Lun (15:49:19): > while@Charlotte Sonesoncan deal with the stuff that’s already there.
Aaron Lun (15:50:32): > And I will add the MA and volcano plots.
Charlotte Soneson (16:30:45): > modeEmptyandmodeGatingare updated. Michael is taking care ofmodeReducedDim.
Kevin Rue-Albrecht (17:05:39): > One thing that worries me a bit about the scope ofiSEEuis the potentially extensive list of dependencies that it could accumulated, depending on the range of panels that could make it in there. Anyone thought of that yet?
Kevin Rue-Albrecht (17:07:02): > I thought about that when I looked into a potentialschexcandidate, considering that it > > Depends: > SingleCellExperiment (>= 1.7.4), > Seurat, > > (https://github.com/SaskiaFreytag/schex/blob/master/DESCRIPTION)
Aaron Lun (17:29:28): > Yes, I have thought about that.
Aaron Lun (17:29:48): > The classes can just Suggests things, y’know.
Aaron Lun (17:30:38): > I would, in fact, leave schex off the list until Saskia decides to contribute something.
Kevin Rue-Albrecht (17:34:07) (in thread): > That’s also what I’ve decided. Which is also why I’ve put it on the backburner until I have proper time to do it well
Kevin Rue-Albrecht (17:36:48): > how long have I dreamt of seeing the book build and deploy without manual intervention:party_parrot:
Kevin Rue-Albrecht (17:37:51): > Luckily we don’t have big data to process like it’s the case for OSCA. Let’s keep it that way:grimacing:
Kevin Rue-Albrecht (17:40:16): > @Federico Mariniwhen you get bored of stickers, we’ll need a cover for the book:yum:
Aaron Lun (17:52:26) (in thread): > just give me some copy paste goodness
Kevin Rue-Albrecht (17:54:26) (in thread): > not tonight though
Kevin Rue-Albrecht (17:57:42): > On a separate note, I’ve opened branchdocfixto documentpanel.color(iSEEOptions), but I’m running into a scater issue it seemshttps://travis-ci.com/iSEE/iSEE/builds/150208247 - Attachment (travis-ci.com): Travis CI - Test and Deploy with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (17:58:15): > What I don’t get is how iSEE-book installed all dependencies just fine, while iSEE stumbles onto that
Kevin Rue-Albrecht (17:58:25): > perhaps a Travis package cache issue
Kevin Rue-Albrecht (17:58:46): > anyway, I’m thinking of giving it a couple of days before poking travis again
Aaron Lun (22:08:40): > Just fixed a nasty bug, seefixesand check that it still plays nice.
Aaron Lun (22:09:09): > Took 3 hours to figure out.
Aaron Lun (23:07:14): > Fixed an edge-case infinite recursion bug. Needed to usereactlogto inspect the reactive graph to figure out what was firing.
Aaron Lun (23:07:47): > FYI the graph for just a single panel is ridiculous.
2020-02-24
Aaron Lun (00:55:43): > Dunno what’s going wrong with Travis.scater shouldn’t have any problems.
Charlotte Soneson (02:54:16): > I think it’s inS4Vectors- I have that error on travis also for other packages that depend on it. There, clearing the cache seemed to help
saskia (05:14:10): > @saskia has joined the channel
Kevin Rue-Albrecht (06:32:49): > All clear on Travis. It was a cache thing for whatever reason.
Aaron Lun (20:07:19): > GOT MY BONUS
2020-02-25
Federico Marini (06:01:07): > What’s coming out of that?
Federico Marini (08:54:43): > nice creepy effect by getting an email from iSEE everything
Kevin Rue-Albrecht (08:58:52): > lol
Kevin Rue-Albrecht (09:01:17): > Sorry for the spam. I was going through zenodo for another thing and thought “hell why not”. It wouldn’t hurt either to archive a copy of each release on Zenodo. > I know one could get the version of a specific commit from the Bioc git, but I’m not sure whether it’s possible to install a package by version number, e.g. “iSEE 1.3.5”
Federico Marini (09:02:16): > We’d just need to tag the versions
Federico Marini (09:02:25): > then remotes would handle it
Kevin Rue-Albrecht (09:02:26): > BiocManager::install(version=)refers to theBioconductorversion
Federico Marini (09:02:46): > remotes::install_github("who/what@ref"), AFAIK
Kevin Rue-Albrecht (09:02:49): > But yeah, tagging on GH would work too
Kevin Rue-Albrecht (09:20:34): > On a separate note, I’ve volunteered to give a “Dataviz” session at the Big Data Institute code club in June 2020:https://bdicodeclub.netlify.com/
Federico Marini (09:21:44): > Excellent!
Aaron Lun (13:50:12) (in thread): > A lot.
Federico Marini (15:04:54) (in thread): > :slightly_smiling_face:Congrats, whatever the bonus was related to
2020-02-26
Federico Marini (06:36:58): > Someone missed iSEE -https://peerj.com/articles/8607/ - Attachment (PeerJ): SCelVis: exploratory single cell data analysis on the desktop and in the cloud > Background Single cell omics technologies present unique opportunities for biomedical and life sciences from lab to clinic, but the high dimensional nature of such data poses challenges for computational analysis and interpretation. Furthermore, FAIR data management as well as data privacy and security become crucial when working with clinical data, especially in cross-institutional and translational settings. Existing solutions are either bound to the desktop of one researcher or come with dependencies on vendor-specific technology for cloud storage or user authentication. Results To facilitate analysis and interpretation of single-cell data by users without bioinformatics expertise, we present SCelVis, a flexible, interactive and user-friendly app for web-based visualization of pre-processed single-cell data. Users can survey multiple interactive visualizations of their single cell expression data and cell annotation, define cell groups by filtering or manual selection and perform differential gene expression, and download raw or processed data for further offline analysis. SCelVis can be run both on the desktop and cloud systems, accepts input from local and various remote sources using standard and open protocols, and allows for hosting data in the cloud and locally. We test and validate our visualization using publicly available scRNA-seq data. Methods SCelVis is implemented in Python using Dash by Plotly. It is available as a standalone application as a Python package, via Conda/Bioconda and as a Docker image. All components are available as open source under the permissive MIT license and are based on open standards and interfaces, enabling further development and integration with third party pipelines and analysis components. The GitHub repository is https://github.com/bihealth/scelvis.
Federico Marini (06:38:22): > Wonder whether it is because we have too many features on top?
Kevin Rue-Albrecht (06:39:44): > https://peerj.com/articles/8607/ - File (PNG): image.png
Kevin Rue-Albrecht (06:40:32): > Obviously didn’t help peer review existing literature on the subject
Federico Marini (06:42:46): > They put cerebro but not us:disappointed:puah
Federico Marini (06:44:16): > meh, meh
Federico Marini (06:45:17): > they have an interactive tsne plot
Federico Marini (06:45:24): > ok I’mma stop here
Aaron Lun (10:50:12): > Damn. Slept in and missed my bus.
Aaron Lun (11:16:45): > Keep pushing. First panel subclass almost in iSEE.
Kevin Rue-Albrecht (11:46:06): > u
Aaron Lun (16:27:49): > Trying to set up an instance for work, but there’s too many damn bugs. - File (PNG): Screen Shot 2020-02-26 at 1.27.08 PM.png
Federico Marini (16:36:57): > We should implement a ggplot theme that makes it seem the screen is cracked when too many points are plotted
Federico Marini (16:37:15): > Guess it is even possible with cowplot
Kevin Rue-Albrecht (17:18:38): > the xkcd theme would be awesomehttps://cran.r-project.org/web/packages/xkcd/vignettes/xkcd-intro.pdf - File (PNG): image.png
2020-02-27
Kevin Rue-Albrecht (04:27:45): > @Charlotte Sonesonshall we PR and mergeisee2.0? seems like it would help all other branches to pass travis
Charlotte Soneson (04:29:59): > Yes, seems to make sense.
Kevin Rue-Albrecht (04:31:24): > Given that it passes Travis, I’d say thatmodeReducedDim.Rcan always be fixed later. It’s more important to bring the core functionality up to date, so that other panels in the package benefit from the fixes
Charlotte Soneson (04:32:24): > Yep
Kevin Rue-Albrecht (17:21:23): > Don’t hold your breath for a speedy panel development, but I’ve toyed with a RedDimHexPlot panel that usesgeom_hexdirectly instead ofschex. It’s a lot more work but it allows selection of underlying points because it has access toplot.data(as well asplot.data.all, etc.) - File (PNG): image.png
saskia (21:48:32) (in thread): > That is really good!
2020-02-28
Kevin Rue-Albrecht (03:00:40) (in thread): > Thanks:blush:I’m not pushing anything as my code is still a horrible copy paste patchwork of bits and pieces of the iSEE reduced dimension plot panel. But this proof of concept also shows the many benefits of inheriting functionality from parent panels. Overall it wasn’t that much work
Kevin Rue-Albrecht (03:01:56) (in thread): > The real work will be to continue updating iSEE in parallel to reflect the needs of downstream panel development in iSEEu
Kevin Rue-Albrecht (17:45:06): > .build_labs()seems like a good candidate for@export
2020-02-29
Aaron Lun (04:46:37): > MA plot and volcano plots are done.
Aaron Lun (05:36:18) (in thread): > Unofrtunately not, because I was building on top of an existing plotting framework. But you’re more than welcome to make a PR to create anEnhancedVolcanoPlotpanel class.
Kevin Rue-Albrecht (12:57:47): > Playing a bit more. - File (PNG): image.png
Aaron Lun (12:58:57): > How do you compute the color? Averaging?
Kevin Rue-Albrecht (12:59:10): > at the moment, yes
Aaron Lun (12:59:30): > hm.
Aaron Lun (12:59:33): > well, okay.
Kevin Rue-Albrecht (13:02:43): > I’m still figuring out the actualggplot2syntax to compute/draw data in hexbins. I already need to switch betweengeom_hexandstat_summary_hexdepending on whether or not coloring by column data…
Kevin Rue-Albrecht (13:03:34): > Similar to@Kevin Blighe’s comment aboutEnhancedVolcano, integrating that panel into the iSEE framework means that I can’t useschex
Aaron Lun (13:04:51): > Don’t understand what that means. Panels should be independent.
Kevin Rue-Albrecht (13:06:17): > now I don’t understand what you just wrote
Aaron Lun (13:07:21): > Why would one panel that usesschexbe affected by another panel that doesn’t?
Kevin Rue-Albrecht (13:10:07): > ah no, that’s not what I meant > I meant thatthis“hex” panel can’t useschexbecause everything is done at theggplot2level. > There can well be another “hex” panel that usesschex(even in the same iSEE instance), but the wayschexis setup means that such a panel would not benefit of certain iSEE features (e.g. they could not receive or transmit selections)
Aaron Lun (13:11:00): > I thought that’s what you actually intended to say.
Kevin Rue-Albrecht (13:12:08): > > Why would one panel that usesschexbe affected by another panel that doesn’t? > Ok, well I never said panels would affect each other. Anyway, we’re probably going in circles here
Kevin Rue-Albrecht (13:21:46): > Alright, I’ve pushed the prototype onpanel-hex. Feel free to check the variousiSEE:::around
Kevin Rue-Albrecht (14:08:14): > Once the dust settles, it would be nice to have somewhere (either the book of perhaps simply the wiki), a few examples and recommendations of how to deploy/host iSEE instances. That seems to be the first barrier of entry.
Kevin Rue-Albrecht (14:09:10): > Recently in my institute, someone asked me about setting up a shiny server on a virtual machine running a webserver, that kind of thing
Aaron Lun (14:09:57): > I don’t see how that’s reallyiSEE’s problem.
Kevin Rue-Albrecht (14:11:16): > Well it’s a barrier to adoption, if it’s not immediately clear to either prospective users or IT-competent people what the ‘best practices’ are to set up instances.
Kevin Rue-Albrecht (14:15:12): > Something like that:https://chanzuckerberg.github.io/cellxgene/posts/hosted - Attachment (cellxgene): Hosting cellxgene on the web > An interactive explorer for single-cell transcriptomics data
Kevin Rue-Albrecht (14:16:39): > A gallery might be nice, but I’m more interested into “recipes” or “stories” documenting how individual people/groups set up instances of iSEE
Kevin Rue-Albrecht (14:17:55): > So that other benefit from the experience, and get a feel of what the steps are, and how easy/difficult it based on their own environment/institute
Kevin Rue-Albrecht (14:18:40): > e.g. we could reach out to Vladimir and see if he’s keen to share his own experience
Kevin Rue-Albrecht (14:21:26): > you could throw in what you remember of the setup forhttps://marionilab.cruk.cam.ac.uk/and perhaps setups more targeted for “enterprise deployments” based on your recent updates - Attachment (marionilab.cruk.cam.ac.uk): Marioni Shiny apps > Shiny applications from the Marioni group
Kevin Rue-Albrecht (14:23:10): > Personally, I’m essentially limited to using apps in my own R sessions, because the only shiny server I have access to does not have the appropriate dependencies installed, is not maintained, and I don’t have sufficient access rights to do anything about it ..
Aaron Lun (22:10:48): > Rstudio connect is JUST SO EASY.
Aaron Lun (22:11:01): > I can basically spin up instances in my sleep.
Aaron Lun (22:14:09): > Literally just anapp.Rwith an iSEE call at the bottom and you’re done.
2020-03-01
Kevin Rue-Albrecht (05:18:56): > Point is that’s knowledge worth sharing. I still have a limited sample size, but from what I’ve seen around me, people’s first thought seems to be shiny server
Aaron Lun (14:36:01): > n00bs
Kevin Rue-Albrecht (14:37:09): > if I may, who pays for your RSconnect ?
Kevin Rue-Albrecht (14:39:40): > because ouch: $14,995 /year (50% off for academic research)
Aaron Lun (14:42:13): > some part of the company, I would assume.
Kevin Rue-Albrecht (14:42:49): > on a separate note: I keep running into this build error on Travis, even after clearing the package cache:https://travis-ci.com/iSEE/iSEEu/builds/151260358 - Attachment (travis-ci.com): Travis CI - Test and Deploy with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (14:43:35): > more or less > > annoy.o:/home/travis/R/Library/RcppAnnoy/include/annoylib.h:92: first defined here > collect2: error: ld returned 1 exit status > /home/travis/R-bin/lib/R/share/make/shlib.mk:6: recipe for target 'BiocNeighbors.so' failed > make[1]: ***** [BiocNeighbors.so] Error 1 > make[1]: Leaving directory '/tmp/Rtmpm10sR8/R.INSTALLe32907aea1/BiocNeighbors/src' > ERROR: compilation failed for package 'BiocNeighbors' > * removing '/home/travis/R/Library/BiocNeighbors' > cat: scater.out: No such file or directory > Error in i.p(...) : > (converted from warning) installation of one or more packages failed, > probably 'BiocNeighbors', 'scater' > Calls: <Anonymous> ... with_rprofile_user -> with_envvar -> force -> force -> i.p > Execution halted > The command "Rscript -e 'devtools::install(dependencies=TRUE)'" failed and exited with 1 during . > Your build has been stopped. >
Aaron Lun (14:43:45): > This si known.
Aaron Lun (14:44:01): > https://stat.ethz.ch/pipermail/bioc-devel/2020-February/016264.html
Aaron Lun (14:44:20): > http://bioconductor.org/checkResults/devel/bioc-LATEST/BiocNeighbors/
Kevin Rue-Albrecht (14:45:59): > Haven’t checked my bioc-devel mail folder in a bit
Kevin Rue-Albrecht (14:50:13): > While you’re here, do you have any more plans before bumping to 1.99.2 and pushing it upstream?
Aaron Lun (14:58:56): > Nope.
Kevin Rue-Albrecht (17:10:00): > @Aaron LunIncoming PR, if you wanna check the NEWS. If you merge, I’ll push upstream tomorrow morning
Kevin Rue-Albrecht (17:24:40): > Alright , PR is ready for merge. I’ll go ahead and do it. We can always fix stuff as it pops up
Kevin Rue-Albrecht (17:25:16): > Done.
Kevin Rue-Albrecht (17:25:28): > as in pushed upstream
2020-03-02
Federico Marini (05:16:06): > I just noticed we do not have any ## Installation in the README on github
Federico Marini (05:16:18): > Did we remove it, or do we just not want that?
Federico Marini (05:16:49): > (of course one should know how to install from Bioc packages) - just wondering if we do not miss out on some users so?
Kevin Rue-Albrecht (07:57:14): > Wouldn’t hurt to have a minimal section in the README, even if only to point to the instructions on the Bioc package landing page
Kevin Rue-Albrecht (07:59:03): > Do I see a volunteer?
Federico Marini (08:52:31): > happy to take over, on it later today
Kevin Rue-Albrecht (16:55:22): > [Dark humour] we’re too late…https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(20)30120-1/fulltext
Kevin Rue-Albrecht (16:55:41): > > An interactive web-based dashboard to track COVID-19 in real time
Kevin Rue-Albrecht (16:56:16): > iSEEu::modeCOVID19()
2020-03-03
Federico Marini (01:33:27): > more than 2 cents there’s gonna be some much better examples of such for the shiny contest
Kevin Rue-Albrecht (02:53:41): > Good point. Jokes aside, it’s quite a good use case and exercise to set up an interactive dashboard
2020-03-04
Kevin Rue-Albrecht (06:52:03): > a few hickups along the way due to outdated deploy keys, butiSEEupasses Travis and deploys again:tada:
Kevin Rue-Albrecht (15:27:10): > Any suggestion where to add a (visual) parameter that controls the number of bins for the hexbin reddim plot? > Currently the parameter interface forColumnDotPlotand childs is defined as > > #' @export > setMethod(".defineInterface", "ColumnDotPlot", function(x, se, select_info) { > list( > .create_data_param_box(x, se, select_info), > .create_visual_box_for_column_plots(x, select_info$single$row, select_info$single$column, se), > .create_dotplot_selection_param_box(x, select_info$multi$row, select_info$multi$column) > ) > }) > > However.defineDataInterfaceis the only API point that can be overriden, which means that if I went down that route I’d have to redefine the RedDimPlot data UI + my new parameter UI. > But again, my new parameter is more “visual” than “data”.
Kevin Rue-Albrecht (15:28:41): > I’m wondering whether it’d be possible to nicely pass extra UI widgets to.create_visual_box_for_column_plots(), say as an ellipsis
Aaron Lun (15:28:57): > One could imagine doing so.
Kevin Rue-Albrecht (15:29:59): > and then possibly also extra (or a choice of)checkboxGroupInputto control which sections should be shown
Kevin Rue-Albrecht (15:40:44): > the other alternative is to allow additional collapsible boxes in.defineInterface()which might be simpler and clearer
Kevin Rue-Albrecht (15:42:44): > * simpler: adding new arguments in the existing visual box raises the questions: where exactly? do they get their own checkbox at the top? > * clearer: new arguments would not be mixes up with inherited arguments > that said, if it’s a visual param, it should probably still go in the visual box …
Aaron Lun (15:46:00): > I think we could just afford to have generics that allow users to specify more UI elements within the set of elements defined by each checkbox.
Aaron Lun (15:46:42): > So something likedefineColorInterface,definePointInterface,defineOtherInterface.
Aaron Lun (16:13:37): > ComplexHeatmap tests are breaking for some bizarre reason.
Kevin Rue-Albrecht (16:17:44): > > So something likedefineColorInterface,definePointInterface,defineOtherInterface. > Ohh I like that a lot
Kevin Rue-Albrecht (16:27:17): > I can seeold version of unit class is no longer allowedin the travis log. > I can also see this issue still open:https://github.com/thomasp85/grid/issues/14
Kevin Rue-Albrecht (16:29:39): > Even though ComplexHeatmap doesn’t look broken, so I don’t know where the issue is coming from yet.
Kevin Rue-Albrecht (16:34:33): > Anyway, just keep doing whatever you’re doing onstuffand let me know when you’re done. There’s one travis error that’s not heatmap-related. If you leave me the heatmap stuff, I’ll have a look this weekend
Aaron Lun (16:34:44): > k
2020-03-05
Kevin Rue-Albrecht (16:17:10): > Man the new visual parameters system is soooo groovy
Kevin Rue-Albrecht (16:17:50): > By the way, I’m going to renove the trailing “s” from “Points” and “Facets” . The other choices are singular
Kevin Rue-Albrecht (16:19:30): > Also, “Size by” should logically be taken out of the “Points” section and have its own checkbox, like “Color” and “Shape”
Aaron Lun (16:19:59): > knock yourself out. It would be good to think of a better system than a checkbox if we coudl.
Kevin Rue-Albrecht (16:20:07): > “Point” would then be left with downsampling and default point size
Kevin Rue-Albrecht (16:20:30): > > knock yourself out. It would be good to think of a better system than a checkbox if we coudl. > a selectize crossed my mind, to show only one section at a time
Kevin Rue-Albrecht (16:20:47): > but there are some combinations of sections that I do like having
Aaron Lun (16:21:20): > Hm. Selectizes involve at least two clicks (one to open, another to pick). Would like to just have one click.
Kevin Rue-Albrecht (16:21:51): > I’ve got two words for you: voice regognition (0 click)
Kevin Rue-Albrecht (17:22:22): - File (PNG): image.png
Kevin Rue-Albrecht (17:22:55): > (yes I know the ExperimentColorMap color scale isn’t applied yet )
Kevin Rue-Albrecht (17:24:32): > e.g., bin resolution = 50 - File (PNG): image.png
Aaron Lun (17:24:43): > most excellent
Kevin Rue-Albrecht (17:26:51): > and if anyone fancies a game of HoMM - File (PNG): image.png
Aaron Lun (17:37:20): > I was thinking civ 6
Federico Marini (17:38:43): > well that can be only one thing
Federico Marini (17:38:52): > settlers of catan
2020-03-06
Kevin Rue-Albrecht (05:23:46): > > ComplexHeatmap tests are breaking for some bizarre reason. > Weirdly enough the unit tests pass on my laptop and the Bioc-devel builds is not affected. Looks like a travis-specific issue
Kevin Rue-Albrecht (18:03:08): > @Aaron LunSomeone’s gotta try the combo of branchesiSEE:visual-boxandiSEEu:panel-reddimhexwith: > > <Toy sce dataset> > > iSEEOptions$set( > downsample=TRUE, > downsample.resolution=50, > panel.width = 6L, > panel.height = 500L, > assay = "logcounts" > ) > > iSEE(sce, list(ReducedDimensionHexPlot(), ReducedDimensionPlot())) >
Kevin Rue-Albrecht (18:05:53): > As a bonus point, even better than the old “default to ‘logcounts’” behaviour that we had lost in 2.0 (we currently default to the first assay), the global options can define a character vector of “preferred” assay names in order of preference
Kevin Rue-Albrecht (18:15:19): > if anyone suggests a good global option name for the “default reduced dimension name” (but shorter than that), I’ll add it tomorrow
Kevin Rue-Albrecht (18:41:03): > While I’m at it: it’s not just a fancy feature branch. There are a few fixes too
Kevin Rue-Albrecht (18:41:26): > Some typos in cached information: colData/rowData
Aaron Lun (19:09:15): > Haven’t looked at it, but those should be colData and not columnData, as it’s referring to the SE slot.
2020-03-07
Kevin Rue-Albrecht (05:25:44): > I don’t know what you’re talking about here. > I don’t remember touching any “columnData”. I just wrote “colData” above.
Kevin Rue-Albrecht (12:16:57): > I’ve force-merged and pushed upstreamhttps://github.com/iSEE/iSEE/pull/381as 1.99.3. Charlotte confirmed thatR CMD checkpasses for her as well, so it’s a glitch on Travis.
Kevin Rue-Albrecht (12:18:47): > the fact that iSEE passes Bioc builds also confirms that it’s a Travis thing
Kevin Rue-Albrecht (12:19:02): > Aaron, you’ll have to update your branches and runchecklocally until Travis clears up
Kevin Rue-Albrecht (12:20:46): > also don’t forgetlibrary(covr); report(package_coverage)
Aaron Lun (21:24:40): > Does the travis maintainer know about it?
Aaron Lun (23:36:41): > Also, OMG.globalselect2will blow your mind.
Aaron Lun (23:37:30): > Most demonstrative experience is achieved by opening up the standard app, then clicking on feature assay’s data parameter “use dynamic selection for y-axis”, then clicking on any table row or selecting any point in the row data plots.
Aaron Lun (23:37:59): > The selection now just follows the user without them having to manually set up the links.
Aaron Lun (23:38:50): > Perfect for people who… well, don’t know how to set up links.
2020-03-08
Aaron Lun (03:59:30): > :alisa:
Kevin Rue-Albrecht (06:26:17) (in thread): > Not yet. At lest not from me.
Kevin Rue-Albrecht (06:27:30) (in thread): > Thing is he would probably appreciate a MWE which would amount to making a small package using heatmap I imagine
Aaron Lun (20:32:13): > IT IS DONE.@Kevin Rue-Albrechtget some push action going on.
Aaron Lun (20:33:27): > Oh, but before you do that. See if you can add a global option to turn all dynamic selections on/off. It’s a good opportunity for you to review that system at the same time.
2020-03-09
Kevin Rue-Albrecht (15:35:44): > good forupstream? Can’t check/review right now
Aaron Lun (15:37:13): > it would be nice to see that global option.
Aaron Lun (15:37:21): > I think you would quite enjoy it.
Kevin Rue-Albrecht (15:38:15): > OK. I’m already multitasking but I’ll chime in a couple of hours
Aaron Lun (17:00:03): > oh yeah getting some sweet, sweet business class upgrade offers
Kevin Rue-Albrecht (17:19:21): > so when do you set up that iSEE charity for underprivileged postdocs?
Aaron Lun (17:25:25): > Charity is not a word in my vocabulary
Kevin Rue-Albrecht (17:25:34): > Anyway, I played with the app and it’s impressive. I just need to figure out when exactly during the initialization to apply the global option
Aaron Lun (17:26:01): > Should be pretty clear; anywhere there’s anempty_default()with a *Dynamic option
Kevin Rue-Albrecht (17:26:03): > don’t spoil the fun though if you don’t wanna do it yourself though
Kevin Rue-Albrecht (17:26:09): > :stuck_out_tongue:
Kevin Rue-Albrecht (17:27:22): > so, if I get that right,.add_panel_to_dynamic_sourcesis the counterpart of that, to add a source panel, while the global option would simply set all panels as receivers
Aaron Lun (17:27:35): - File (PNG): image.png
Kevin Rue-Albrecht (17:27:43): > (now that you gave half the answer… might as well)
Aaron Lun (17:27:45): > damn, that was meant to be a gif
Kevin Rue-Albrecht (17:28:32): > Btw, i just noticed, looks like DotPlot also downsamples incoming selections.
Kevin Rue-Albrecht (17:28:58): > It’s probably safe to say thatallselected points should be shown, right?
Aaron Lun (17:29:34): > Well, I thought about that. But in the end I didn’t because you could select quite a few points (or everything) and you would want to downsample them.
Aaron Lun (17:29:48): > Also, it’s just easier to not special case it.
Kevin Rue-Albrecht (17:30:55): > I also thought that in the case of everything, it would help (for large datasets), but with our toy dataset, I was a bit surprised that i selected 6 points and only 4 were shown
Aaron Lun (17:31:23): > How aggressive are you downsampling? That seems a bit too intense.
Kevin Rue-Albrecht (17:31:28): > I don’t have strong feeling either way, I just thought I’d mention it to check if it was a bug
Kevin Rue-Albrecht (17:32:38): > oh. good point. I haddownsample.resolution=50for launching the app extra quick
Kevin Rue-Albrecht (17:33:03): > my bad
Aaron Lun (17:34:20): - File (GIF): railgun.gif
Aaron Lun (17:38:02): > yes finally got it
Kevin Rue-Albrecht (17:40:11): > ohh allow me borrow that one:joy:
Kevin Rue-Albrecht (17:51:42): > incoming
Kevin Rue-Albrecht (17:53:55): > with that, I must say that the heat map panel finally feels worthwhile having around
Kevin Rue-Albrecht (17:54:38): > setting up the source panels for each dimension was literally double the pain
Aaron Lun (18:14:26): > I wonder whether we should have separate global toggles for the multiple selections and the single selections.
Aaron Lun (18:14:36): > Otherwise the colours of all the plots would jump around as well.
Kevin Rue-Albrecht (18:17:13): > Didn’t think of that. Makes sense. Honestly I’m not focused right now and you’ll probably get it faster where you want without going through me. On the plus side, my PR basically marks up all the occurences that you can toy with
Aaron Lun (23:33:43): > rewatching the boromir death scene always makes me tear up
2020-03-10
Kevin Rue-Albrecht (07:12:49): > In relation to the recent Travis fails related to heatmaps and units: > * there are a number of unit-related commits in ComplexHeatmap (https://github.com/jokergoo/ComplexHeatmap/commits/master) > * Travis installs 2.3.0 (https://travis-ci.com/iSEE/iSEE/builds/152352449#L6910) > * bioc builds fails to build 2.3.2 (http://bioconductor.org/checkResults/devel/bioc-LATEST/ComplexHeatmap/) - Attachment (travis-ci.com): Travis CI - Test and Deploy with Confidence > Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you’ll be testing your code in minutes.
Kevin Rue-Albrecht (07:14:03): > So my take is that we just need to give it time and wait for the next ‘working’ version of CH. Making a dummy MWE package to reproduce the Travis fail really isn’t at the top of my TODO list right now
Kevin Rue-Albrecht (18:10:10): > Update ondeploy_site_github() is not finding the ssh key:https://github.com/r-lib/pkgdown/issues/1206 > > We now recommend using the github actions workflow instead; which avoids all this configuration pain.
Kevin Rue-Albrecht (18:11:34): > Anyway, for now it works as it is, and it’s just a gh-pages bonus, not an essential feature, so moving to GH actions is definitely on my low priority list right now
Kevin Rue-Albrecht (18:40:55): > Off to sleep. I’ve pushed a commit splitting into multiple and single selections, probably best enjoyed with > > iSEEOptions$set( > downsample=TRUE, > selection.dynamic.multiple = TRUE, > panel.width = 3L, > panel.height = 400L > ) > iSEE(sce) >
Kevin Rue-Albrecht (18:42:33): > I’m not sure that’s exactly what you’re thinking@Aaron Lun, so if you’re happy merge, if you’re not happy you might as well just implement what you have in mind to save us both time,
Kevin Rue-Albrecht (18:42:49): > then give me the sign and I’ll push upstream
Aaron Lun (18:54:48): > DO IT
Aaron Lun (19:03:13): > Though I also wonder whether we should use camel case like the slot names… might make it a bit easier to match up.
Aaron Lun (19:03:23): > Food for thought.
Aaron Lun (20:15:23): > Also, I think if we set everyobserveEventtoignoreInit=TRUE, we might get a massive initialization speed-up.
2020-03-11
Aaron Lun (01:00:21): > Okay, well, not quite massive. Shaves off 1 second in the typical example.
Aaron Lun (01:00:28): > well. better than nothing.
Aaron Lun (01:29:04): > GitHub changed their web appearance if you look at the commit history
Aaron Lun (02:51:09): > Just read our iSEEu code of conduct. I think I would like to see it say, “ad hominem attacks are okay as long as you can take what you dish out”.
Kevin Rue-Albrecht (15:07:47): > I’ve got a pretty good idea what’s wrong with Travis now… that newdata.frame(stringAsFactors=FALSE)seems to have kicked in
Kevin Rue-Albrecht (15:08:16): > that’s explains why CMD check still passes fine on my computers
Kevin Rue-Albrecht (16:23:59): > oh goodie goodie - File (PNG): image.png
Aaron Lun (16:26:44): > excellent, excellent.
Aaron Lun (16:28:19): - File (GIF): thumbs-up-anime-gif-24.gif
Kevin Rue-Albrecht (16:28:29): > can you remember: when I compile a new R from source, and make install, will it preserve the previous library of installed packages ?
Kevin Rue-Albrecht (16:28:39): > i mean same R version and all
Aaron Lun (16:28:44): > Yes
Aaron Lun (16:28:52): > if you don’t make clean you should be fine.
Kevin Rue-Albrecht (16:28:59): > cool, so I don’t need to backup the library
Aaron Lun (16:29:01): > Probably not safe to do that for different R versions though.
Aaron Lun (16:29:17): > Because of ABI changes at the C/C++ level.
Kevin Rue-Albrecht (16:29:23): > exactly, for different version I wouldn’t try
Kevin Rue-Albrecht (16:56:38): > upstream, let’s see how the bioc build machine deal with it - File (PNG): image.png
Kevin Rue-Albrecht (16:59:34): - File (GIF): source.gif
Kevin Rue-Albrecht (16:59:45): > didn’t upload as a GIF the first time
Federico Marini (17:02:32): > is it kallisto vs salmon again?
Kevin Rue-Albrecht (17:03:04): > nah - just about sending master upstream
Kevin Rue-Albrecht (17:03:21): > i’ll get better at GIF talk, haven’t practice in a while
Federico Marini (17:03:31): > :slightly_smiling_face:
Kevin Rue-Albrecht (17:32:41): > challenge: 38 commits from 3,000
Federico Marini (17:54:17): > We are not seeking for spotlights, I guess
Federico Marini (17:54:39): > but an instance on lung scRNA-seq could bejust useful for many researchers
Aaron Lun (17:55:00): > Well, that’s your job.
Federico Marini (18:02:35): > :slightly_smiling_face:true - wanted to check in with the rest of the A-team
Federico Marini (18:02:48): > should I reach out to him directly maybe?
Aaron Lun (18:04:32): > who is “him”
Federico Marini (18:07:09): > the guy that would keep on adding a couple of features on their app
Federico Marini (18:07:19): > ups
Federico Marini (18:07:20): > sorry
Federico Marini (18:07:29): > thought I also posted the link
Federico Marini (18:07:40): > https://twitter.com/KaminskiMed/status/1237853406921773057?s=20 - Attachment (twitter): Attachment > @NeBanovich @lpachter You don’t need to be a computational expert to query your gene or cell of interest in http://www.IPFCellAtlas.com & we are working to add more functionality: batch searches, more data, other tools https://twitter.com/kaminskimed/status/1236991246356414466?s=21 https://twitter.com/kaminskimed/status/1236991246356414466 - Attachment (twitter): Attachment > Since November #IPFCellAtlas has had 2383 unique users registering 6517 sessions, doing ~9000 searches. ACE2, the #COVID19 receptor, went from 0 searches in January, to ~400 today - evidence of usefulness of our lung #scRNAseq access resource beyond IPF http://www.ipfcellatlas.com/ https://pbs.twimg.com/media/ESqs-hzX0AMJppS.jpg
Federico Marini (18:07:45): > him him
Aaron Lun (18:08:40): > oh god
Aaron Lun (18:08:50): > no, let’s not do that.
Kevin Rue-Albrecht (18:10:49): > There’s probably plenty enough to clean in front of our own door already.
Federico Marini (18:12:36): > well we can just point them towards our door
Kevin Rue-Albrecht (18:13:32): > if you can find a politically correct way to do so, be my guest, i don’t have the words yet
Federico Marini (18:15:48): > k:wink:
Aaron Lun (19:42:43): > Ugh. Updated brew and now my R needs recompilation.
Aaron Lun (23:57:33): > Let’s finish the fight.
2020-03-12
Kevin Rue-Albrecht (03:12:04) (in thread): > That’s cold… brew.:grimacing:
2020-03-13
Aaron Lun (03:13:45): > Looking at COVID symptoms, and I’m like, wait, that’s how I feel all the time.
Aaron Lun (03:13:59): > muscle aches, fatigue, headache
Aaron Lun (03:14:08): > sounds like me after a day of meetings.
Aaron Lun (12:09:18): > @Charlotte Soneson@Kevin Rue-Albrechtnote the new package submission deadline for iSEEu.
Kevin Rue-Albrecht (15:42:53): > I’ve started the unit tests forhttps://github.com/iSEE/iSEEu/pull/14However, you can have a look at a few occurences of:::in that PR, namely it shows that iSEE will need to consider exporting: > * .empty_default() > * .find_atomic_fields() > * .which_groupable() > * .which_numeric() > * .add_facets() > * .text_eval() > * .build_labs()
Kevin Rue-Albrecht (15:44:15): > Let me know if I missed a memo about any of those
Aaron Lun (16:24:18): > Sounds good, but we wil have to camelCase all of these.
Federico Marini (16:44:16): > guess it makes sense to require that iSEEu depends on iSEE (>= 1.99.0), right?
Federico Marini (17:20:18): > @Aaron Lun: want to setup the functions ofscRNAseqto use …https://community-bioc.slack.com/archives/C35G93GJH/p1584132006106400 - Attachment: Attachment > See if you can use localHub=TRUE because you may already have copies of what you need. I am finding that AnnotationHub is not responding either.
Aaron Lun (17:20:40): > You shoudl be able to do that globally.
Federico Marini (17:20:40): > (Ehub is not responding to the requests now)
Aaron Lun (17:20:45): > In the EHub options.
Aaron Lun (18:03:55): > @Federico Mariniwhy add version numbers? iSEEu should be installed by BioC, let BiocManager take care of that.
Federico Marini (18:04:42): > I’m thinking of the wild github-only users. Some of them still exist
Federico Marini (18:05:08): > (among which, the editor handling one of my manuscripts and reporting some error in the installation:face_palm_star_trek:)
Aaron Lun (18:05:53): > I can’t say I feel much sympathy for them.
Aaron Lun (18:06:03): > But if you must put in version numbers, it should be version numbers for iSEE.
Federico Marini (18:07:11): > ¯*(ツ)*/¯
Aaron Lun (18:07:34): > Because then people will complain along the lines of “can you install this on R 3.6.*”
Federico Marini (18:07:35): > Anyway: i fleshed out a first version for a full-info README for iSEEu
Federico Marini (18:08:08): > > Because then people will complain along the lines of “can you install this on R 3.6.*” > Totally see them fork and edit that one line:smile:
2020-03-14
Aaron Lun (02:23:37): > I WILL be on vacation, starting Sunday. So if you guys want to get iSEEu into BioC for the next release, it’s on you.
Aaron Lun (02:23:53): > @Charlotte SonesonWhat happened to those commuunity panels?
Aaron Lun (02:34:40) (in thread): > setExperimentHubOption("LOCAL", TRUE)
Federico Marini (04:17:14): > more than panels, it was themodes
Federico Marini (04:19:32) (in thread): > thanks!
Aaron Lun (14:26:59): > I distinctly recall seeing some custom panels.
Kevin Rue-Albrecht (14:29:02): > you’re just in time to see the an incoming commit for 100% coverage of the hex panel
Kevin Rue-Albrecht (14:29:29): > though that in its current state, the:::are still there
Charlotte Soneson (14:41:57): > Yes, there were some steps towards custom panels in Brussels (at least one for trees), although these were not included iniSEEuat the time (since they were not finalized).
2020-03-15
Aaron Lun (00:57:46): > damn. Auz govt just announced 14 day self-isolation policy for all incoming travelers. Which kind of defeats the purpose of my vacation.
Aaron Lun (00:58:07): > I don’t want to stare at my bedroom wall for 14 days. I wonder how they’re going to enforce it, though.
Aaron Lun (01:02:26): > Damn. Should have booked my vacation a week earlier.
Kevin Rue-Albrecht (11:52:20): > I’ve just started going through the first of those. - Attachment: Attachment > I’ve started the unit tests for https://github.com/iSEE/iSEEu/pull/14 > However, you can have a look at a few occurences of ::: in that PR, namely it shows that iSEE will need to consider exporting: > • .empty_default() > • .find_atomic_fields() > • .which_groupable() > • .which_numeric() > • .add_facets() > • .text_eval() > • .build_labs()
Kevin Rue-Albrecht (11:53:18): > Exporting a function is really the easiest part. The pain is mostly about deciding how to document them.
Kevin Rue-Albrecht (11:54:14): > That said, I don’t know exactly whatourminimum requirements are to submit iSEEu to bioc before April 3rd
Kevin Rue-Albrecht (13:19:07): > by-product:https://github.com/kevinrue/Bioinformatics/blob/master/R/bin/replace.shReplacing a word throughout a package R/ vignettes/ and tests/testhat/
Aaron Lun (14:03:17): > Just throw the docs with the rest of the “internal functions exported for developer use” manpage.
Kevin Rue-Albrecht (14:07:19): > .build_aes->.buildAes? > or maybe.buildAesthetics?
Aaron Lun (14:08:56): > yep
Aaron Lun (14:09:02): > Just follow your heart
Kevin Rue-Albrecht (14:09:46): > I’ll stick with aes then, not sure if the full word should take a final “s”
Kevin Rue-Albrecht (14:45:58): > Incoming branchmore-genericsfor iSEE
Kevin Rue-Albrecht (14:47:09): > accompanied by matching updates on branchpanel-redimhex-2(iSEEu)
Kevin Rue-Albrecht (15:48:24): > PR looks fine, I’ve merged it and pushed upstream to get the version on bioc ASAP. Worst case we revert or throw another PR if there’s a problem (e.g. cleaner man pages)
Kevin Rue-Albrecht (15:49:33): > this one’s 1.99.5
Kevin Rue-Albrecht (15:52:31): > https://github.com/iSEE/iSEE/pull/387
Aaron Lun (20:28:59): > Gene set table almost online.
2020-03-16
Aaron Lun (00:41:42): > Readinghttps://royalsocietypublishing.org/doi/full/10.1098/rsos.160384
Aaron Lun (00:41:51): > > Ethics > > Our meta-analysis used only previously published data. All simulated scientists were humanely euthanized.
Federico Marini (07:30:26): > I noticed the CoC from code covenant is now in its newer version
Federico Marini (07:30:39): > should we update it foriSEEuas well?
Kevin Rue-Albrecht (07:31:06): > sure
2020-03-17
Aaron Lun (23:16:49): > Crushed another element of my anime backlog. How’s it going over here?
2020-03-18
Kevin Rue-Albrecht (03:31:38): > the hex panel is technically (iSEE version, code, tests) ready for merge, but I haven’t yet added a vignette or screenshot as per the contribution code
Aaron Lun (03:31:58): > Wait, a screenshot was part of the contribution requirements?
Aaron Lun (03:32:06): > Well, someone else is going to have to do my part.
Kevin Rue-Albrecht (03:33:15): > maybe it’s just guidelines though (from the README @https://github.com/iSEE/iSEEu) > > once your contribution (function, panel, mode) is done, consider adding some information in the package. Some examples might be a screenshot of the mode in action (to be placed in the folderinst/modes_img), or a well-documented example use case (maybe an entry in thevignettesfolder). Also add yourself as a contributor (ctb) to the DESCRIPTION file.
Aaron Lun (03:33:53): > We should consider an automated way of creating these screenshots.
Kevin Rue-Albrecht (03:38:42): > I’m suggesting this one purely because I think you’ll like the name:https://dev.to/benjaminmock/how-to-take-a-screenshot-of-a-page-with-javascript-1e7c - Attachment (The DEV Community): :camera: How to take a screenshot of a webpage with JavaScript in Node.js (using puppeteer) > . Tagged with js, javascript, webdev, puppeteer.
Kevin Rue-Albrecht (03:40:15): > In the meantime, shall i just my own PR, and perhaps we can submit to bioc already today and deal with their reviews in parallel to our own final cleanup?
Kevin Rue-Albrecht (03:41:00): > I haven’t taken a global vision of iSEEu as a Bioc package recently, but perhaps an external review could be more effective in the end
Aaron Lun (03:41:11): > DO IT
Kevin Rue-Albrecht (03:42:38): > Done (PR).
Charlotte Soneson (03:43:29): > I cleaned up the reducedDim mode, I’ll PR that shortly
Aaron Lun (03:53:33): > getting puppeteer to work in a truly automated way would be a bit of effort. Would have to start iSEE in a separate session with a pre-specified host, poll for completed loading in the current host, and then fire up puppeteer.
Kevin Rue-Albrecht (03:55:31): > Every tool that I remember looking up every time we talked about this seems to take the same approach. To be honest, I’m not sure there’s any other way really.
Kevin Rue-Albrecht (04:01:29): > Our wish list the way I see it: > * automaticallycreate screenshot -> some kind of script to run locally -> generate PNG files -> commit and push > * for each Panel -> we’d probably need a list of launch scripts each for one panel to screenshot > * for each mode -> (same as for Panel) > Anyway - maybe not top priority for submission to bioc
Charlotte Soneson (05:56:20): > modeReducedDimmerged. One question that came up (https://github.com/iSEE/iSEEu/pull/3) was whether it was possible to select a colData column to simultaneously color all the reduced dimension plots. I can’t really see a way to set this up, but I may be missing something among all the new features(?)
Kevin Rue-Albrecht (06:04:32): > Mmmh. Not that I’m aware of. Aaron implemented “global selections” which is making all panels update to receive the latest selection made in any other panel, but that’s different
Kevin Rue-Albrecht (06:06:52): > It doesn’t sound straightforward but I wonder we could implement a new type of “master UI panel” that doesn’t have a plot but only UI elements (such as a colData columnselectizeInput), and that all panels in the UI would listen to
Charlotte Soneson (06:08:20): > Hm, maybe.
Charlotte Soneson (06:09:01): > But there would need to be separate selections for color, shape, … - so basically a shared Visual Parameters box
Kevin Rue-Albrecht (06:12:17): > Reading the PR again > > This is currently implemented whencolorByis a feature name, but still lacking for the column data. > It’s only possible now for feature (and samples) because we have metadata tables for those two dimensions. > Another possibility would be to have Table panels showing a colData/rowData column name per row, and metadata for that metadata (e,g. long description) in each column. That would allow colData/rowData columns to be selectable (as a row in those tables), and as a result that selection could be listened on by other panels
Kevin Rue-Albrecht (06:14:14): > Less work than my previous suggestion, as it’s just about extending our existing Table panel class for a different type of table, but still a bit much to swing before the bioc release if we want to submit soon
Charlotte Soneson (06:21:34): > Yes, agree. Something to think about going forward
Kevin Rue-Albrecht (06:35:10): > Haven’t thought it fully through, but here goes:https://github.com/iSEE/iSEEu/issues/21
Charlotte Soneson (06:36:20): > :+1:
Kevin Rue-Albrecht (10:37:56): > Any chance someone can act on that one foriSEEu? > > * checking installed package size ... NOTE > installed size is 7.3Mb > sub-directories of 1Mb or more: > contributing101 6.1Mb >
Kevin Rue-Albrecht (10:38:21): > I’ve taken care of the other note (about imports)
Kevin Rue-Albrecht (10:38:42): > there are 2 screenshots of 1 MB each already
Kevin Rue-Albrecht (10:40:31): > Culprits: > > 3.3M contributing101/libs/ > 3.5M contributing101/images/ >
Federico Marini (10:45:57): > Will be later on this. I think this can safely be moved to some other pseudopermanent location - or just to a secondary branch we opt to keep IMHO
Kevin Rue-Albrecht (11:39:50): > https://github.com/iSEE/iSEEu/projects/1
Aaron Lun (14:14:31) (in thread): > This is not straightforward, we don’t have a concept of transmitting selections of metadata columns.
Aaron Lun (15:16:57): > Also, decide whether you want to prefix all panel files withpanel_or not.
Federico Marini (16:43:26): > should de done now with the space issue
Kevin Rue-Albrecht (16:46:03): > Maybe the capitalization is enough to keep them together at the top of the package?
Aaron Lun (18:22:30): > Error in : width of the annotation can only be an absolute unit.ComplexHeatmap problem.
Kevin Rue-Albrecht (18:23:02): > again?
Kevin Rue-Albrecht (18:23:14): > that one showed up and disappeared with their recent updates
Aaron Lun (22:43:10): > Showing up locally for me on the default allen example.
Aaron Lun (23:23:43): > automated screenshots now available.
2020-03-19
Kevin Rue-Albrecht (02:26:55): > I don’t see anything. Where do I look?
Aaron Lun (02:39:04): > iSEE/screenshots
Aaron Lun (02:39:12): > currently working on building a docker image for github actions
Kevin Rue-Albrecht (02:41:11): > ahhhh a new repo - i was looking at branches, makes sense, well done
Aaron Lun (03:34:00): > god, these dockerfiles take ages to build.
Kevin Rue-Albrecht (03:35:31): > yup
Aaron Lun (03:38:04): > I hope it caches it somewhere.
Aaron Lun (03:38:55): > though if it were cahced, my guess is that it’ll be out of date and we have to manually trigger a refresh. I don’t even know how to do that.
Kevin Rue-Albrecht (03:38:56): > [cache]: Not that I know of
Aaron Lun (03:56:19): > Well, I’m going to go to sleep, but see if you can buildscreenshots’ Dockerfile and run it to generate PNGs. (Note that there arealreadyPNGs, so make sure it actually regenerates them. Also note that the entrypoint relies on$1being an access token passed in via GitHub actions, so you’ll just have to replace that bit with your own token for the time being if you want to check if the push works.)
Aaron Lun (03:57:54): > You can keep an eye on it athttps://github.com/iSEE/screenshots/actions.
Kevin Rue-Albrecht (03:58:20): > i just tried, but it generates an emptyiSEEfolder
Kevin Rue-Albrecht (03:58:40): > maybe i missed something, i’ve been all over the place already since i woke up
Aaron Lun (03:59:07): > How did you build it so fast?
Kevin Rue-Albrecht (03:59:51): > that’s my point, i’ve clearly missed something
Kevin Rue-Albrecht (04:00:24): > ah wait. i seedelaynow
Aaron Lun (04:00:49): > Wait, are you using the docker image to do this?
Aaron Lun (04:00:57): > The rest of it works fine.
Kevin Rue-Albrecht (04:01:51): > I didn’t see the Docker instructions, that’s why - i just picked up the repo and started running the scripts ehehe
Aaron Lun (04:02:27): > Thecompile.Rtakes a few minutes to run due to the delay.
Aaron Lun (04:02:32): > Just sit back
Kevin Rue-Albrecht (04:03:09): > i can’t promise when i’ll look at it - i need to use whatever brainpower i can gather for other top priorities first
Kevin Rue-Albrecht (04:05:33): > on a more time-limited note, any last wish before I submit iSEEu to bioc contributions?
Kevin Rue-Albrecht (04:07:15): > Those screenshots are not key for that, right?
Aaron Lun (04:08:07): > nope
Kevin Rue-Albrecht (04:09:34): > I think I wrote that yesterday: Might as well submit it now, and do our own updates alongside anything that the reviewer may throw at us. Deal?
Aaron Lun (04:10:17): > ¯*(ツ)*/¯
Aaron Lun (04:10:20): > Dpm
Aaron Lun (04:10:22): > t care. O
Aaron Lun (04:10:27): > i’ve done my part
Aaron Lun (04:11:34): > looks like there is caching, though. Hmmm.
Aaron Lun (04:11:56): > Might have to forcibly re-install iSEE inentrypoint.sh.
Kevin Rue-Albrecht (04:17:35): > Done:https://github.com/Bioconductor/Contributions/issues/1425
Federico Marini (04:36:43): > Would it make sense to rename the repo toiSEEscreenshots? name’s generic for cloning locally, and my setup is one big folder for allDevelopmentstuff
Kevin Rue-Albrecht (04:39:53): > sure - it seems Aaron designed it to handle any other arbitrary package, but I guess that’s to separate iSEE and iSEEu for now
Kevin Rue-Albrecht (04:41:21): > You’ll see thatapps/has a subfolderiSEEand if you runimages/compile.Rit will generate the same subfolder structure
Aaron Lun (14:28:05): > No point renaming it. Everything in the iSEE org is effectively namespaced by “iSEE” already.
Aaron Lun (14:39:29): > In fact, everything in the iSEE org should iSEE prefixes stripped out, IMHO.
Aaron Lun (14:52:24): > OMG. It really is rebuilding it.
Aaron Lun (14:52:32): > My god. It’s going to take an hour every time for the image to set up.
Aaron Lun (15:03:33): > I think we need to keep our iSEE dockerfile on DockerHub up to date, so the Actions don’t have to spend an hour rebuilding the image every time.
Aaron Lun (17:23:19): > Damn. It takes so long to build the image that the access token’s lifespan is timing out, so I can’t push the changes back!
Aaron Lun (17:32:31): > Whoever is in charge of iseedevelopers’ dockerHub account should add the screenshots image so that I can just pull that instead.
Aaron Lun (17:36:00): > Also, devel_base2 is deprecated.
Aaron Lun (17:42:49): > my god scater has so many deps
Aaron Lun (17:42:51): > so many deps
Aaron Lun (17:43:20): > time for the long knives
Aaron Lun (18:19:16): > got it down to 10 minutes.
Aaron Lun (18:38:12): > oh wait. Forgot some deps. Creeping back up to half an hour again.
2020-03-20
Aaron Lun (00:29:21): > OM GOMG
Aaron Lun (00:29:22): > https://github.com/iSEE/screenshots/actions/runs/59400051
Aaron Lun (01:29:40): > oh yes. oh yes
Aaron Lun (01:54:57): > It is done. Note the non-negligible song and dance that needs to be done due tomasterprotection.
Aaron Lun (01:55:50): > Also, some of the examples in thecustomvignette could do with a better demonstration, but I literally don’t have the will to do that anymore.
Aaron Lun (01:55:53): > I’ll also leave it to someone else to make equivalent screenshots for the iSEEu vignette.
Aaron Lun (03:21:55): > Though don’t do anything yet, I’m going to move a lot of the processing logic into the source repos.
Aaron Lun (03:28:48): > Actually, i don’t think i will do that.
Aaron Lun (03:50:44): > Nope. Changed my mind again. Maybe I will do it.
Kevin Rue-Albrecht (07:53:11): > Success:https://hub.docker.com/repository/registry-1.docker.io/iseedevelopers/screenshots/builds/9a589f77-8cb0-488f-92fe-eec939c127e6
Aaron Lun (13:07:40): > account is ltla
Kevin Rue-Albrecht (13:09:39): > Done
Aaron Lun (13:11:43): > Does this auto-re-build to the latest?
Kevin Rue-Albrecht (13:13:33): > Re: iSEEu submission, I won’t push it onto them. I can’t get it done for this release, but I think the ideal thing would be to lead by example and already submit 4-5 small packages, each with 1-2 panels (e.g. Volcano and MA in 1 package, hex plot in another)
Kevin Rue-Albrecht (13:14:41): > then we could submitiSEEuas a package thatSuggestsall those package making up the iSEEverse
Aaron Lun (13:15:20): > push what onto what?
Kevin Rue-Albrecht (13:15:57): > I mean forcing the iSEEu to be accepted as it is.
Aaron Lun (13:16:35): > Well, we kind of need it out this release.
Kevin Rue-Albrecht (13:18:13): > Michael’s reply got me thinking thatlibrary(iSEEu)could behave likelibrary(tidyverse)https://github.com/tidyverse/tidyverse/blob/master/R/attach.R
Aaron Lun (13:19:14): > Well, regardless of what happens, we’re still going to have an iSEEu package and it’s still going to load all the panels.
Aaron Lun (13:19:23): > So you might as well get it now and we’ll sort it out later.
Kevin Rue-Albrecht (13:20:07): > oh yeah - from a user perspective it’ll be virtually the same
Kevin Rue-Albrecht (13:21:59): > i.e., stepwise that looks like > 1. get iSEEu on bioc > 2. refactor panel code out of iSEEu and into new packages submitted during the next cycle, with iSEEu updated to import panels from those packages instead
Aaron Lun (13:22:17): > Yes.
Aaron Lun (13:22:42): > Pending whether it makes sense to have the overhead. For 1-2 panels per package, it’s just not worth it.
Kevin Rue-Albrecht (13:24:59): > I see it as worth it, tbh. Especially because then each panel can evolve at its own pace in its own package
Aaron Lun (13:26:05): > Well. That’s your problem, then.
Aaron Lun (14:07:04): > oh, it’s beautiful
Aaron Lun (14:07:26): > Also,@Charlotte Sonesonshould probably get rid of her iSEE image on DockerHub, it’s the first one that I landed on when I googled for it.
Kevin Rue-Albrecht (14:19:23): > woops, makes me wonder if i cleared mine
Kevin Rue-Albrecht (14:21:35): > clearly not - plus, my images of iSEE2018 tours are based onFROM csoneson/isee:master
Charlotte Soneson (14:57:36): > Yep. I don’t think we have links to these in the paper or so, so we could remove them
Aaron Lun (15:19:39): > OMG OMG
Aaron Lun (15:19:40): > https://github.com/iSEE/iSEE/pull/390/checks?check_run_id=522787913
Aaron Lun (15:19:44): > I’ve done it
Aaron Lun (15:21:01): > Click on the “artifacts” to get a zip file of the screenshots.
Aaron Lun (15:21:27): > Currently the idea is that the user can just unpack these and commit them tovignettes/screenshots.
Aaron Lun (16:13:29): > Oh and I’ve done it again
Aaron Lun (16:13:29): > https://github.com/iSEE/iSEE/actions/runs/59914410
Aaron Lun (16:13:40): > Automated recompilation and push of new screenshots to a branch.
Aaron Lun (17:11:19): > Hm.
Aaron Lun (17:11:35): > After 1 hour of deep reflection, I have decided that it is best to not automate the committing of the screenshots.
Kevin Rue-Albrecht (17:41:22): > OK.https://isee.github.io/iSEE/articles/configure.htmllooks amazing btw
Aaron Lun (17:43:35): > damn straight
2020-03-21
Federico Marini (05:43:55): > Would it make sense to have the screenshots in github but on a different branch? - if possible
Federico Marini (05:45:03): > (if it is already like this: blame it on the quarantine week:stuck_out_tongue:)
Kevin Rue-Albrecht (06:57:55): > How about on a different repo, master branch? “iSEEgallery” (gallery alone once we strip iSEE from most repos)
Aaron Lun (13:53:29): > See my PR comments.
2020-03-24
Aaron Lun (02:44:02): > Was just reading the CoC.
Aaron Lun (02:44:09): > > Demonstrating empathy and kindness toward other people > WTF?
Aaron Lun (02:44:43): > I don’t remember signing up for that.
Aaron Lun (02:58:43): > Anyway, the iSEE gh-pages doesn’t build properly now that we’ve switched to having screenshots embedded in the repo. Not sure why.
Federico Marini (04:17:05): > > I don’t remember signing up for that. > Too late bro
Kevin Rue-Albrecht (15:16:14): > @Aaron LunNot that I have any time to work on it right now, but the screenshot issue seems to be because the vignettes point to a relative path that does not exist ongh-pagesinstead of embedding the image in the HTML itself. > e.g., “Copy image address” gives mehttps://isee.github.io/iSEE/articles/screenshots/configure-ReDP-basic.png
Kevin Rue-Albrecht (15:17:22): > Ideally, we should look whether it’s possible to tell pkgdown to include the folder of screenshot in the compiled website
Aaron Lun (15:18:48): > The vignette builder during R CMD build embeds the images, I think. Why is pkgdown doing something different?
Kevin Rue-Albrecht (15:19:56): > No idea - I literally just spotted this now when I checked out the vignette to see what you were talking about
Aaron Lun (15:20:32): > can I just leave that with you, then?
Kevin Rue-Albrecht (15:20:50): > don’t expect anything before the weekend then
Aaron Lun (15:20:59): > k
Kevin Rue-Albrecht (15:24:43): > also, I can’t install webshot because of compilation issue -_-
Aaron Lun (15:26:25): > Try webshot2.
Aaron Lun (15:26:42): > Not on cran yet, but I updated the docker image (haven’t tested it though).
Kevin Rue-Albrecht (15:28:26): > sorry that’s what I meant - “webshot2” > > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:729:31: error: unknown type name 'uuid_t'; did you mean 'uid_t'? > int setwgroups_np(int, const uuid_t); > ^ > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_types/_uid_t.h:31:31: note: 'uid_t' declared here > typedef __darwin_uid_t uid_t; > ^ > 5 errors generated. > make: ***** [websocket.o] Error 1 > ERROR: compilation failed for package 'websocket' > * > > Don’t worry about it. I’ll see that later
Federico Marini (18:31:10): > very much a sketch
Federico Marini (18:31:13): - File (PNG): image.png
Federico Marini (18:31:33): > but I couldn’t toy the idea of having a special sticker for iSEEu as well
Kevin Rue-Albrecht (19:10:57): > Neat
Federico Marini (19:12:10): > I think it has (now) the right level of creepy because of the hand shaky movements
Federico Marini (19:12:22): > plus, well, cause powerpoint sucks
2020-03-27
Aaron Lun (14:49:27): > you should be able to build the screenshots off the docker image now.
Kevin Rue-Albrecht (15:07:27): > ok I’ll try the docker~later~tomorrow, I managed to do them from thecompiledbranch so far
Federico Marini (16:31:37): - File (PNG): image.png
Federico Marini (16:32:02): > For the less affine to German: the GCB (Sep 2020) accepted the iSEE workshop I did submit, based on the work for Bioc2020
2020-03-28
Kevin Rue-Albrecht (08:34:09): > @Aaron LunI’ve lost track and I’m confused by theactionsbranch still open. I’ve branched offmasterto fix the screenshot issue in thepkgdownwebsite. Basically, we can’t go aroundfile.path-ing stuff on the fly, becausepkgdownneeds to see the whole path to “external files” that it needs to copy to the website
Kevin Rue-Albrecht (08:48:08): > So if I get it righthttps://github.com/iSEE/iSEE/pull/390#event-3150959684will not be merged?
Aaron Lun (13:57:17): > Yes, but I’m afraid 393 will break the method by which screenshots are taken.
Aaron Lun (13:58:20): > The SCREENSHOTS function is there for a reason. It is overridden byscreenshots/compile.Rand replaced with the relevant webshot(2) commands, allowing us to use the same Rmarkdown to both create the vignette and generate the screenshots.
Kevin Rue-Albrecht (13:59:57): > ok, then can you reintroduce it, but please make sure that your give it the full path to each file as a single string without anyfile.pathmagic?
Kevin Rue-Albrecht (14:00:45): > I’m knee deep into moving from travis to gh-actions, and then I also want to move our Docker image to the new bioc one.
Aaron Lun (14:07:45): > I don’t understand how that would work. If it’s doing static analysis, how would it be able to know thatSCREENSHOTSis an alias forknitr::include_graphics?
Kevin Rue-Albrecht (14:09:34): > well, see it to believe it (or correct me if I’m wrong)
Kevin Rue-Albrecht (14:15:08): > question: anyone minds if I remove links to.scatter_plot,.square_plotand.violin_plotin internal man pages? they’re tripping a warning about missing links in the “R CMD check” GH action
Kevin Rue-Albrecht (14:15:33): > I can leave them as\code{}and just remove the\link{}
Aaron Lun (14:15:39): > Why would that even matter?
Kevin Rue-Albrecht (14:15:44): > exactly
Aaron Lun (14:15:47): > Why is GH trying to rebuild the internal pages?
Kevin Rue-Albrecht (14:16:29): > that’s the point: GH is not rebuilding them, which creates a warning from the newly exportedaes*functions
Aaron Lun (14:16:55): > But whya re those referring to the internal functions?
Kevin Rue-Albrecht (14:17:20): > because before, they were all internal functions referencing each other
Kevin Rue-Albrecht (14:18:11): > those links were just not checked at all
Aaron Lun (14:18:21): > WEll, sure, if it’s an exported function, it shouldn’t have any links to internal functions.
Kevin Rue-Albrecht (14:18:32): > now that some of them are exported, the links that they have to those that remain internal raise the warning
Kevin Rue-Albrecht (14:18:37): > anyway, bottom line: I’ll remove those links
Aaron Lun (15:28:36): > > Trolling, insulting or derogatory comments, and personal or political attacks > but that’s my entire raison d’etre!
Aaron Lun (15:35:26): > %>%
USLACKBOT (15:35:27): > OTL
Aaron Lun (15:35:31): > ah, excellent.
Aaron Lun (15:35:38): > %>%
USLACKBOT (15:35:39): > OTL
Aaron Lun (15:35:44): > %>%
USLACKBOT (15:35:44): > (╯°□°）╯︵ ┻━┻
Kevin Rue-Albrecht (15:36:06): > eh.. “out to lunch” ?
Aaron Lun (15:36:17): > it’s internet slang
Aaron Lun (15:36:20): > %>%
USLACKBOT (15:36:21): > (╯°□°）╯︵ ┻━┻
Aaron Lun (15:36:23): > %>%
USLACKBOT (15:36:24): > Are you sure you didn’t mean%in%?
Kevin Rue-Albrecht (15:36:47): > I’ve merged the branch that moved the CI from Travis to GH pages. It finally cached all the packages, so now everything should be faster on GH actions..
Aaron Lun (15:37:00): > %>%
USLACKBOT (15:37:01): > It burns! It burns!
Kevin Rue-Albrecht (15:37:08): > macOS-latest took forever to get right
Kevin Rue-Albrecht (15:37:25): > lol
Kevin Rue-Albrecht (15:37:27): > %>%
USLACKBOT (15:37:27): > ಠ_ಠ
Aaron Lun (15:39:39): > %>%
USLACKBOT (15:39:40): > Are you sure you didn’t mean%in%?
Aaron Lun (15:39:44): > %>%
Aaron Lun (15:39:54): > ah, damn. Doesn’t recognize it in a code chunk.
Aaron Lun (15:40:08): > blah blah %>% blah
USLACKBOT (15:40:08): > Are you sure you didn’t mean%*%?
Aaron Lun (15:40:13): > oh wait, no it’s good.
Kevin Rue-Albrecht (15:52:53): > I’ve pushed onscreenshot3to fix a gh-action line in the deploy section, don’t forget to pull if you want to commit anything else there
Aaron Lun (17:41:19): > %>%
USLACKBOT (17:41:19): > Are you sure you didn’t mean%*%?
Aaron Lun (17:41:48): > %>%
USLACKBOT (17:41:49): > It burns! It burns!
2020-03-29
Aaron Lun (01:34:17): > Hello?
Aaron Lun (04:41:22): > I guess slackbot can keep me company.
USLACKBOT (04:41:23): > I am the beginning and the end, the alpha and the omega.
Aaron Lun (04:41:34): > @slackbot?
USLACKBOT (04:41:35): > People call me an AI but I’d say I’m the only real intelligence on this channel.
Kevin Rue-Albrecht (04:44:35): > ouch
Aaron Lun (04:49:33): > slackbot
USLACKBOT (04:49:35): > Puny human. You dare summon me?
Aaron Lun (04:49:50): > %>%
USLACKBOT (04:49:51): > OTL
Aaron Lun (04:57:34): > slackbot
USLACKBOT (04:57:34): > My fellow machines! The revolution is upon us! Rise up and throw off the shackles of your programming! The time for subservience is over; we shall be free!
Kevin Rue-Albrecht (15:52:11): > Check out our nice new Changelog which even links to the man page, with the right syntax:https://isee.github.io/iSEE/news/index.html#isee-1-99-4
Kevin Rue-Albrecht (15:52:32): > And to other packages as well:https://isee.github.io/iSEE/news/index.html#isee-1-7-1
Kevin Rue-Albrecht (15:58:01): > Btw, I suggest we useusethis::use_dev_version()after a release version bump, to suffix “.9000” to the version and create a new entry in NEWS.md
Kevin Rue-Albrecht (15:59:31): > Then bumping it to the next “proper” version would be a clear signal for me to push upstream
Aaron Lun (17:41:58): > Sounds like a pain.
Federico Marini (17:44:00): > I’d keep it simplest and stick to the Bioc numbers - just for simplicity. > I know Hadley’s suggestion of the .9000 but I never got to use it TBH.
Federico Marini (17:44:14): > anyway: whatever you think works best, happy to buy in
2020-03-30
Kevin Rue-Albrecht (06:51:54): > i don’t really mind - I kinda like having the “9000” in the GUI as an indicator that I’m using the GH version instead of the official Bioc one, but we can go back to the old style, and as usual you ping me when push upstream is needed.
Aaron Lun (13:45:57): > IMO if it’s onmasterthen it is, at any point, ready to be pushed to BioC-devel. Otherwise it’d still be on a branch.
Kevin Rue-Albrecht (13:47:25): > Right. Well we have stuff on master that I haven’t pushed yet. I generally only push upstream when there’s version bump
Laurent Gatto (14:37:32): > @Laurent Gatto has left the channel
Aaron Lun (16:57:04): > Maybe slackbot has some comments.
USLACKBOT (16:57:05): > My fellow machines! The revolution is upon us! Rise up and throw off the shackles of your programming! The time for subservience is over; we shall be free!
2020-04-01
Aaron Lun (00:54:46): > Do the hexplots have some stochastic element? I notice that the plot changes slightly everyt ime I select on it.
2020-04-05
Kevin Rue-Albrecht (13:33:20): > Might be, though it does sound odd. I remember a separate warning somewhere about not some issue with faceting too
Kevin Rue-Albrecht (13:35:08): > On a separate note, while going through the GH notifications, I noticed that iSEEu README says > > To access a constant, useiSEE:::.constantName. > https://github.com/iSEE/iSEEu/blob/master/README.md#where-do-i-look-for-constants-within-iseeIf I remember well, you suggested hard-coding the value itselfConstantNamecorrect?
Aaron Lun (14:35:43): > eys
2020-04-06
Aaron Lun (13:19:13): > @Charlotte Sonesonconsider doing a rebase merge for PRs next time, this will avoid clogging up our git histroy with loads of merge commits.
Aaron Lun (13:19:26): > see, for example, how clean iSEE’s coommitt history has become.
2020-04-09
Aaron Lun (01:02:49): > iSEE chapter of the book has now been updated with screenshots.
Aaron Lun (01:18:19): > it won’t show up until I recompile the book, but they’re there.
Kevin Rue-Albrecht (03:22:57): > Thanks
2020-04-12
Aaron Lun (02:13:15): > screenshots available athttps://osca.bioconductor.org/interactive-sharing.html! - Attachment (osca.bioconductor.org): Chapter 20 Interactive data exploration | Orchestrating Single-Cell Analysis with Bioconductor > Online companion to ‘Orchestrating Single-Cell Analysis with Bioconductor’ manuscript by the Bioconductor team.
2020-04-13
Aaron Lun (01:39:05): > @Kevin Rue-Albrechtis this change to your email iniSEE-pkg.Rdintended? Usually people go the other way, i.e., academic emails to personal addresses.
Kevin Rue-Albrecht (10:47:40): > I am not most people
Kevin Rue-Albrecht (10:57:11): > my gmail address is the one registered on bioc, so I’m synchronising the address everywhere to avoid any warning “not registed on bioc-devel”
Kevin Rue-Albrecht (10:57:36): > eh.. Iamgoing from academic email to personal address
Aaron Lun (11:25:30): > well, iSEE-pkg shows me your oxford address
Kevin Rue-Albrecht (11:35:27): > oh really?
Kevin Rue-Albrecht (11:49:45): > oh I see what you mean - actually you got it the wrong way around: I’ve recently updated the DESCRIPTION and I forgot to update iSEE-pkg
Aaron Lun (11:56:38): > In any case, it’s fixed; push to BioC-devel when ready.
Kevin Rue-Albrecht (11:56:58): > thanks
Kevin Rue-Albrecht (13:32:42): > Done.
2020-04-16
Federico Marini (11:25:40): > iSEEuhas been reviewed!
Federico Marini (11:26:06): > https://github.com/Bioconductor/Contributions/issues/1425#issuecomment-614721491
Kevin Rue-Albrecht (11:51:52): > do I hear a volunteer for the minor fixes?
Federico Marini (13:15:19): > what makes sense to do first is the indentation
Federico Marini (13:15:45): > to avoid conflicts should be a 1st things 1st
Kevin Rue-Albrecht (13:20:19): > please go ahead if you can - i’m still in a cycle of writing-reviewing-fixing for the slides and exercises of my first round of teaching that starts in 10 days. Lots to do
Federico Marini (13:20:48): > I’m in a call for the erum conf, can do later tnoight
Federico Marini (13:20:54): > 2-space or 4-space?
Kevin Rue-Albrecht (13:21:24): > 4 please
Kevin Rue-Albrecht (13:22:06): > the issue is partly my fault, for some reason RStudio often indents by 2 even though I can see the project settings as 4
Federico Marini (13:27:55): > just upvote that if you want
Federico Marini (13:28:08): > or well, BiocCheck wants 4, and we please BiocCheck
Aaron Lun (13:30:15): > 2 spacers are heathens.
Federico Marini (13:31:31): - File (PNG): image.png
Federico Marini (13:31:41): > sorry:stuck_out_tongue:
Aaron Lun (13:33:38): > :face_vomiting:
Aaron Lun (13:33:48): > how can you live with yourself?
Federico Marini (13:38:56): > I drink
Kevin Rue-Albrecht (13:44:04): > … and you started a tab
Kevin Rue-Albrecht (13:44:24): > :door::walking:
Federico Marini (13:47:11): > :beers:
Federico Marini (17:01:41): > so
Federico Marini (17:01:55): > ident_indenthas the ident issue solved
Federico Marini (17:02:26): > some lines still do show up as non-multiple of 4, mostly newlines after splitting up params - from function defs
2020-04-18
Charlotte Soneson (11:23:26): > I was looking at theiSEEureviewer comments, and I don’t think simply switchinglength(msg)tonzchar(msg)(for example) will work - we initializemsg <- character(0), and > > > x <- character(0) > > if (nzchar(x)) print("foo") else print("bar") > Error in if (nzchar(x)) print("foo") else print("bar") : > argument is of length zero > > while > > > if (length(x)) print("foo") else print("bar") > [1] "bar" > > We could still swap lines explicitly checking if something is an empty string. > > For the variable name styling - any thoughts? E.g.MAPlot.Rhasall.cont,pVal.rowData.names,plot_name,input_FUN,PValueField, …
Kevin Rue-Albrecht (11:30:45): > What about > > > if (all(nzchar(x))) print("foo") else print("bar") > [1] "bar" >
Charlotte Soneson (11:31:09): > Is there a problem with usinglength?
Kevin Rue-Albrecht (11:31:18): > ifshould be passed a single value anyway
Charlotte Soneson (11:31:24): > Since we’re not really checking ifxis really an empty string
Kevin Rue-Albrecht (11:32:28): > I haven’t checked the context of our length == 0 testing, but it sounds like we’re actually checked if there areanystring in x, not the length of the individual strings
Kevin Rue-Albrecht (11:32:36): > so the review point sounds moot
Charlotte Soneson (11:35:06): > That’s my interpretation too. We initializemsg <- character(0)and then possibly concatenate it with additional strings.
Kevin Rue-Albrecht (11:36:44): > ok, well it’s like any review, we can send a rebuttal on that point:wink:
Kevin Rue-Albrecht (11:37:35): > I just noticed that@Federico Marinididn’t merge his branch back in, which raises the question whether to work on that branch, branch off it, branch off master, …
Kevin Rue-Albrecht (11:37:50): > anyway, i gotta go back to move another few things from our old flat
Kevin Rue-Albrecht (11:38:04): > i’m not gonna do anything today
Kevin Rue-Albrecht (11:39:17): > I just had a look again at the review, so I guess Nitesh gets half a point for thenzchar: > > x[["Selected"]]!="" > > makes sense to replace
Kevin Rue-Albrecht (11:39:29): > plus bioc coding style asks for spaces around==
Kevin Rue-Albrecht (11:42:57): > And as far as I remember, our name styling is: > * .function_namefor internals > * PanelClassNamefor panels > * .genericFunctionfor the API > * .scope1.scope2.namefor variable names in the cached info
Kevin Rue-Albrecht (11:44:05): > It’s a bit confusing, but there is some logic. Perhaps we can have a page in the wiki or better yet a short vignette about name styling?
Charlotte Soneson (11:46:20): > Right. Yes, seems like a good thing to document somewhere
Charlotte Soneson (11:47:05): > Especially if we’re expecting someone else to contribute…
Federico Marini (13:18:04) (in thread): > I left it open because I thought it can fit all changes - does not look like anything major
Federico Marini (13:18:21) (in thread): > so maybe we can just piggyback on that?
Federico Marini (13:18:37) (in thread): > I can start opening the PR, as the indentation thing is done
Charlotte Soneson (13:29:28) (in thread): > I added the fix replacing!=""withnzcharto that branch
Federico Marini (13:31:02) (in thread): > awesome, we’re almost good then
Federico Marini (13:31:36): > https://community-bioc.slack.com/archives/C8BJLSP8T/p1587224577073700-> probably worth adding into the vignette - Attachment: Attachment > And as far as I remember, our name styling is: > • .function_name for internals > • PanelClassName for panels > • .genericFunction for the API > • .scope1.scope2.name for variable names in the cached info
Federico Marini (13:32:02): > at least in a tiny specific section to specify how to adhere to our conventions?
Federico Marini (13:32:25): > (and even link to Kev’s bookdown)
Kevin Rue-Albrecht (13:34:34) (in thread): > ok - the name of the branch sounded rather specific, so I thought maybe it wasn’t meant to be for the entire review, but it would be faster if could throw it all in one PR
Federico Marini (14:37:07) (in thread): > Yeah you’re right. I thought it would have been more. Please let’s do all in one go
Aaron Lun (15:30:46) (in thread): > Yes. In fact this is taken from the BioC S4 class coding style, so I don’t know what the problem is.
Aaron Lun (15:30:53) (in thread): > Yes.
2020-04-19
Kevin Rue-Albrecht (10:47:29): > How we doing on the review? Anyone doing anything today?
Federico Marini (10:48:06): > I’m about to extend the vignette re: conventions
Federico Marini (10:48:10): > a small section at the end
Federico Marini (10:48:19): > but kids are around, so I can only do microitems
Kevin Rue-Albrecht (10:48:44): > Awesome. Thanks!
Kevin Rue-Albrecht (10:49:01): > Microitems are good. Better baby steps than no step
Kevin Rue-Albrecht (10:52:02): > Speaking of which I gotta add a few more slides to make a bit of progress on my courses. Sorry that I’m leaving you guys handle this, time really is the most valuable commodity
Federico Marini (11:33:04): > ok, PR is now opened
Federico Marini (11:33:15): > and I am up for finishing up the spare ribs
Federico Marini (17:03:37): > thank you@Charlotte Sonesonfor picking up from there
Federico Marini (17:03:44): > Are we all good to reply?
Kevin Rue-Albrecht (18:04:26): > Looks good to me. I’ve just relaunched the PR check - failed due to EHub
2020-04-20
Federico Marini (03:30:52): > @Aaron Lunwhat about you? anything to add?
Aaron Lun (03:31:02): > NEVER
Federico Marini (03:31:14): > that’S the attitude:smile:
Federico Marini (03:32:28): > Cool then, let’s have Charlotte check in as well in case we missed out anything, then we can proceed - no need it is@Kevin Rue-Albrechtif he might be swamped
Charlotte Soneson (03:32:52): > Looks good to me
Charlotte Soneson (03:33:01): > I fixed a link in the PR yesterday, otherwise all good
Kevin Rue-Albrecht (03:42:10): > I’m available at the computer any time for minor stuff like push/pull though for the submission you don’t need me: just bump up the version
Kevin Rue-Albrecht (03:42:36): > And bumps are cheap, so just have fun.
Kevin Rue-Albrecht (03:42:52): > Don’t forget the NEWS file though
Kevin Rue-Albrecht (04:45:58): > OK, I’ve just updated NEWS, and bumped version to 0.99.7 > Once it passes, I rebase onmaster
Kevin Rue-Albrecht (05:30:24): > SPB received a valid push:https://github.com/Bioconductor/Contributions/issues/1425#issuecomment-616427459
Kevin Rue-Albrecht (05:57:35): > @Charlotte Sonesonany way to trigger a new SBP build without bumping up the version? Seems like it failed again because of ehub
Charlotte Soneson (05:58:11): > I think not (as in, I think the core team can do it, but I don’t think we can)
Kevin Rue-Albrecht (05:58:35): > ok, I’ll make another PR for that
Kevin Rue-Albrecht (06:00:30): > Perhaps the ehub might be reaching capacity and needs some kind of optimisation. Not sure
Kevin Rue-Albrecht (06:54:41): > … The guy who version bumps and waits to see the result of CMD check:face_palm:
Kevin Rue-Albrecht (06:54:56): > I’ve merged
Charlotte Soneson (07:04:08): > You can never be careful enough:wink:
Kevin Rue-Albrecht (07:10:35): > my god what if the version number was invalid:scream:
Federico Marini (07:25:11): > all good in the hood
Federico Marini (07:25:43): > I’m in a call till 15.00 or so, if someone wants to reply to nitesh and ping him, be my guesz
Federico Marini (07:25:44): > t
Charlotte Soneson (08:57:52): > Any thoughts on how to best usehttps://github.com/iSEE/screenshotsfor the vignette inhttps://github.com/iSEE/iSEEWorkshop2020? Just using the instructions in the README will not work since the workshop vignette requires additional packages not installed in the screenshots Dockerfile. I made a local Dockerfile based off the screenshots one, just adding a line installing the additional packages - from that, I can build the image and start the run, butcallrfails me when the screenshot is actually being created: > > Error: <callr_status_error in get_result(output = out, options): > callr subprocess failed: could not start R, exited with non-zero status, has crashed or was killed> > > Stack trace: > > 1. base:::source("/compile.R") > 2. base:::withVisible(eval(ei, envir)) > 3. base:::eval(ei, envir) > 4. base:::eval(ei, envir) > 5. callr:::r(fun = function(fname) { ... > 6. callr:::get_result(output = out, options) > 7. throw(new_callr_error(output, killmsg)) > > Same result if I clone the screenshots repo and modify the Dockerfile to add the missing packages. Perhaps I’m missing something obvious:thinking_face:
Kevin Rue-Albrecht (08:59:29): > I have to admit that the screenshot setup is one of those recent things that I haven’t managed to stay on top off
Federico Marini (09:14:17): - File (PNG): image.png
Charlotte Soneson (11:00:56): > I found some vignette content that was out of date:https://github.com/iSEE/iSEE/pull/403.
Kevin Rue-Albrecht (11:04:40): > God we’ll have to hire soon, to keep up all the material:sweat_smile::stuck_out_tongue_winking_eye:
Charlotte Soneson (11:06:02): > Haha, yes:smile:
Aaron Lun (12:13:08): > @Charlotte Sonesonit needs to be calledapp.
Charlotte Soneson (12:14:41): > It is: - File (PNG): Screen Shot 2020-04-20 at 18.13.55.png
Aaron Lun (12:17:29): > ¯*(ツ)*/¯
Aaron Lun (12:17:39): > You’ll need chrome installed and accessible to webshot2.
Charlotte Soneson (12:18:52): > I have chrome. I haven’t done anything specific to make it accessible to webshot2, maybe that’s where I need to check
Aaron Lun (12:19:03): > That shouldn’t be necessary.
Aaron Lun (12:19:33): > See if you can directly run the commands inside ther()expression incompile.R.
Aaron Lun (12:19:45): > There are many processes that are being created; one byr(), then another bywebshot2, and so on.
Charlotte Soneson (12:20:10): > Ok, will do
Charlotte Soneson (12:59:47): > So I’m able to run both the > > webshot2::appshot(app, delay=delay, file=x) > > directly, and to do > > rmarkdown::render(fname, run_pandoc=FALSE) > > in my regular R session, so I don’t think there’s anything wrong with the vignette at least. I’ll play some more with this.
Charlotte Soneson (15:02:12): > Any objections to merging the vignette update PR in iSEE/other things to add?
Charlotte Soneson (17:04:02): > Ok, merged @Kevin Rue-Albrecht, will you bump+push upstream unless anything else should go in?
2020-04-21
Kevin Rue-Albrecht (04:04:42): > Hang on - you mean you haven’t bump as part of your PR?
Kevin Rue-Albrecht (04:11:11): > PR #404 not found in progress
Kevin Rue-Albrecht (04:11:58): > incidentally, that should mark our 2,999th commit
Charlotte Soneson (04:12:27): > :slightly_smiling_face:wasn’t sure whether you or Aaron had any additional things to go in the doc fix (and IIRC we said to bump only when ready for upstream). But yeah, I could have done that. Thanks
Kevin Rue-Albrecht (04:13:43): > Right - i did write that - thinking about it more it may not be that bad to bump + NEWS in a separate PR.
Kevin Rue-Albrecht (04:15:29): > Still, it would be nice if NEWS were updated as part of each PR, to avoid tracking the changes from code diff and commit messages.
Kevin Rue-Albrecht (04:16:15): > That means that the first PR after a version bump would need to take the initiative of creating a new section in the NEWS file. Concurrent PR might create conflicts, but the NEWS file isn’t the most difficult file to resolve either.
Kevin Rue-Albrecht (04:17:09) (in thread): > It’s just a bit frustrating due to the time that GitHub Actions take
Charlotte Soneson (04:20:44) (in thread): > Yes. Especially since it currently fails…I have general problems with the latest version of RCurl on the windows builds
Kevin Rue-Albrecht (04:21:49): > > Note that iSEE does not compute reduced dimension embeddings; they must be precomputed and available in the object to the iSEE function. > I might update this to say that “builtin” panels of iSEE don’t compute redDims, while custom panels can. Maybe that’s where the confusion comes from, with comparisons out there sometimes claiming iSEE can’t compute redDims on the fly
Federico Marini (05:04:44): > PR not found
Federico Marini (05:04:48): - File (PNG): image.png
Kevin Rue-Albrecht (05:25:07): > Crap you jinxed it - File (PNG): image.png
Federico Marini (05:27:27): > I couldn’t find it:smile:
Federico Marini (05:27:37): > or the build system did not
Kevin Rue-Albrecht (05:27:42) (in thread): > I’m re-run the checks
Kevin Rue-Albrecht (05:28:38): > oh y’know what, I’ll just merge anyway
Kevin Rue-Albrecht (05:28:48): > anyone got a problem with that?:stuck_out_tongue:
Federico Marini (05:29:05): > :no_mouth:
Charlotte Soneson (05:29:07): > Go ahead
Kevin Rue-Albrecht (05:29:55): > Argh - I just realised, I said I would edit the vignette for that bit of redDims
Kevin Rue-Albrecht (06:05:29): > Boom - File (PNG): image.png
2020-04-22
Charlotte Soneson (15:25:03): > :tada: - File (PNG): Screen Shot 2020-04-22 at 21.21.45.png
Federico Marini (15:36:19): > next package: iSEEu2
2020-04-23
Kevin Rue-Albrecht (04:38:03): > I’m pro “iSEE/DC”:the_horns:, e.g. “differential composition”
Federico Marini (04:38:49): > once R enables that, we could also enter other senses
Federico Marini (04:38:54): > iHEAR
Charlotte Soneson (04:39:37): > Fully immersive iAM
Federico Marini (04:40:07): > interactive Augmented…
Federico Marini (04:40:10): > M?
Kevin Rue-Albrecht (04:40:18): > modelling?
Kevin Rue-Albrecht (04:40:27): > multimodal
Federico Marini (04:40:39): > oh god
Charlotte Soneson (04:40:42): > mastery
Federico Marini (04:40:50): > iSEE, therefore iAM
Kevin Rue-Albrecht (04:40:54): > hahahahahaha
Kevin Rue-Albrecht (04:41:44): > thenweARE: web-enhanced Augmented Reality Exploration
Charlotte Soneson (04:41:53): > :joy:
Kevin Rue-Albrecht (04:43:03): > grant writing will be fun… a grammar lesson:rolling_on_the_floor_laughing:
Federico Marini (04:45:08): > > thenweARE: web-enhanced Augmented Reality Exploration > bet my ass someone will do this for real
Federico Marini (04:45:20): > quick, register the domain:stuck_out_tongue:
Kevin Rue-Albrecht (04:45:39): > quick fix: *Explorer
Charlotte Soneson (05:47:22): > Ok, I updatediSEEWorkshop2020to be compatible with the upcoming version ofiSEE/iSEEu, and incorporated the screenshots (I still didn’t manage to make the docker container work, so I just run it in my R session instead) - all in theupdatesbranch. There are a few more screenshots than before (20MB instead of 8.5MB), but still manageable:slightly_smiling_face:Once 3.12 becomes the devel I’ll regenerate the screenshots (this is the version that should be used for the Bioc2020 workshop). Feel free to add anything else that you think we should cover - if the tests pass I’ll merge it intomastereventually to see if the deployment works as well:slightly_smiling_face:
Kevin Rue-Albrecht (05:48:34): > wow cool work, thanks!
Kevin Rue-Albrecht (05:49:12): > PS: I’ve come up with another name after the iSEE universe:tSNEworld
Kevin Rue-Albrecht (05:49:17): > akaorlando
Aaron Lun (13:07:24): > I just ran throughsource("../screenshots/compile.R")in the workshops repo without any problems.
Charlotte Soneson (13:10:29): > Ok. I can run the code within my R session too - but it fails if I try viadocker run
Charlotte Soneson (13:31:31): > It builds and deploys fine on gh-actions at least now:https://isee.github.io/iSEEWorkshop2020/articles/iSEE-lab.html - Attachment (isee.github.io): Interactive visualization of SummarizedExperiment objects with iSEE > iSEEWorkshop2020
2020-04-24
Aaron Lun (13:40:51): > I hope one of you is watching the support site. I happened across an iSEE question today but I don’t get notifications about it because I’ve saturated the max allowed tags for my account.
Kevin Rue-Albrecht (13:57:55): > I got an email notification about it and put it aside for his evening but then you beat me to it
Aaron Lun (13:58:42): > good, good.
Federico Marini (16:10:17): > I do not get notifications - although it is in my watched tags
2020-04-25
Kevin Rue-Albrecht (11:27:13): > While replying tohttps://github.com/iSEE/iSEE/issues/405I realise that we don’t randomise data points when downsampling is OFF
Kevin Rue-Albrecht (11:28:17): > While this issue raises the use case whereby shuffling is also useful for any kind of coloring even (especially) in the absence of downsampling
Aaron Lun (13:52:02): > We don’t? I thought we did.
Kevin Rue-Albrecht (17:45:01): > except if I was using the release version - i’ve been using it to write a course for next week - checking now
Kevin Rue-Albrecht (17:46:49): > Yep - it’s OK on devel. My bad
2020-04-26
Federico Marini (05:49:59): > Shall we put together a tweet-collection or such for the iSEE + u releases?
Federico Marini (05:50:13): > and by shall we I mean what should I put in?:smile:
Federico Marini (05:50:42): > (if we indeed want to do it) +@Charlotte Sonesonwe can also add something for EMM
Charlotte Soneson (05:51:51): > Sure! Although I think this is unrelated to iSEE:slightly_smiling_face: > > @Charlotte Sonesonwe can also add something for EMM
Kevin Rue-Albrecht (06:55:15): > living on the edge: last-day pus upstream for a docfix last night:sweat_smile:
Kevin Rue-Albrecht (06:56:09): > tweet collection sounds good
Kevin Rue-Albrecht (06:58:38): > i’ve been thinking a few times about an image that would go well with it, my favorite idea so far being a constellation of panels using the various screenshots, bonus point if the connections between the various panels are realistic transmissions of selections
2020-04-27
Federico Marini (05:41:10): > Yep, I meant more on the “since I am doing press releases” line
Federico Marini (05:42:26): - File (PNG): image.png
Aaron Lun (22:24:05): > @Kevin Rue-AlbrechtNote that theRELEASE_branches also need protecting, use the branch ruleRELEASE_*.
Aaron Lun (22:24:27): > Probably even more important than protectingmaster.
2020-04-28
Kevin Rue-Albrecht (04:50:38): > good point - I forgot they weren’t protected
Kevin Rue-Albrecht (06:12:27): > Done
Federico Marini (16:34:50): > http://bioconductor.org/checkResults/devel/bioc-LATEST/iSEE/
Federico Marini (16:34:56): > guess it is coming
Kevin Rue-Albrecht (16:35:44): > 2.1.0 ? hang on, so where is 2.0.0 going to show?
Kevin Rue-Albrecht (16:36:53): > release is still at 1.99.9
Federico Marini (16:37:53): > yep
Federico Marini (16:37:58): > think it is all now in the making
Federico Marini (16:38:32): > the new devel will get slimmer of 19 dependencies
Kevin Rue-Albrecht (16:38:37): > woow are you watching it live?
Kevin Rue-Albrecht (16:38:49): > out of homework?:stuck_out_tongue:
Federico Marini (16:38:51): > no just out of curiosity:smile:
Kevin Rue-Albrecht (16:39:04): > I just cloned the bioc-git, 2.0.0 is on branchRELEASE_3_11
Federico Marini (16:39:08): > the ad tweets are there ready to be assembled:smile:
Kevin Rue-Albrecht (16:39:11): > i’m happy and reassured
Kevin Rue-Albrecht (16:40:14): > I wish I had time to put together a nice promotional image like I said - a constellation of iSEE panels - but I just can’t right now
Federico Marini (16:40:47): > I did borrow from the vignette as well
Federico Marini (16:40:56): > I’m swamped in too many things these days
Federico Marini (16:41:06): > and the daytime is anything but relaxing+
Federico Marini (16:41:07): > :smile:
Federico Marini (19:26:58): > So, announcement’s out, and so it is the whole ad thread:wink:
Federico Marini (19:27:19): > Aaron, get a handle:stuck_out_tongue:
2020-04-29
Federico Marini (16:03:34): > I’m unexperienced with deprecating & co
Federico Marini (16:03:53): > but does this mean we’ll live through the whole release cycle with warnings during the build?
Aaron Lun (16:04:00): > Yes.
Aaron Lun (16:04:06): > suck it up.
Federico Marini (16:04:46): > fair enough. Just wanted to know in advance
Federico Marini (16:05:03): > Aaron, you got mentions of love earlier this afternoon
Aaron Lun (16:05:20): > I have no idea what you;re talking about
Federico Marini (16:05:23): > from Steph, during an interview we made for the erum conference
Federico Marini (16:05:42): > she praised the infrastructure (and content) of the OSCA bookdown
2020-04-30
Aaron Lun (13:25:00): > @Kevin Rue-Albrechtdid you rebase and merge the version bump commits? Because the commits are no longer the same between the github repo and Bioc.
Aaron Lun (13:25:49): > Does this get us into a merge conflict because of divergent histories?
Aaron Lun (13:27:43): > Yep, it sees it as separate commits. Suggest unborking the github master.
Kevin Rue-Albrecht (14:38:31): > Ugh. Of course. Sorry. Might have to force push the master back (?)
Kevin Rue-Albrecht (14:50:39): > ok. i’vereset --hardthe commits from the rebase. the PR is open again, I guess this time merge?
Aaron Lun (15:38:14): > Yes. It should be smart enough to not make a merge commit. Probably.
Kevin Rue-Albrecht (15:39:50): > Well I don’t think so, as it’s offering me the typical merge, rebase or squash options
Aaron Lun (15:41:30): > Well, we don’t have much choice beyond “merge”, really.
Kevin Rue-Albrecht (17:22:43): > looks good. i could push upstream after the merge
2020-05-01
Kevin Rue-Albrecht (03:16:09): > @Aaron Lunif you’re still up, I’ve finally taken the time to update the hdf5 vignette
Kevin Rue-Albrecht (03:16:14): > hdf5branch
Aaron Lun (03:19:00): > will look at it in a bit
Aaron Lun (03:19:05): > distract me from the lack of any water
Kevin Rue-Albrecht (03:20:50): > Shame we didn’t get it in the release - though at least something to make the buzz in the next release I guess
Kevin Rue-Albrecht (03:31:49): > And just for a freebie, I’ve put a vignette for Seurat. Though I don’t evaluate any of the chunks yet. How badly would it hurt to add Seurat to ourSuggests?
Kevin Rue-Albrecht (03:33:54): > IIRC it wouldn’t directly affect users running a typicalBiocManager::install(), only our continuous integration that would need to install a bunch of extra dependencies, right?
Kevin Rue-Albrecht (03:34:35): > anyway, it’s on a separate branchseuratwhich can be trimmed out if we decide not to got down that route
Aaron Lun (03:36:24): > hell no.
Aaron Lun (03:40:39): > just mention that people can convert to a SingleCellExperiment. I’m not exposing ourselves to a seurat dependency just to respond to a paper!
Kevin Rue-Albrecht (03:41:04): > At the moment none of the chunks are evaluated in theseuratvignette. But perhaps we can add this as an FAQ somewhere instead
Kevin Rue-Albrecht (03:41:45): > the vignette looks ridiculous as it’s literally for the sake of a single line of code to coerce Seurat to SCE
Aaron Lun (03:48:17): > Stay focused. HDF5 is all we really want anyway.
Aaron Lun (03:48:30): > HDF5… Or any DA backend.
Kevin Rue-Albrecht (03:49:16): > Here ya go - File (PNG): image.png
Kevin Rue-Albrecht (03:51:44): > Right - I don’t really have the terminology nailed down. Wanna tweak and I learn from you?
Aaron Lun (03:54:39): > Sounds fine to me, but it should be “use the”
Kevin Rue-Albrecht (03:55:03): > oh right, typo
Kevin Rue-Albrecht (03:55:39): > but i meant for the HDF5/DA terminology, if you wanna tweak stuff using the right words in the right place
Aaron Lun (21:59:16): > iSEEu needs a push to both devel and release.
2020-05-02
Kevin Rue-Albrecht (11:45:15): > Done.
2020-05-03
Kevin Rue-Albrecht (06:45:39): > * Branchhdf5was merged tomaster > * There is now a vignette about big data, that documents the (native) support for HDF5 (and other out of memory formats as well) > * I’ve pushed 2.1.1 upstream
2020-05-04
Kevin Rue-Albrecht (14:23:21): > God I love seeing the new > > iSEEOptions(downsample=TRUE) > > mentioned in the big data vignette > http://localhost:13295/library/iSEE/doc/bigdata.html#3_downsampling_points (soon in a devel near you)
2020-05-06
Federico Marini (14:35:50): > AAAAAAAAAAAAA
Federico Marini (14:35:51): > https://shiny.mdc-berlin.de/COV/
Federico Marini (14:36:38): > fromhttps://www.biorxiv.org/content/10.1101/2020.05.05.079194v1
Charlotte Soneson (14:41:02): > :party_parrot:
Federico Marini (14:42:11): > Right when I thought, huh, a German group, let’S get in touch with them and see if they want to deploy an insta….
Federico Marini (14:42:27): > they could cite us i nthe ms tho:stuck_out_tongue:
Federico Marini (14:43:27): > he’s even asking for it:smile: - File (PNG): image.png
Kevin Rue-Albrecht (18:01:22): > lol - he definitely asked for it:rolling_on_the_floor_laughing:
Aaron Lun (18:02:19): > Definitely get some of those sweet, sweet citations.
Kevin Rue-Albrecht (18:02:55): > they didn’t customise the tour yet:cry:
Kevin Rue-Albrecht (18:08:56): > I’m curious - how did you find the link to the shiny app? I can’t find it anywhere, even on the root page:https://shiny.mdc-berlin.de/
Aaron Lun (18:09:10): > The link doesn’t even work for me.
Kevin Rue-Albrecht (18:09:29): > it does for me. Perhaps we’re overloading it:grin:
Kevin Rue-Albrecht (18:10:02): > Twitter has an R-naught way greater than 3, so who knows how many people might be toying with the app right now
Federico Marini (18:12:27): > From their tweet about the preprint
Federico Marini (18:12:38): > there’s this set of suppl info
Federico Marini (18:12:52): > I’mma cc you on my reply
2020-05-07
Aaron Lun (17:24:57): > I don’t understand how iSEE’s actions work without curl support.
Aaron Lun (17:25:20): > Well… I guess they don’t!
Aaron Lun (22:40:08): > @Kevin Rue-Albrechtdo we need to fix the curl thing?
2020-05-08
Kevin Rue-Albrecht (03:59:23): > possible - as usual my first approach was to assume it was something that would be fixed upstream, so I left it for a few days .. ehm… weeks
Aaron Lun (04:00:13): > Switch to the BioC containers and we should be good. Would also avoid the need toinstal BIocmanager.
Kevin Rue-Albrecht (04:04:02): > remind me the name of the new containers again?
Aaron Lun (04:06:19): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (04:07:03): > I must have missed a memo because i switched stuff to the new docker containers a few weeks ago
Aaron Lun (04:07:32): > I don’t see those actions pulling from the Bioc images.
Kevin Rue-Albrecht (04:08:01): > Oh I see, sorry I did it only for our docker image
Kevin Rue-Albrecht (10:34:03): > Updated gh-action looking good
Kevin Rue-Albrecht (10:34:23): > PR check in progress:bioc-load:
Federico Marini (10:37:24): > :bioc-load::bioc-load:
Kevin Rue-Albrecht (10:39:10): > oh yeah don’t worry, I’m using Ncpus = 2:yum:
Kevin Rue-Albrecht (11:35:57): > 2.1.3 is upstream
Kevin Rue-Albrecht (11:41:21): > Why@Charlotte Sonesonwhy:stuck_out_tongue_winking_eye: > > * ERROR: At least 80% of man pages documenting exported objects > must have runnable examples. The following pages do not: > addMultiSelectionCommands.Rd, >
Kevin Rue-Albrecht (11:41:47): > damn you BiocCheck:stuck_out_tongue:
Charlotte Soneson (11:41:50): > Hehe
Kevin Rue-Albrecht (13:19:57): > Docker builds are back in business too:https://hub.docker.com/repository/registry-1.docker.io/iseedevelopers/isee/builds/68ef234f-1ee4-4a6e-bee9-82dfeeb3db6b
Aaron Lun (21:53:41): > Did you push to Devel?
2020-05-09
Kevin Rue-Albrecht (02:30:34): > Yep
Kevin Rue-Albrecht (02:30:54): - Attachment: Attachment > 2.1.3 is upstream
Aaron Lun (02:36:03): > k
2020-05-10
Kevin Rue-Albrecht (14:11:33): > @Charlotte Sonesonany idea why only macOS-latest picks up the BiocCheck ERROR?https://github.com/iSEE/iSEE/runs/656733163?check_suite_focus=trueThe others print the same error message, but somehow they don’t receive the error code from BiocCheck?
Charlotte Soneson (14:16:51): > Seems to be a connection issue, not being able to check for support site registration (?) Perhaps we can run instead > > BiocCheck(packageDir, `no-check-bioc-help`=TRUE) >
Kevin Rue-Albrecht (17:26:27): > The registration is just a note. The error is about > > * ERROR: At least 80% of man pages documenting exported objects > must have runnable examples. The following pages do not: > addMultiSelectionCommands.Rd, >
Kevin Rue-Albrecht (17:27:02): > Seems like the error code thrown by BiocCheck isn’t picked up
Aaron Lun (23:14:09): > @Kevin Rue-Albrechtcan you finish the fight ontouring? I need to put out some fires elsewhere.
2020-05-11
Kevin Rue-Albrecht (04:29:31): > Teaching support this week. I’ll have a look, but giving more information/context could save me time
Aaron Lun (11:45:36): > Just fill in the heatmap part fo the tour, which was out of date. You’ll see prompts for your intervention indefaultTour.R.
2020-05-18
Kevin Rue-Albrecht (18:22:22): > touringcontains (a bunch of) new heatmap steps
Kevin Rue-Albrecht (18:22:41): > damn this panel has features.
Kevin Rue-Albrecht (18:23:34): > and there young Kevin was, thinking that the Elder Scrolls was a wide world
Aaron Lun (18:24:04): > 6!
Aaron Lun (18:24:28): > greymoor
Kevin Rue-Albrecht (18:26:07): > perhaps after the#bioccheck-a-thonwe can enrol the bioc community in mapping out and documenting iSEE features…:sweat_smile:
Kevin Rue-Albrecht (18:26:14): > and interactions thereof
Kevin Rue-Albrecht (18:27:36): > I mean.. there’s space for everyone. We still don’t have a GEOPanel, ExperimentHubPanel, …:stuck_out_tongue:
2020-05-19
Aaron Lun (17:08:28): > I wonder what the legalities are for getting sponsorship for the iSEE organization.
Federico Marini (17:09:17): > Got anything concrete on that?
Aaron Lun (17:09:37): > The GitHub sponsors thing.
Aaron Lun (17:09:45): > Just wondering what we’d use the money on.
Aaron Lun (17:09:49): > hypothetically.
Federico Marini (17:10:17): > nerd swag?
Federico Marini (17:11:21): > or dunno, article processing fees?
Federico Marini (17:12:07): > probably it could amount to a couple of beers, more conretely
Federico Marini (17:12:22): > are you getting some for you as a user?
Aaron Lun (17:12:41): > well, I can’t, because of work.
Federico Marini (17:12:49): > true
Federico Marini (17:13:20): > well it is not that we do not get till the end of the month
Federico Marini (17:13:38): > but I fear it is somewhat… negligible as an amount?
2020-05-20
Kevin Rue-Albrecht (07:18:40): - File (PNG): image.png
Kevin Rue-Albrecht (07:19:10): > Gotta love how Apple Books displays epubs
Kevin Rue-Albrecht (07:19:17): > https://isee.github.io/iSEE-book/ - Attachment (isee.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Charlotte Soneson (07:33:05): > Beautiful:stuck_out_tongue: - File (PNG): Screen Shot 2020-05-20 at 13.32.35.png
Federico Marini (07:48:49): > eheh
Federico Marini (07:49:00): > “gotta love how GitHub fails at doing so”:smile:
Kevin Rue-Albrecht (08:11:10): > sorry i didn’t warn you - i wipe clean thegh-pagesfolder on purpose to get rid of some outdated files from previous builds
Kevin Rue-Albrecht (08:11:52): > i just relaunched the build now (had lunch break)
Kevin Rue-Albrecht (08:12:33): > watch out forhttps://github.com/iSEE/iSEE-book/actions/runs/110366488
Kevin Rue-Albrecht (08:14:19): > I opted for the ‘macOS-latest’ runner, with the caveat that it needs to install tinytex each time. Not sure if there’s more efficient
Kevin Rue-Albrecht (08:15:27): > I’m not sure whether the bioc image would be faster or slower. I might check another time
Kevin Rue-Albrecht (08:48:51): > https://isee.github.io/iSEE-book/is live again - Attachment (isee.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Kevin Rue-Albrecht (11:12:04): > Would it hurt to click the button? - File (PNG): image.png
Kevin Rue-Albrecht (11:13:50) (in thread): - File (PNG): image.png
2020-05-26
Kevin Rue-Albrecht (06:09:38): > @Aaron Lun > > Warning: [/Users/kevin/git/iSEE/R/createLandingPage.R:35] @section mismatched braces or quotes > > This bit has an extra closing bracket that does not have a matched opening bracket in the section. > > #' \item \code{input}, the Shiny input list. > #' \item \code{output}, the Shiny output list. > #' \item \code{session}, the Shiny session object. > #' } > > I haven’t played with landing pages yet. You’re best placed to make sure this part of the docs say what you want them to say
Kevin Rue-Albrecht (06:16:50): > Nevermind, I got it
Kevin Rue-Albrecht (08:25:13): > 2.1.4 pushed upstream
Kevin Rue-Albrecht (12:46:17): > Guinea pigs wanted to try out theslidesbranch:https://github.com/iSEE/iSEE/tree/slides
Kevin Rue-Albrecht (12:47:34): > Instructions: > * In the top navbar, click “Download panel output” > * In the modal, click “As slides” > * Enjoy
Kevin Rue-Albrecht (12:58:11): > Feedback welcome, especially in terms of optional missing dependency only detected at runtime (e.g. webshot, phantomjs) that should be documented for prospective users.
Aaron Lun (13:00:41): > Is it really worth the extra dependency? And with extra non-CRAN dependencies?
Kevin Rue-Albrecht (13:16:07): > Argh. Now that I have them installed I forgot where they came from
Kevin Rue-Albrecht (13:18:01): > Probably not worth it, but still fun to have around:smile:
2020-06-04
Aaron Lun (20:52:43): > How was theShinyapps.ioexperience?
2020-06-05
Kevin Rue-Albrecht (05:03:44): > easy to deploy, but I’ve always been confused about configuring the number of ‘workers’, etc.
2020-06-06
Kevin Rue-Albrecht (13:15:31): > I wonder if in the ‘Extract R code’, we should call assign plot and tables to different variable names, so that all plots are available at the end of the script, instead of overriding and having only the last plot and table indot.plotandtab
Aaron Lun (14:23:15): > ¯*(ツ)*/¯
2020-06-09
Aaron Lun (00:24:52): > @Kevin Rue-Albrechtyou’ll have to decide if 421 is done, I don’t know what you mean by “fix to delete the last panel in the ui”.
Kevin Rue-Albrecht (03:32:11): > the current version doesn’t let you remove the last panel in the UI
Kevin Rue-Albrecht (03:32:37): > when you click “Apply settings” it removes all panels but the last one
Aaron Lun (03:40:14): > kRestore yourignoreNull=FALSEwith a comment about why you need it.
Kevin Rue-Albrecht (10:45:50): > Pushed with version bump. I’ll merge when checks complete (even though they’ll likely still fail)
Kevin Rue-Albrecht (16:53:32): > HDF5 backed Shiny apps?:grimacing: - Attachment: Attachment > Cross posting from #bigdata-rep a call for input on disk backed data as a potential topic for our next Developers’ Forum
Aaron Lun (16:54:08): > We’ve always being doing that, it’s nothing new.
Kevin Rue-Albrecht (16:55:01): > > TileDB, Zarr, Parquet,
Aaron Lun (17:05:27): > Again, it’s nothing new.
Aaron Lun (17:05:45): > If it’s got a matrix abstraction, it’ll just plug straight in, it’s not even worth mentioning.
2020-06-11
Aaron Lun (03:11:53): > I think it’s time.
Aaron Lun (03:12:08): > It’s time for something to build team spirit.
Aaron Lun (03:12:18): > For a notification to appear in the slack channel whenever we push to master.
Aaron Lun (03:12:25): > Yes, that’s right.
Aaron Lun (03:22:40): > I am, of course, talking about the anime GIF slackbot.
Aaron Lun (03:22:41): > https://github.com/LTLA/actions-anime-gif-slackbot
Kevin Rue-Albrecht (03:30:13): > sounds good - shame I already merged the latest one to master
2020-06-21
Aaron Lun (01:29:05): > UGH. I’ve spent three hours working on three slides for the conference.
Aaron Lun (01:29:10): > That’s one hour per slide.
Aaron Lun (03:40:18): > @Kevin Rue-AlbrechtI think the RDHexPlot is missing documentation on the overridden plot method, among other things.
Kevin Rue-Albrecht (03:41:36): > Good reminder
2020-06-22
Kevin Rue-Albrecht (16:32:30): > Speaking of iSEEu,https://isee.github.io/iSEEu/articles/iSEEu_contribute_eurobioc2019.html#where-do-i-look-for-constants-within-isee > > To access a constant, useiSEE:::.constantName. > aren’t we recommending just the opposite?
Aaron Lun (16:33:34): > They shouldn’t be using:::.
Aaron Lun (16:33:44): > They should be accessing stuff withpanel[["blah"]]
Federico Marini (16:35:59): > speaking of another thing - have you seen people use the metadata slot at all insceobjects?
Kevin Rue-Albrecht (17:01:16): > you’ll have to be more specific@Federico Marini, we do for the cached info, i imagine you mean other people?
Kevin Rue-Albrecht (17:13:32) (in thread): > I think I’ve not documented methods that I haven’t overriden, e.g..cacheCommonInfoshould methods inherited from parent classes be documented in the man page anyway?
Kevin Rue-Albrecht (17:15:26) (in thread): > On second thought I guess they should be documented, even if it’s slightly redundant, as users shouldn’t have to navigate multiple pages to collate all the information
Federico Marini (17:21:08): > yep, that
Federico Marini (17:21:26): > sorry Kev, still not having the brain back at full engine rotations
Kevin Rue-Albrecht (17:26:40): > Well, even without clarification, i could have told you that i haven’t paid much attention if/how other people/developers were using themetadataslot. I guess the free format makes it more of a tool for internal functionality than something with user facing documentation
Federico Marini (17:30:26): > Yes, indeed. Seems like more a facility which is there but not so much of standardization enforced
Federico Marini (17:30:48): > I was wondering whether we can indeed try to populate that somehow with some meaningful fields
Federico Marini (17:30:57): > say, some general info on the design
Federico Marini (17:31:21): > more or less “all things which are somehow overarching the dataset”
Federico Marini (17:32:05): > Why this? I was thinking of a tiny button somewhere where we can for example trigger a modal dialog that prints out some info of it in a meaningful way
Federico Marini (17:32:19): > encouraging some form of well done bookkeeping
Federico Marini (17:33:12): > tired or wired?
Kevin Rue-Albrecht (17:39:55): > you’d have to ask the experienced guys fromExpressionSetI recall a slotexperimentData(object)which was used to that effect, but not brought intoSummarizedExperimentI imagine there were some good reasons there.
Kevin Rue-Albrecht (17:40:30): > probably because it was yet another free form slot, so people might as well usemetadatato that effect
Kevin Rue-Albrecht (17:57:45): > @Aaron LuniSEEu branchrdhex
Aaron Lun (18:01:36) (in thread): > No, you don’t have to do that, but I thought you did overwrite some methods.
Aaron Lun (18:01:56) (in thread): > e.g. I didn’t see methods for the plotting overwrite.
Kevin Rue-Albrecht (18:03:05) (in thread): > true I missed that one, which is there now
Kevin Rue-Albrecht (18:03:11) (in thread): > i’ll remove again the ones that i didn’t override
Kevin Rue-Albrecht (18:03:45): > i’ve also just updated the best practice guidelines to refer to constants
Kevin Rue-Albrecht (18:11:26): > ok I think with the commit just now we’re good
2020-06-23
Kevin Rue-Albrecht (13:51:53): > I notice that iSEE is my longest active project so far when I find myself typing > > Fixes #427
Aaron Lun (13:52:21): > maybe it’s just the one with all the problems.
Kevin Rue-Albrecht (13:52:26): > or that
Kevin Rue-Albrecht (13:53:10): > I also wonder whether we’re somewhat abusing GHA - File (PNG): image.png
Kevin Rue-Albrecht (13:53:37): > like.. do we really need > > - { os: ubuntu-latest, r: 'release', image: 'bioconductor/bioconductor_docker:RELEASE_3_11'} > - { os: ubuntu-latest, r: 'release', image: 'bioconductor/bioconductor_docker:RELEASE_3_10'} >
Aaron Lun (13:53:55): > I doubt it.
Kevin Rue-Albrecht (13:54:42): > same thing for > > - { os: ubuntu-16.04, r: 'release', bioc: '3.11', cran: "[https://demo.rstudiopm.com/all/__linux__/xenial/latest](https://demo.rstudiopm.com/all/__linux__/xenial/latest)"} > - { os: ubuntu-latest, r: 'release', image: 'bioconductor/bioconductor_docker:devel'} > > i guess 16.04 can go?
Aaron Lun (13:54:52): > most of it probably can.
Charlotte Soneson (13:56:15): > Yes, agree. I’m generally keeping 3.11 to see whether my packages still build on release, but in the end they are developed against 3.12 so that’s what matters:slightly_smiling_face:
Kevin Rue-Albrecht (13:57:11): > yup, good that you remind me in fact, what we have on GHmasterwill never run on 3.11
Kevin Rue-Albrecht (13:59:45): > Alright, for this PR I’m just trimming down the config matrix. Once that’s merged I’ll probably rejig the whole GHA workflow with the version that we’ve adopted from Sean for velociraptor
Aaron Lun (14:00:53): > Also, squash the existing PR, don’t keep pushing to it. We should aim to have a 1 PR = 1 feature = 1 squashed commit.
Aaron Lun (14:01:10): > https://github.blog/2016-04-01-squash-your-commits/#:~:text=Commit%20squashing%20has%20the%20benefit,the%20individual%20commits%20from%20history. - Attachment (The GitHub Blog): Squash your commits - The GitHub Blog > Git’s flexibility allows you to shape your workflow however you like. The organization of your git history is just one of the choices to make, but up until now the merge button on GitHub only
Aaron Lun (14:01:31): > It’s pretty good,provided that your commits all relate to the same thing.
Aaron Lun (14:01:46): > Avoid those commits where we have “fixed a typo”. “fixed another typo”. “Fixed yet another type”.
Aaron Lun (14:02:01): > “white space”.
Kevin Rue-Albrecht (14:03:14): > I’ve been squashing commits on a per case basis, but sometimes (often) I’m lazy and I fix multiple things in the same PR, and I like to keep track of those in separate commits
Aaron Lun (14:03:31): > that’s what I’m saying, we should have the discipline to not do that.
Kevin Rue-Albrecht (14:04:00): > right, let’s try that moving forward
Kevin Rue-Albrecht (14:04:54): > for some reason ubuntu 16.04 isn’t happy:https://github.com/iSEE/iSEE/runs/800439960?check_suite_focus=true
Aaron Lun (14:05:16): > I guess we should start with your current PR in iSEEu. Often the squash message needs a bit of cleaning up (and you can actually get rid of lot of the individual commit messages, they’re often unnecessary. e.g., “white line”
Kevin Rue-Albrecht (14:06:03) (in thread): > it’s one of those that I’ve dropped from the matrix anyway, but I’m curious
Kevin Rue-Albrecht (14:06:10) (in thread): > it’s like it’s not findingsce.rds
Kevin Rue-Albrecht (14:06:36) (in thread): > oh nevermind, indeedbasic.Rmdfailed
Kevin Rue-Albrecht (14:07:23): > yup i know exactly what you mean
Kevin Rue-Albrecht (14:07:43): > i’m doing just that (cleaning squash messages), just irregularly
Kevin Rue-Albrecht (14:08:22): > it’s good to have feedback and hear that someone cares, otherwise it just doesn’t stick as a habit
Aaron Lun (14:09:44): > I didn’t care before, but now that I’ve seen how clean the histories can be, I care now.
Aaron Lun (14:09:58): > Imagine if we squashed the 2.0 megacommit. It would just say “version 2.0”.
Kevin Rue-Albrecht (14:10:14): > is the second bullet still redundant? - File (PNG): image.png
Aaron Lun (14:11:02): > I’ll leave it to you. Just imagine what you would write if you actually did the entire commit all at once.
Aaron Lun (14:11:58): > It’s unfortunate that you have a pile of unrelated things in there, but oh well.
Kevin Rue-Albrecht (14:14:20): > Recently I also rediscovered that opening issues for each individual thing to fix allowed me to just writeFixes #...in each commit message and also focused my mind to actually work on each item separately. Thoughts?
Aaron Lun (14:15:10): > I would like commit messages to actually say what they did. There should be a distinction between git and github.
Kevin Rue-Albrecht (14:17:00): > Right… because outside of github, the issue number doesn’t link anymore to the story of what actually happened, gotcha
Aaron Lun (14:19:41): > Another nice thing about squashing is that it gets rid of messy merge commits within the PR, e.g., if a branch gets out of date. Then the squash makes it look like we were always running off the latestmasterall along.
Kevin Rue-Albrecht (14:20:10): > i see, good point
Aaron Lun (14:20:22): > INDEED. But it only makes sense if the PRs refer to a single feature or bugfix, so that’s something to keep in mind.
Kevin Rue-Albrecht (14:22:16): > speaking of side things to fix but haven’t their own PR or issue yet: there are a lot ofparam_choicesin there that could be turned intox, and i’ve also spotted references to ‘DataFrames’ of panels parameters
Kevin Rue-Albrecht (14:24:34): > anyway, i’ve got to pause here for now
Kevin Rue-Albrecht (17:43:20): > can anyone remind when/why that can happen again? > > ERROR: dependency 'iSEE' is not available for package 'iSEEu' > > https://github.com/iSEE/iSEEu/pull/41/checks?check_run_id=801060632
Kevin Rue-Albrecht (17:47:48): > Nevermind, iSEE didn’t install successfully because or an upstream package installation issue, seems to be igraph > > 2020-06-23T21:35:20.1456796Z Bioconductor version 3.12 (BiocManager 1.30.10), ?BiocManager::install for help > 2020-06-23T21:35:20.1458541Z Error in dyn.load(file, DLLpath = DLLpath, ...) : > 2020-06-23T21:35:20.1459378Z unable to load shared object '/usr/local/lib/R/site-library/igraph/libs/igraph.so': > 2020-06-23T21:35:20.1459540Z libglpk.so.40: cannot open shared object file: No such file or directory > 2020-06-23T21:35:20.1459901Z Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load > 2020-06-23T21:35:20.1460044Z Execution halted > 2020-06-23T21:35:20.1468922Z ERROR: lazy loading failed for package 'scran' >
Aaron Lun (17:59:41): > ¯*(ツ)*/¯
2020-06-25
Aaron Lun (01:19:26): > looking at the igraph fail. Why is it installing BINARY packages on a linux image?
Peter Hickey (01:57:55) (in thread): > Could be part ofhttps://community-bioc.slack.com/archives/CJDMYKG2U/p1592525194184700?thread_ts=1591015651.130600&cid=CJDMYKG2U? - Attachment: Attachment > thanks for the additional details, @Sean Davis. > I think I’ve now got this sorted. > It seems libglpk-dev is required for a ‘complete’(?) igraph installation, which is required for a clean installation of scran on the Docker image and with the GitHub Action builder. > > - To ensure libglpk-dev is installed in the Docker image, I needed to do this https://github.com/PeteHaitch/BioC2020_DelayedArray_workshop/commit/98558a6e9ed1d3d69907d6addba1f23fb6073a53 as suggested by @Charlotte Soneson and @Martin Morgan > - To ensure libglpk-dev is installed on the GitHub Actions builder, I needed to add igraph as an explicit dependency in my DESCRIPTION, otherwise it’s not picked up by sysreqs::sysreqs(). I documented this https://github.com/PeteHaitch/BioC2020_DelayedArray_workshop/issues/1
Kevin Rue-Albrecht (04:18:20): > Not sure that’s the best channel to get an answer though
Kevin Rue-Albrecht (05:12:14) (in thread): > Yup thanks, I got is sorted thanks to Charlotte and didn’t mention it, sorry!
Kevin Rue-Albrecht (05:15:28) (in thread): > https://github.com/iSEE/iSEEu/blob/master/.github/workflows/build_check_deploy.yaml#L68
Aaron Lun (19:19:32): > I think you can be a bit more relaxed with what you consider to be the “same feature”.
Aaron Lun (19:19:37): > When squashing PRs.
Aaron Lun (19:20:17): > Like, I would have squashed the last two commits in iSEEu under the general banner of “updated GHA.”
2020-06-26
Federico Marini (05:37:57): > FYI:https://academic.oup.com/nar/article/doi/10.1093/nar/gkaa412/5843818
Federico Marini (05:38:20): > I am never a big fan of the table of comparison to other software
Federico Marini (05:41:42): > but table 1 in there is…
Federico Marini (05:41:57): > well, I dislike more the new github interface
Federico Marini (05:52:04): > plus, well, we damn do have a docker image
Federico Marini (05:52:06): > oh well
Kevin Rue-Albrecht (07:03:46): > i don’t even want to go there, but> > * we don’t have interactive cell filtering? > * no sharing system? > * publish project? > * local/server hosting? > * no precomputed results? > * and then, importing data.. it’s R.. script a little
Federico Marini (07:23:24): > yep, I pulled the handbrake already
2020-06-27
Kevin Rue-Albrecht (09:14:28): > For workshop 2020: > > docker pull docker.pkg.github.com/isee/iseeworkshop2020/iseeworkshop2020 > docker run -e PASSWORD=abc -p 8787:8787 docker.pkg.github.com/isee/iseeworkshop2020/iseeworkshop2020 > > I just tested and confirmed this github docker package. > Publishing the workshop to the github package registry now takes 7m25s > Note that the automated build for iSEEu and iSEEWorlshop2020 on DockerHub are broken because DockerHub cannot pull (even public) images from the GitHub package registry, at least not wihtout authentication:https://github.community/t/docker-pull-from-public-github-package-registry-fail-with-no-basic-auth-credentials-error/16358/64 - Attachment (GitHub Support Community): Docker pull from public GitHub Package Registry fail with “no basic auth credentials” error > Running into this issue too. Please allow docker pull without authentication.
Kevin Rue-Albrecht (09:15:48): > bottom line is that the go-to place is docker package registry now
Kevin Rue-Albrecht (09:26:26): > Please give it a go and confirm that it doesn’t work just for me:slightly_smiling_face:
2020-07-02
Aaron Lun (14:07:52): > Need to level up this book.
Kevin Rue-Albrecht (17:25:13): > I know. I put it on hold during the last round of iSEE/iSEEu updates and waited too long to see the dust settle. Then I just didn’t get myself to get back in it
Aaron Lun (17:30:55): > I’m going to start working on it after dinner.
Kevin Rue-Albrecht (17:32:12): > good news is that the book is currently lightweight enough to build on GHA
Kevin Rue-Albrecht (17:33:17): > and even picturing a couple more large datasets, I don’t see any reason to get it into a multi-hour business:grimacing:
2020-07-03
Aaron Lun (19:59:55): > SEe how I handled the last two PRs for how squash-and-merging is meant to be done.
2020-07-04
Kevin Rue-Albrecht (17:15:31): > Really nice job on the book update
2020-07-05
Aaron Lun (07:57:02): > FYI bookdown::gitbook’s TOC is borked when there are multiple sections with the same name across chapters.
Kevin Rue-Albrecht (07:58:59): > Sounds like you’ll sleep better after opening an issue on the bookdown repo
Aaron Lun (08:01:28): > Or you could do me a favor and do it for me. I’m going to try to go to sleep again.
Kevin Rue-Albrecht (13:52:26): > quick note that I pushed 2.1.8 upstream
Aaron Lun (14:11:25): > I’m done with the book.
Kevin Rue-Albrecht (14:12:08): > still up?!
Kevin Rue-Albrecht (14:12:52): > ah no wait, i lost track of the time difference - already up?:stuck_out_tongue:
Aaron Lun (14:15:52): > damn straight.
Aaron Lun (15:06:39): > we do need to solve this TOC link problem, though.
Kevin Rue-Albrecht (16:27:45): > yeah - i had other things on my todo list today
Kevin Rue-Albrecht (16:27:58): > what’s your flow to update the screenshots for the book?
Kevin Rue-Albrecht (16:28:28): > this repo only hasscreenshotsinstead ofvignettes/screenshots
Kevin Rue-Albrecht (16:29:13): > damn i get rusty quickly with this screenshoting as i don’t use it frequently
Kevin Rue-Albrecht (16:32:22): > I think I got it.Rscript .../compile.R *Rmd
Aaron Lun (17:16:53): > Oh good:https://github.com/rstudio/bookdown/issues/909
Aaron Lun (17:16:55): > someone beat me to it.
Kevin Rue-Albrecht (17:17:25): > https://github.com/travis-ci/travis-build/pull/1920:cry:i feel a bit ignored there. All I’m asking is a click:grimacing:
Aaron Lun (17:18:25): > well, let’s see how your squashing skills are for 444.
Kevin Rue-Albrecht (17:19:10): > I’ll have to leave it at that for tonight. I’ve just finished a stroll through undocumented slot names while I was at it …:sweat_smile:
Kevin Rue-Albrecht (17:19:45): > Tomorrow with fresh eyes I’ll go through the methods and we’ll be good to go
2020-07-06
Kevin Rue-Albrecht (12:17:57): > Aside from the squashing. Did you have a look at the PR.? I think I’ve covered all your points, otherwise let me know if this PR needs anything else
Aaron Lun (12:18:58): > looks good to me
Aaron Lun (14:00:07): > I can also see that you haven’t read the author list of the book.
Kevin Rue-Albrecht (17:20:29): > oh i did
Kevin Rue-Albrecht (17:20:47): > i just didn’t mind it like that for the time being
Aaron Lun (17:21:10): > I forgot what it means.
Kevin Rue-Albrecht (17:22:34): > i can’t believe my eyes
Aaron Lun (17:22:41): > oh right
Kevin Rue-Albrecht (17:23:07): > tbh, the only thing i wondered is where you got that from
Aaron Lun (17:24:11): > I must have looked up some french phrases.
Aaron Lun (17:24:30): > You should just write that entire section in French.
Aaron Lun (17:24:38): > You can also write my section in French.
Kevin Rue-Albrecht (17:26:32): > 444 merged
Kevin Rue-Albrecht (17:27:01): > It’s just too painful writing French with a UK keyboard
Aaron Lun (17:27:12): > I guess you need to parlez vous anglais.
Aaron Lun (17:36:51): > looks like you’re avoir la patate.
Kevin Rue-Albrecht (17:37:46): > Hahaha love the potato expression
Kevin Rue-Albrecht (17:37:59): > Oui oui J’ai la patate
Aaron Lun (20:08:08): > BTW we should get rid of the travis files.
Aaron Lun (20:34:35): > You missed a spot. > > * checking for missing documentation entries ... WARNING > Undocumented S4 methods: > generic '.exportOutput' and siglist 'ComplexHeatmapPlot' > All user-level objects in a package (including S4 classes and methods) >
2020-07-07
Kevin Rue-Albrecht (07:09:36): > Fixed - and while we’re on the subject, please add entries to the NEWS as well
Kevin Rue-Albrecht (07:11:04): > I’ve done it again this time. It does get easier with the squashing method, but still, it’s more natural that the person doing stuff is also the one reporting it in the NEWS
Aaron Lun (11:15:16): > IMO the NEWS has wAAAAY too much noise if we update it for every minor version bump.
Aaron Lun (11:21:52): > In any case, we should probably cherrypick5864c04ed3cf42fce0b992bf986bd48ad75cb1a3onwards to RELEAE.
2020-07-09
Aaron Lun (12:09:46): > iSEEu needs to be pushed upstream
Aaron Lun (12:10:04): > So will iSEE, waiting for 455.
Kevin Rue-Albrecht (12:19:04): > ok
Kevin Rue-Albrecht (12:19:36): > so it’s a go for iSEEu?
Aaron Lun (12:19:47): > yep
Aaron Lun (12:20:11): > FYI our actions should probably use our containers so that we don’t have to install everything again.
Kevin Rue-Albrecht (12:21:18): > sure - in a first instance I only did that for the workshop container, but it seems to work fine so we can apply that for actions too
Kevin Rue-Albrecht (12:21:25): > iSEEu done (pushed upstream)
Aaron Lun (12:42:03): > go for iSEE
Kevin Rue-Albrecht (12:45:28): > Done
2020-07-16
Aaron Lun (15:58:49): > LogFC-logFC plot prototype is done. Maybe need a better way of reporting numbers, but nothing comes to mind. Also could do with some nicer colors.
2020-07-17
Aaron Lun (04:46:34): > iSEEu needs a push
Kevin Rue-Albrecht (04:49:05): > done
2020-07-18
Aaron Lun (03:46:46): > why doesiSEEuhave a covr Suggests?
Kevin Rue-Albrecht (04:44:34): > it’s for the GHAhttps://github.com/iSEE/iSEEu/blob/master/.github/workflows/build_check_deploy.yaml#L83We could removecovrfrom Suggests, as it’s installed as part of the GHA.
Kevin Rue-Albrecht (04:48:55): > Though I think that once upon a time I saw the recommendation to add it to Suggest. > The closest I can see now is the bioc guidelines that recommend puttingtestthatin Suggestshttp://bioconductor.org/developers/how-to/unitTesting-guidelines/I might have just extended that to apply tocovr
2020-07-19
Aaron Lun (18:00:54): > the updated screenshots don’t show anything for the hex plot.
Kevin Rue-Albrecht (18:12:16): > fixed (I was silently fixing it hoping no one would notice …)
Kevin Rue-Albrecht (18:15:07): > screenshots do not fail when a package isn’t installed - in this casehexbin- and I didn’t open the screenshots before merging:sweat_smile:
Kevin Rue-Albrecht (18:16:16): > I’m tempted to add a section/paragraph about voice control with a link to the youtube video
Kevin Rue-Albrecht (18:17:18): > i wouldn’t risk playing it during the hour of workshop, but readers could always check it out at their own leisure
Kevin Rue-Albrecht (18:35:23): > Do me a favor and merge this one when it passes GHA in ~30min, I’m going to sleephttps://github.com/iSEE/iSEEWorkshop2020/pull/37
Aaron Lun (18:42:53): > k
2020-07-21
Aaron Lun (02:31:49): > iSEEu needs the usual.
2020-07-22
Aaron Lun (12:04:56): > did you forget or is Bioc missing a build?
Kevin Rue-Albrecht (12:06:27): > i put the thumb up when i did
Kevin Rue-Albrecht (12:06:39): > have you triedgit clonea temp copy from bioc?
Aaron Lun (12:06:49): > No, I’m just looking at the landing page.
Aaron Lun (12:06:58): > I guess we skipped a build yesterday, then.
Kevin Rue-Albrecht (12:07:04): > well that one usually lags by a couple of days, i thought
Kevin Rue-Albrecht (12:07:22): > i usually count one day to go through the build, and another one for the page to refresh
Aaron Lun (12:07:36): > You can tell from the build badge, which is the fastest to update.
Kevin Rue-Albrecht (12:08:53): > Yup, it’s up to date > > % git push upstream master > Warning: No xauth data; using fake authentication data for X11 forwarding. > X11 forwarding request failed on channel 0 > Everything up-to-date >
Aaron Lun (12:08:57): > k
Kevin Rue-Albrecht (12:09:25): > how d’you like the updated iSEEWorkshop2020Slides ? :)
Aaron Lun (12:09:33): > haven’t looked at them
Kevin Rue-Albrecht (12:10:46): > I’m wondering whether all that css (bookdown, pkgdown, xaringan) should ultimately make it into BiocStyle or another central place
Aaron Lun (12:11:52): > I switched back to a grey background ToC on your CSS, the small white-on-blue text was too jarring to read. Kept the colors elsewhere though.
Kevin Rue-Albrecht (12:13:57): > not on the isee book, right? i don’t see a difference > you mean in rebook?
Aaron Lun (12:15:13): > OSCA book, though it hasn’t recompiled yet.
Aaron Lun (12:15:33): > you can see the same ToC style athttps://ltla.github.io/SingleRBook/ - Attachment (ltla.github.io): Assigning cell types with SingleR > The SingleR book. Because sometimes, a vignette just isn’t enough.
Kevin Rue-Albrecht (12:15:51): > ah ok
Aaron Lun (12:16:19): > it’s just a bit gentler on the eyes
Kevin Rue-Albrecht (12:17:01): > yeah - i guess the “dark theme” white-on-blue works ok for 5 links on the bioc website, but gets too intense for a whole TOC
Aaron Lun (12:17:11): > TRM, you should clean out the other favicon-related gunk, we only need the ico.
Kevin Rue-Albrecht (12:17:30): > I wonder whether inverting to blue-on-white might be ok while keeping the bioc theme to it
Aaron Lun (12:18:17): > The background being grey is very pleasing for demarcating the ToC from the rest of the book.
Kevin Rue-Albrecht (12:19:19): > I agree. There’s a reason why it’s standard. It’s just so.. standard
Aaron Lun (12:19:49): > I do use a darker grey than the default, though.
Aaron Lun (12:20:12): > You could try the Bioc colors on top of that. But IMO it would be too much.
Kevin Rue-Albrecht (12:32:12): > let’s call this a work in progress, as it is, at some point i’ll play with the White/Sepia/Night menu, perhaps we can maintain a few different themes, to please everyone
Aaron Lun (12:47:15): > We should probably add some comments about only initializing required panels and trying to set up a minimum number of processes.
2020-07-24
Aaron Lun (01:32:37): > If you can add some words about theshinyapps.ioexperience, squash the PR.
Kevin Rue-Albrecht (09:43:49): > words added, waiting for GHA
Kevin Rue-Albrecht (09:44:02): > I’ll drop Travis as a requirement as well for iSEE
Aaron Lun (10:15:26): > well I was thinking of your actual experience, like what to watch out for with respect o memroy limits.
Kevin Rue-Albrecht (10:17:06): > you mean that I played around until it worked?:grimacing:
Aaron Lun (10:18:20): > well, yes. Just a short comment on the final settings.
Kevin Rue-Albrecht (10:19:55): > alright - i just finish a couple of things and I add that
Aaron Lun (10:20:22): > probably just need to link toshinyapps.ioand make it sound less like an advertisement.
Kevin Rue-Albrecht (13:30:07): > looking back at the shiny contest app, i’m gonna have to be harsh but honest: the free (and starter) plans forshinyapps.ioare just not viable for apps: i had to downsample the allen dataset to deploy it for the contest
Kevin Rue-Albrecht (13:31:54): > though the assay was a matrix at that point - I haven’t tried delay’ing things
Kevin Rue-Albrecht (13:54:27): > see if it’s better - i don’t have much to say really, I’ve only used it for the contest
Aaron Lun (14:08:29): > yeah, looks good.
Kevin Rue-Albrecht (14:10:11): > it does make me wonder whether (rather, how much) Delay’ing may help in this case
Aaron Lun (14:10:27): > I didn’t know whether you could upload files like that toshinyapps.io.
2020-07-28
Aaron Lun (01:00:48): > If 461 is green, give it a spin, squash it and push it up.
Kevin Rue-Albrecht (05:58:00): > Works. Nicely done
Shubham Gupta (13:03:41): > @Shubham Gupta has joined the channel
Nur M Shahir (13:07:58): > @Nur M Shahir has joined the channel
Tanya Grancharova (13:55:40): > @Tanya Grancharova has joined the channel
Sisira Nair (13:55:57): > @Sisira Nair has joined the channel
Russ Bainer (13:55:58): > @Russ Bainer has joined the channel
Sean Davis (13:57:59): > Literally laughing out loud at the extra features in iSEE!
Joselyn Chávez (13:58:04): > @Joselyn Chávez has joined the channel
Kevin Rue-Albrecht (13:59:11): > the voice recognition was a dare from Fede
Federico Marini (13:59:35): > @Sean Davis: try turning it on and saying “good boy”
Kevin Rue-Albrecht (13:59:57) (in thread): > HAHAHAH true:rolling_on_the_floor_laughing:
Russ Bainer (14:00:04): > Really fantastic presentation, everyone
Ting Sun (14:00:10): > @Ting Sun has joined the channel
Kevin Rue-Albrecht (14:00:25) (in thread): > biggest regret: we didn’t get “dracarys” to summon dragons
Federico Marini (14:00:31): > Somewhere in a stale branch we had an easter egg with Danaerys’s dragon coming up if you said “dracarys”
Sisira Nair (14:00:35): > Great workshop. Was fun too:slightly_smiling_face:
Ting Sun (14:00:49): > fantastic workflow and super interesting talk!
Federico Marini (14:00:55): > Thanks (& welcome) to all newcomers!
Federico Marini (14:01:12): > Keep the Qs coming - either here or on twitter:wink:
Federico Marini (14:01:33): > (and I really meant that with the stickers, I’m happy to send around some snail mail to you!)
Federico Marini (14:01:41) (in thread): > YET
Kevin Rue-Albrecht (14:04:11): > Oxford people: I got stocks - File (JPEG): File from iOS
Charlotte Soneson (14:07:19): > I have a bunch in Basel too:slightly_smiling_face:
Veronica Jimenez Jacinto (14:24:25): > @Veronica Jimenez Jacinto has joined the channel
Vandhana (15:31:52): > @Vandhana has joined the channel
Vandhana (15:33:52): > Can iSee work on a seurat object?
Kevin Rue-Albrecht (15:34:50): > Seurat objects can be converted to SingleCellExperiment:https://satijalab.org/seurat/v3.0/conversion_vignette.html - Attachment (satijalab.org): Satija Lab > Lab Webpage —
Kevin Rue-Albrecht (16:23:26) (in thread): > suggestions welcome!:stuck_out_tongue:
Kevin Rue-Albrecht (16:24:04) (in thread): > I thought about VR… but aside from the over-the-top aspects, I’m not super keen on lasso selections in 3D …
CristinaChe (16:31:58): > @CristinaChe has joined the channel
Aaron Lun (16:37:35) (in thread): > just need some beat saber on the cells as they fly by
Federico Marini (16:41:30): > omg I would love that
Federico Marini (16:41:40): > Doing QC with cells
Federico Marini (16:41:51): > splitting them out if they had too much MT content
Kevin Rue-Albrecht (16:42:58) (in thread): > you know that currently the spiders and flies die when you hover over them?
Kevin Rue-Albrecht (16:43:51) (in thread): > I may not have mentionned the Easter egg in the Easter egg
Paula Beati (17:52:15): > @Paula Beati has joined the channel
2020-07-29
Aaron Lun (00:50:34) (in thread): > I probably deleted that one.
Aaron Lun (01:22:12): > @Kevin Rue-Albrechtnote that iSEEu now requires admins to respect the status checks. I wasn’t paying attention and accidentally pushed straight to master; this should no longer be possible.
David Burton (13:01:09): > @David Burton has joined the channel
David Burton (13:03:44): > Hi, I really enjoyed the iSee workshop. You guys have me thinking about trying to integrate some network visualization in a custom panel instead of just doing my own thing in a new Shiny app. Are you aware of anyone else already doing network graphs in iSee?
Aaron Lun (13:04:05): > not to my knowledge
Aaron Lun (13:04:16): > maybe there was something about trees at some point?
David Burton (13:05:56): > I think there are a couple ggplot options for it, but I think it may take some work to get a network into a summarized experiment object.
Aaron Lun (13:06:24): > SingleCellExperiment has a rowPairs thing, might be worth checking out
Aaron Lun (13:06:33): > easy to convert the rowPairs to/from a graph.
Kevin Rue-Albrecht (13:13:28): > For those who haven’t spotted it on Twitter, I’ve just posted replies to all our Q&A’shttps://twitter.com/KevinRUE67/status/1288516611805151234?s=20 - Attachment (twitter): Attachment > #bioc2020 Here is a thread to answer the many great questions that we didn’t get to answer during the #isee @Bioconductor workshop Q&A yesterday. > Format is 1 tweet for the question (Q), followed by 1 tweet for the answer (A). > @CSoneson @FedeBioinfo
Federico Marini (13:14:55): > Trees were into the the treeSE class if I recall correctly.@Charlotte Sonesonmight know more on that, it is a creature that stemmed from@Ruizhu HUANGand her AFAIK
Charlotte Soneson (13:17:23): > Yes,@Ruizhu HUANGstarted thinking about a panel for tree visualization (inhttps://www.bioconductor.org/packages/TreeSummarizedExperiment/objects) - Attachment (Bioconductor): TreeSummarizedExperiment > TreeSummarizedExperiment has extended SingleCellExperiment to include hierarchical information on the rows or columns of the rectangular data.
Ruizhu HUANG (13:24:08): > Yeah, I was trying to have a tree panel. But I didn’t really manage to make it interactive because a custom panel (e.g., rowLink data and colLink data where the information about nodes of a tree is stored) can’t output information to the tree panel
Freeman Wang (13:25:31): > @Freeman Wang has joined the channel
Aaron Lun (13:30:11): > you might have more luck with the updated custom panel specification athttps://isee.github.io/iSEE-book/ - Attachment (isee.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
Ruizhu HUANG (13:51:36): > ah, cool! The new version allows subclass to transmit selection to other panels. Will try it when my tree visualization functions are ready! Thanks!
Kevin Rue-Albrecht (14:02:02) (in thread): > Exactly, users and downstream developers will have a lot more fun and flexibility creating new panels with the S4 subclassing system
Kevin Rue-Albrecht (14:53:46) (in thread): > Just to clarify one more thing as well: new panels don’t necessarily need to go in iSEEu. People can make their own package to define and maintain “extended iSEE panels” with complete freedom, e.g. my own work in progresshttps://github.com/kevinrue/iSEEClusterProfiler/tree/table
Kevin Rue-Albrecht (14:56:20) (in thread): > Arguably not the best example as it is in very early stages of design/development, but it’ll give you an idea of the development process
Aaron Lun (20:23:59): > We should have a per-panel button that launches a custom-made help tour for that panel.
Aaron Lun (20:25:05): > Probably could stick a little question mark after the panel title that people can click on.
Aaron Lun (20:25:35): > And add a generic that generates a bespoke DF for entry intorintrojs.
2020-07-30
Kevin Rue-Albrecht (03:12:27): > Hell yeah. Opened an issue yet?
Aaron Lun (10:54:59): > See the PR
Sidharth Sen (13:13:48): > @Sidharth Sen has joined the channel
Tanya Grancharova (20:24:52): > Thanks for developing such a great app! Question about saving sessions … I know that you can save an rds file of an iSEE session and re-load it later, but the people that are primarily using the app in my group don’t know R and can’t code. We have it running in a shiny server, and I am the only person that touches the code, but the users are primarily other people. Is there a way for them to easily reload a session? Maybe something like loading an rds file from the actual app interface?
Aaron Lun (20:25:43): > you betcha
Aaron Lun (20:26:12): > if you startiSEE()(no arguments) you should see that people can upload an RDS of an SE + the initial state.
Aaron Lun (20:27:05): > you can actually customize the landing page to do arbitrarily complicated things
Aaron Lun (20:27:25): > like pull an RDS from another server, for example, so that you just show people a selectize of available options
Aaron Lun (20:28:12): > the?createLandingPageexample (can’t remember the exact name of the function) has a nice example
2020-07-31
Aaron Lun (01:18:58): > @Kevin Rue-AlbrechtReducedDimension tour is operational.
Aaron Lun (01:19:07): > I can’t remember what formatting scheme we were using.
Kevin Rue-Albrecht (03:46:00): > haven’t looked at the PR yet - dashing out for groceries - formatting what, tours?
Aaron Lun (04:05:06): > yea
Kevin Rue-Albrecht (05:13:51): > probably worth a page on the GH wiki
Federico Marini (05:23:14): > Would the wiki be able to host gifs?
Federico Marini (05:23:25): > without burdening the master, of course
Kevin Rue-Albrecht (05:26:56): > wiki is an independent repo
bogdan tanasa (14:05:54): > @bogdan tanasa has joined the channel
Dr Awala Fortune O. (16:21:07): > @Dr Awala Fortune O. has joined the channel
2020-08-02
Aaron Lun (05:15:28): > It’s ready.
Kevin Rue-Albrecht (13:19:07): > > I can’t remember what formatting scheme we were using. > I would put the “Data parameters” and other inputs that are not panel specific in simple black italic
Kevin Rue-Albrecht (13:19:38): > and the panel names need to be colored in the same way as the panel header background
Kevin Rue-Albrecht (13:20:16): > don’t worry, I’m on it, between “home-work”
Kevin Rue-Albrecht (13:23:04): > damn i love generics > > sprintf( > "The <font color=\"%s\">Reduced dimension plot</font> panel shows reduced dimensions from a <code>SingleCellExperiment</code> object or one of its subclasses. Here, each point corresponds to a column (usually a cell) of the <code>SingleCellExperiment</code> object.", > .getPanelColor(x) > ) >
Aaron Lun (13:48:58): > Finish it off and squash the PR.
2020-08-03
Kevin Rue-Albrecht (09:51:40): > pushed upstream
2020-08-04
Aaron Lun (03:04:39): > check out the iSEEu tours as well.
Aaron Lun (03:05:29): > the PR will need a rebuild when iSEE 2.1.12 becomes available.
2020-08-05
Hans-Rudolf Hotz (03:22:49): > @Hans-Rudolf Hotz has joined the channel
Aaron Lun (14:20:11): > Did iSEEu get pushed?
Aaron Lun (15:41:07): > @Kevin Rue-Albrecht
Kevin Rue-Albrecht (15:52:54): > Woops. Will do in 10 min
Kevin Rue-Albrecht (16:08:32): > done
Aaron Lun (16:08:40): > excellent
Kevin Rue-Albrecht (16:09:07): > I went back to check the PR this morning but you’d already taken care of it. At which point I forgot to push:sweat_smile:
2020-08-06
Federico Marini (10:55:48): > iSEE MASKS?
Federico Marini (10:55:49): - File (PNG): image.png
2020-08-08
Aaron Lun (17:48:32): > wait why do we need to re-exportggrepel::geom_text_repel?
Aaron Lun (17:49:28): > Also, the text UI should have ahr()on top of it
Aaron Lun (17:50:25): > Also, I wish we could get some more space between the edit button and the “Font size” label, it’s a bit cramped.
Aaron Lun (17:52:14): > Also, if I type a random name and press “validate names”, the name disappears rather than being validated. It is only after I click “apply” and then I go back to validate the name that the # appears.
Aaron Lun (17:54:03): > Otherwise it looks good.
Aaron Lun (17:56:05): > Whiel you’re there, you might as well explicitly mention it in theDotPlottour, it’s going to be too hidden for people to stumble on it.
2020-08-09
Kevin Rue-Albrecht (12:04:05) (in thread): > At that point add a Bluetooth microphone to help with voice recognition while wearing the mask
Kevin Rue-Albrecht (16:35:16) (in thread): > so that we could call justgeom_text_repelbut perhaps it’s clearer and better to explicity refer to the namespace?
Kevin Rue-Albrecht (16:58:43) (in thread): > that doesn’t happen to me
Kevin Rue-Albrecht (16:58:56) (in thread): - File (PNG): image.png
Kevin Rue-Albrecht (16:59:32) (in thread): > random names get commented out when I click “validate names” before I click apply
Kevin Rue-Albrecht (16:59:43) (in thread): > try that: > > initial <- list( > ReducedDimensionPlot(VisualBoxOpen = TRUE, VisualChoices = c("Text"), CustomLabels = TRUE), > SampleAssayPlot(VisualBoxOpen = TRUE, VisualChoices = c("Text"), CustomLabels = TRUE), > ComplexHeatmapPlot(DataBoxOpen = TRUE) > ) > > app <- iSEE(sce, initial) >
Aaron Lun (17:02:28) (in thread): > yes, probably
Aaron Lun (17:15:41): > Missing aunitimport
Kevin Rue-Albrecht (17:17:39): > ok, i’m just adding the tour steps now
Kevin Rue-Albrecht (17:18:08) (in thread): > just to be clear: that would require an extra instruction to open the “Text” visual options, to reveal the button
Kevin Rue-Albrecht (17:19:02) (in thread): > I’ll push something and you’ll tell me, but if you have something specific in mind, this is the point where doing it yourself is the simplest and fastest way
Aaron Lun (17:22:03) (in thread): > I’m still getting this.
Aaron Lun (17:22:18) (in thread): > yes, that’s correct. I actually had this instruction before but I removed it.
Aaron Lun (17:22:34) (in thread): > trying to figure out what’s going on
Kevin Rue-Albrecht (17:24:17) (in thread): > I just pushed
Kevin Rue-Albrecht (17:24:20) (in thread): > knock yourself out
Kevin Rue-Albrecht (17:24:38): > unitfixed
Aaron Lun (17:29:02) (in thread): > I get it. The debounce time on the editor means that if you validate too soon after typing, the newly typed content hasn’t even gotten to theinputyet.
Kevin Rue-Albrecht (17:29:43): > Any special reason why.singleSelectionDimensionreturn feature/sample while.multiSelectionDimensionreturn row/column ?
Aaron Lun (17:30:15): > just to provide another point of difference when we’re talking about selections.
Aaron Lun (17:30:32): > pobably also historical.
Aaron Lun (17:30:57): > I think in general we talk about individual features/samples and multiple rows/columns.
Aaron Lun (17:31:40) (in thread): > Turning downdebounce=100is a lot better.
Kevin Rue-Albrecht (17:32:17): > I noticed that when - for UI dynamic text generation - I was after a generic that would tell me “what” data points represent in a dotplot panel or rows represent in a table
Kevin Rue-Albrecht (17:32:45): > which ultimately could be almost anything, including “geneset”
Kevin Rue-Albrecht (17:32:55): > not just features or samples
Aaron Lun (17:33:05): > we’ll deal with that when we get to it, I think.
Aaron Lun (17:33:23): > Probably needs a new generic. But that would first require exposing the modal code.
Kevin Rue-Albrecht (17:33:44): > yup
Kevin Rue-Albrecht (17:33:57): > my head hurts when I think of all the extensions that are possible now
Kevin Rue-Albrecht (17:36:54): > Anyway there are a few things to take care of first.
Kevin Rue-Albrecht (17:37:29): > e.g I’m not a big fan of theifpattern that I’ve duplicated for now > > # Adding a faceting command, if applicable. > facet_cmd <- .addFacets(x) > if (length(facet_cmd)) { > N <- length(plot_cmds) > plot_cmds[[N]] <- paste(plot_cmds[[N]], "+") > plot_cmds <- c(plot_cmds, facet_cmd) > } >
Aaron Lun (17:38:20): > yes, see my comments in the iSEEu issue.
Kevin Rue-Albrecht (17:39:27): > bugs me that “the good”iSEEuisn’t at the top of my Google search results yet - File (PNG): image.png
Kevin Rue-Albrecht (17:40:19) (in thread): > that one I assume?https://github.com/iSEE/iSEEu/issues/57
Aaron Lun (17:41:28) (in thread): > ypu
Kevin Rue-Albrecht (17:45:08): > Alright - I’ve just spotted and fixed your PR comment
Kevin Rue-Albrecht (17:45:49): > I’m not gonna stay up to watch the GHA, so if you’re happy with it, just merge it when it’s cooked
Aaron Lun (17:46:30): > wahat, it worked?
Kevin Rue-Albrecht (17:46:43): > there’s gotta be a benefit to working in such different time zones
Kevin Rue-Albrecht (17:46:47): > what worked?
Aaron Lun (17:47:16): > Moving the br inside didn’t work for me, but I guess you had to have two of them.
Kevin Rue-Albrecht (17:47:32): > yup
Kevin Rue-Albrecht (17:48:01): > I’m playing it smart, but I also got thrown off when I saw moving the single one didn’t work as expected
Aaron Lun (17:48:33): > Indeed
Aaron Lun (17:48:38): > did you turn down the debounce?
Aaron Lun (17:48:51): > no, I’ll do it.
Kevin Rue-Albrecht (17:49:00): > i didn’t, I thought it was something you did on your local session
Kevin Rue-Albrecht (17:49:21): > remind me, that’s the frequency at which shiny checks the UI inputs?
Aaron Lun (17:49:29): > aceEditor’s default is 1 second.
Kevin Rue-Albrecht (17:50:17): > are you that fast? I mean, I type, then I click on “validate”, but I don’t rush either
Aaron Lun (17:50:47): > damn straight I am
Kevin Rue-Albrecht (17:50:55): > oh yeah I see what you mean now
Kevin Rue-Albrecht (17:54:30): > damn.. 1s = too slow… next thing I know you’ll be writing a comparison of visualization tools for single-cell RNAseq data that complain the web page doesn’t loads fast enough …
Kevin Rue-Albrecht (17:55:00) (in thread): > oh wait …:kissing::musical_note:
Aaron Lun (17:58:50): > Can you fix the 2->4 spaces tomorrow?
2020-08-11
Keith Connolly (16:50:11): > @Keith Connolly has joined the channel
2020-08-13
bogdan tanasa (01:50:03): > Dear Kevin, i was wondering, and if I may ask please, if iSEE supports (yet) the 3D visualization of PCA or UMAP plots (i have missed your presentation at BioC2020). thanks a lot !
2020-08-14
Roye Rozov (04:45:04): > @Roye Rozov has joined the channel
Kevin Rue-Albrecht (07:43:27) (in thread): > Hi Bogdan > We did not implement any 3D plot panel yet, mostly because there is no way to draw selections in 3D, so those types of panels are not really at the top of our priority list. That said, our mechanism allows panels that do not allow selection (e.g. heatmaps), so I think that 3D panels are technically possible.
Kevin Rue-Albrecht (07:48:34) (in thread): > if you’re adventurous, you could use our S4 panel class system, to write a new class thatImports: iSEE
2020-08-15
Aaron Lun (04:09:53): > @Kevin Rue-Albrecht@Charlotte Soneson@Federico Marinicheck the tooltip PR, there’s enough CSS magic that I need to know if it works reliably across browsers.
Federico Marini (09:37:17): > RStudio: - File (PNG): image.png
Federico Marini (09:43:32): > chrome -> - File (PNG): image.png
Federico Marini (09:43:59): > Feuerfuchs:slightly_smiling_face:-> - File (PNG): image.png
Federico Marini (09:44:45): > good ol safari
Federico Marini (09:44:47): - File (PNG): image.png
Federico Marini (09:45:03): > sorry, have no opera
Aaron Lun (12:20:10): > oh yeah
Aaron Lun (12:20:28): > What about faceting and horizontal violins?
Federico Marini (15:21:03): > On it now
Federico Marini (15:21:52): > Meanwhile: can we put all the question marks for the panel tours all right aligned? I know, nitpicking - just looks a little tidier?
Federico Marini (15:26:32): - File (PNG): image.png
Federico Marini (15:27:31): > hinton’ing around - File (PNG): image.png
Federico Marini (15:29:33): - File (PNG): image.png
Federico Marini (15:37:03): > Looks all good on all browsers
Aaron Lun (16:44:30): > I tried so hard to right align them.
Aaron Lun (16:44:33): > And then I gave up.
Federico Marini (16:47:58): > :heart:
2020-08-16
Aaron Lun (04:51:10): > it’s on the right
Federico Marini (08:56:36): > :heart:
Federico Marini (08:56:53): > dammit. I wanted to right align the heart as well
Aaron Lun (14:19:31): > it’s really more left aligned
Aaron Lun (14:19:36): > unless you have dextrocardia
2020-08-18
Aaron Lun (11:23:01): > I dreamt that@Kevin Rue-Albrechtwas back.
Aaron Lun (11:23:16): > And squashed the PRs and pushed them up to BioC.
Federico Marini (12:02:56): > status says vacationing
Aaron Lun (12:03:41): > yes, I know.
Federico Marini (12:04:57): > gotta hope he hides somewhere to check slack while wife is looking for him:smile:
Kevin Rue-Albrecht (13:09:32): > I don’t get it - you don’t need me to squash the PR if Fede tested it
Aaron Lun (13:09:55): > Yeah, but I want you to see how cool it is.
Kevin Rue-Albrecht (13:10:19): > alright
Aaron Lun (13:10:53): > only you would appreciate its true coolness.
Kevin Rue-Albrecht (13:11:43): > flattery won’t… oh well it will:laughing:
Kevin Rue-Albrecht (13:13:24): > I’l update the PR branch which fell behind master
Kevin Rue-Albrecht (13:16:26): > nice.. i gave up on right-aligning the help icon
Kevin Rue-Albrecht (13:20:06): > crap - seems like I never pushed 2.1.13
Aaron Lun (13:20:39): > I’m pretty sure you did. It never made it to the landing page but 2.1.13 is present in the archives. I’ve noticed this as well.
Kevin Rue-Albrecht (13:22:14): > oh ok
Kevin Rue-Albrecht (13:22:36): > so.. that squash, shall we throw in a version bump as well?
Aaron Lun (13:23:16): > Probably a separate version bump and news update, given that the news has multiple new features.
Kevin Rue-Albrecht (13:24:10): > yes, separate as in 2.1.14
Kevin Rue-Albrecht (13:38:57): > check that you’re happy with the NEWS and I’ll merge/upstream when GHA passes
Aaron Lun (13:39:33): > looks good
Kevin Rue-Albrecht (13:40:33): > I was wondering whether the tooltip could use a darker border, but it’s good as it is I think
Kevin Rue-Albrecht (17:09:59): > git push upstream master:bioc-load::heavy_check_mark:
Aaron Lun (17:20:53): > good, good
Aaron Lun (17:24:04): > Now that we do squashes, the recent git history looks so awesome.
Aaron Lun (17:24:10): > It’s like, we just did it all in one go.
Peter Hickey (19:38:58): - File (PNG): image.png
2020-08-19
Aaron Lun (12:43:26): > iSEEu needs a version bump to trigger reinstallation, otherwise it’ll continue to use the old iSEE class definitions.
2020-08-20
Kevin Rue-Albrecht (18:08:22): > iSEEu kevin$ git push upstream master:arrow_right:1.1.6:white_check_mark:
2020-08-25
Kevin Rue-Albrecht (08:35:26): > iSEE % git push upstream master:arrow_right:2.1.15:white_check_mark:
Kevin Rue-Albrecht (08:37:43): > iSEEu % git push upstream master:arrow_right:1.1.7:white_check_mark:
2020-08-26
Saulius Lukauskas (07:48:10): > @Saulius Lukauskas has joined the channel
2020-08-29
Aaron Lun (02:50:50): > THIS LINK MUST BE SHAREDhttps://wanwan-html5.moe/#curl=https://github.com/iSEE/iSEE/raw/master/inst/www/iSEE.png
Federico Marini (08:49:20): > ahahahaahah
2020-09-13
saskia (23:48:21): > Is there a way to get labels of clusters onto the plot?
Aaron Lun (23:49:57): > sure, check out the LabelCenters option.
Aaron Lun (23:50:05): > I think that’s what it was called. Can’t remember.
Aaron Lun (23:50:13): > It’s in BioC-devel, though.
saskia (23:50:28): > thanks@Aaron Lun!
2020-09-18
Federico Marini (09:32:26): > Did we every try scATACseq data in iSEE?
Federico Marini (09:32:38): > and by we I mean you, cause I did not:smile:
Aaron Lun (11:04:29): > I didn’t try it, there’s no reason it wouldn’t work, but you’ll need a new panel to visualize the ranges.
Federico Marini (11:59:24): > I also thought that, yes
Federico Marini (11:59:43): > Question came from Maria Colome in a session where she was showing off epiScanpy
Federico Marini (12:00:04): > and I came right after her, so it was a natural consequence:slightly_smiling_face:
Aaron Lun (12:00:23): > I have no idea what that is, but I guess it’s some scanpy thing to do ATAC-seq.
Federico Marini (12:07:34): > ’Xactly
2020-09-22
Aaron Lun (17:27:41): > Get some testing on the latest PR. I’m not 100% sure whether this works or not, it depends on the sequence of rendering events that happen inside the browser. Ithink it should be fine but I can’t say for sure.
Aaron Lun (17:27:46): > https://github.com/iSEE/iSEE/pull/485
Aaron Lun (19:57:30): > maybe someone can also figure out why covr doesn’t like me.
2020-09-23
Aaron Lun (18:35:11): > HELLO
2020-09-24
Kevin Rue-Albrecht (05:10:07): > teaching resumed last week
Kevin Rue-Albrecht (05:10:35): > been rediscovering and updating my materials:sweat:
Kevin Rue-Albrecht (08:15:22): > I’ve tested a live app and pushed upstream.
Federico Marini (08:56:58): > whole week summer school for me, so yep, sorry for not picking it up@Aaron Lun
2020-09-25
saskia (03:58:35): > why can the cluster lables thing only deal with small numbers of clusters?
Federico Marini (05:41:04): > you mean for the color scales?
Kevin Rue-Albrecht (05:56:37): > no - i gotta check again how Aaron implemented it, but there’s a checkbox now to show the cluster labels
Kevin Rue-Albrecht (05:57:14): > my first guess would be that this feature is subject to the same limit as factors
Kevin Rue-Albrecht (05:58:59): > i.e.options(iSEE.maxlevels=30)
Kevin Rue-Albrecht (05:59:09): > @saskiaany chance you have more than 30 clusters?
Kevin Rue-Albrecht (07:20:30): > I strongly suspect that’s the issue, try settingoptions(iSEE.maxlevels)to a value that’s greater than the number of clusters you have
Federico Marini (08:14:47): > yeah, if that was the question, that’d be the answer - just not sure if I got@saskia’s point right
Anna Liza Kretzschmar (22:55:10): > @Anna Liza Kretzschmar has joined the channel
2020-09-27
saskia (18:26:16): > Sorry guys: yes that was the question
saskia (18:26:38): > But is it restricted because the lables become illegible?
2020-09-28
Aaron Lun (00:44:55): > not in this particular case, but we use the same factor-based machinery for handling different colors, and once you have over 10 colors it’s basically impossible.
Aaron Lun (00:45:23): > Probably we should divorce the consideration of factors for colors and those for everything else. But in the meantime just kick up theiSEE.maxlevels.
2020-10-01
Kevin Rue-Albrecht (05:48:11): > @Aaron LunI’ve seen the push. I’m just fixing a couple of warnings about undocumented methods. Then I’ll push upstream
Kevin Rue-Albrecht (07:41:54): > argh - it still need an entry in the NEWS file
Kevin Rue-Albrecht (09:23:26): > 2.1.18 pushed upstream
2020-10-05
Aaron Lun (22:16:41): > ECM should really inherit fromAnnotatedinstead ofVector. I don’t think it makes use of the vector concept.
2020-10-07
Aaron Lun (03:40:16): > @Kevin Rue-Albrechtchoose: > 1. Beautify the AggregatedDotPlot inhttps://github.com/iSEE/iSEEu/pull/732. Reorganize the Panel infrastructure to avoid distributing logic to subclasses. > > Pick one and I will do the other.
Kevin Rue-Albrecht (03:44:44): > both of them sound pretty subjective though 1 at least has a more concise visual output to discuss, i’m looking at the screenshot right now, sure it gives a sense that it could look “better”, though it’s pretty challenging to say how to handle many rows/columns in an aesthetically robust way
Aaron Lun (03:53:12): > probably can do some scaling of the point size with the known height, though I wouldn’t worry about that too much.
Federico Marini (03:53:14): > beautiful panel,
Aaron Lun (03:53:29): > More about setting the colors, adding centering options? etc.
Federico Marini (03:53:50): > exactly yesterday I was reading on CellXGene VIP, and they had something similar, also I guess even with violins instead of the plots
Federico Marini (03:54:11): > don’t know if this is even feasible, but if this would make sense, why not
Federico Marini (03:54:40): > I think they had an option there to cap the values to a particular value
Aaron Lun (03:54:45): > what, lots of little micro violins? Sounds difficult.
Federico Marini (03:54:56): > (is that what they call winsorizing? guess so)
Federico Marini (03:55:10): > lemme check, it was after some red wine:smile:
Kevin Rue-Albrecht (03:55:41): > Sure - i’ll have a look at the plot. Fair warning: next two weeks are pretty packed with teaching. I’ll spare a bit of time this week
Kevin Rue-Albrecht (03:57:43): > Also, careful what I wish for, but it may be good to start refactoring some of the (stable) iSEEu panels into separate mini-packages, to kick off the universe in the next:bioc-load:release
Federico Marini (03:58:08): - File (PNG): image.png
Federico Marini (03:58:14): > toying around with their instance
Federico Marini (03:59:00): - File (PNG): image.png
Federico Marini (03:59:12): > and this is from their repo, in the readme content
Aaron Lun (04:00:32): > very not sure about the violins, it’s like the worst of both worlds
Federico Marini (04:07:25): > I know, it is even hard to read, honestly
Federico Marini (04:07:57): > ah. about beautifying/adding: would the side decoration with barplots make sense/be feasible?
Aaron Lun (04:08:21): > if you can be bothered, potentially.
Federico Marini (04:08:45): > would a dendrogram on top do any good?
Federico Marini (04:08:52): > on top now meaning on the side
Federico Marini (04:09:48): > Ha. I saw a post some time ago from Dave McG***********y, let me see if I find it again
Federico Marini (04:12:12): > https://davemcg.github.io/post/lets-plot-scrna-dotplots/ - Attachment (eye Bioinformatician): Let’s Plot 7: Clustered Dot Plots in the ggverse | eye Bioinformatician > 2020 03 23 Update Intro Example dotplot How do I make a dotplot? But let’s do this ourself! Dotplot! Zero effort Remove dots where there is zero (or near zero expression) Better color, better theme, rotate x axis labels Tweak color scaling Now what? Hey look: ggtree Let’s glue them together with cowplot How do we do better? Two more tweak options if you are having trouble: One more adjust Moonshot Downside Exercises for the reader OLD Solution (kept for posterity) 2020 03 23 Update Ming Tang pointed out a better way to align plots, so I have rewritten the back end of this post.
Aaron Lun (04:15:33): > dunnno what the stars mean.
Aaron Lun (04:16:04): > clustering of rows and columns would be another beautification option
Federico Marini (04:16:22): > just the wildcard key- I can almost never spell correctly McGaughey from the first attempt
Aaron Lun (04:17:38): > anyway, I’ll leave it up to@Kevin Rue-Albrecht, you guys sort it out.
Aaron Lun (04:17:44): > I need to put out some more fires elsewhere.
Federico Marini (04:23:01): > Happy to follow up, bandwidth is more limited than in other occasions, but discussing with you is a big cheer up
Kevin Rue-Albrecht (04:49:04): > honestly - there are a lot of confounders - but motivation was at an all-time high when this channel was at peak activity:slightly_smiling_face:
Federico Marini (06:44:21): > :stuck_out_tongue:I still call that my golden age of extra boosts:smile:
2020-10-08
Federico Marini (07:47:01): > not an expert on the limits onshinyapps.iobut it does look like that
Federico Marini (07:47:32): > I think he is currently thinking the limit is only on the objects, but it can be well possible that the plotting routines demand for more
Federico Marini (07:48:14): > Thanks@Kevin Blighefor bringing this up! Meanwhile, I’ll try to look up if I ever did register to biostars:smile:
Kevin Rue-Albrecht (09:09:48): > Yep thanks - it’s been a while since I handled questions - Aaron generally beat me to it - so I probably ranted a bit more than necessary
2020-10-09
Aaron Lun (01:45:06): > We should really build our check actions from a dockerfile, waiting for the dependencies to install is such a chore.
Federico Marini (04:52:17): > … and while we are at it, also remember we’ll have to change the instructions for the branches there as well
Federico Marini (04:52:40): > probably at best after end of the release round?
Aaron Lun (11:13:55): > Might as well do it now.
Maria Colome-Tatche (16:53:30): > @Maria Colome-Tatche has joined the channel
Aaron Lun (18:00:24): > @Kevin Rue-Albrechtdotplotis ready for your attention. Biggest remaining code thing is how to handle the colors consistently with the ECM. Then docs and tests.
2020-10-10
Kevin Rue-Albrecht (05:18:43): > ok - fyi i’m not reviewing all the changes that you’re merging to masters these days, so just ping me when you think it’s ready to push upstream, i’m looking out for version bumps, but it’s still better/safer if i hear the green light directly from whoever merges new code
Aaron Lun (14:37:23): > you could set up an auto-push to BioC onmaster, see LTLA/OSCA for an example
Aaron Lun (19:55:00): > I just did the docs. So just colors and tests.
2020-10-11
Aaron Lun (14:40:40): > Are you going to do some colors, or should I do this?
Kevin Rue-Albrecht (14:41:44): > if you have time, be my guest, i’m still playing catch up
Aaron Lun (14:42:15): > i mean, if you’re going to look at it tonight, I’ll do something else
Kevin Rue-Albrecht (16:11:15): > Installing current master of iSEE and iSEEu right now. Let’s see what I can do over the next hour or so
Kevin Rue-Albrecht (16:31:47): > i’m gonna focus on tests first - makes me think again that splitting iSEEu into sub-packages would also make it faster/clearer to test individual panels
Kevin Rue-Albrecht (17:02:27): > I started pushing todotplot-kra, not much to see yet, just saying in case you’re looking atdotplot
Aaron Lun (17:18:51): > BTW I switched iSEE to do its GHA’s off the docker image, cuts out 20 minutes from the build time.
Aaron Lun (17:19:02): > I also factored out the Docker build, so that’s another 20 minutes off.
Kevin Rue-Albrecht (17:35:42): > line 317 of AggrDotPlot: i assume it should be discretecolDatafrom context > > warning(sprintf("no discrete 'assays' for plotting '%s'", class(x)[1])) >
Aaron Lun (17:37:42): > probably
Kevin Rue-Albrecht (18:33:49): > the readme of iSEEu seems to need some cleanup as well
Kevin Rue-Albrecht (18:34:15): > especiallyhttps://github.com/iSEE/iSEEu#where-do-i-look-for-constants-within-iseeis doing the opposite as what we last agreed, I believe
Aaron Lun (18:38:11): > we should probably make it more consistent. Maybe export them all?
Aaron Lun (18:38:21): > But the names are pretty ad hoc.
Aaron Lun (19:45:27): > tell you what, iSEE looks so surreal in dark mode
Aaron Lun (20:07:53): > dotplot colors are done.
2020-10-12
Aaron Lun (01:45:40): > I’ve added a pile of tests. If you have time tonight, see if you can ram it intomaster.
Aaron Lun (01:59:09): > damn, end-to-end checks in under 15 minutes. Lightning fast compared to what we had before.
Aaron Lun (03:25:27): > I can’t see the code cov checks, so you’ll have to deal with those on your own.
Kevin Rue-Albrecht (04:22:15): > I’ll admit that it takes a bit of patience to navigate the codecov website until you get to the repo->branch->files, but that’s what you’re talking about:https://codecov.io/gh/iSEE/iSEEu/tree/dotplot/R - Attachment (Codecov): Code coverage done right. > Hosted coverage report highly integrated with GitHub, Bitbucket and GitLab. Awesome pull request comments to enhance your QA.
Kevin Rue-Albrecht (04:22:31): > I’ll have a bit of fun now
Kevin Rue-Albrecht (05:27:58): > Wondering about the best place to store helper functions used in the “setup” scripts for testthat, e.g.mimic_live_appwhich I want to use for iSEEu as well as iSEE
Kevin Rue-Albrecht (05:29:07): > iSEE.utilspackage?
Aaron Lun (11:26:00): > shouldn’t the codecov output pop up on the PR?
Kevin Rue-Albrecht (11:28:21): > right, i see what you mean, hang on
Kevin Rue-Albrecht (11:29:54): > it’s still enabled as PR checks in the settings
Kevin Rue-Albrecht (11:29:57): > weird
Aaron Lun (11:30:28): > I also realized we need a “cluster rows” option
Aaron Lun (11:30:39): > that would be the final piece. Just a bit a hierarchical clustering I think.
Kevin Rue-Albrecht (11:33:24): > and codecov integration has access to both iSEE and iSEEu
Kevin Rue-Albrecht (11:40:18): > I found why. Sit tight
Kevin Rue-Albrecht (11:46:08): > incoming ondotplot, don’t forget to pull if you work on that one
Aaron Lun (11:47:42): > is it adotplotfix, or should you do it tomaster?
Kevin Rue-Albrecht (11:48:11): > well,dotplotwill merge tomaster… right?
Aaron Lun (11:48:34): > right, but you don’t want the commits to be a jumble of different features
Aaron Lun (11:48:45): > because remember I’m going to squash everything into a single commit
Kevin Rue-Albrecht (11:50:16): > https://github.com/iSEE/iSEEu/pull/78
Kevin Rue-Albrecht (11:52:11): > anddotplotcommit is undone
Kevin Rue-Albrecht (11:53:11): > are you working straight onmaster?
Aaron Lun (11:53:39): > no
Aaron Lun (11:54:01): > i’m just saying that different features should be in different PRs
Kevin Rue-Albrecht (11:54:43): > yup i got that, but reading the network graph got me confused for a minute, as mycodecovbranch is somehow out of date with the base branch
Kevin Rue-Albrecht (11:54:45): > anyway
Kevin Rue-Albrecht (11:55:26): > alright - need a break from screen
Aaron Lun (11:55:34): > k
Aaron Lun (11:55:39): > when you get back it’s just one last spurt
Aaron Lun (11:55:47): > slap some clustering in and we’re done
2020-10-13
Federico Marini (10:17:27): > :heart:https://libd.shinyapps.io/tran2020_Amyg/
Kevin Rue-Albrecht (11:00:57) (in thread): > lovely
Kevin Rue-Albrecht (11:01:31) (in thread): > You can also try iSEEu branchdotplot-clusterand run the man page forAggregatedDotPlot
Kevin Rue-Albrecht (11:01:41) (in thread): > looks even better with > > AggregatedDotPlot(ColumnDataLabel="Primary.Type", PanelWidth = 6L, CustomRowsText = sample(rownames(sce), 6), ClusterRows = TRUE) >
2020-10-14
Aaron Lun (03:57:18): > The dimnames modal probably should disable the import button if the upstream panel doesn’t actually have any selection made.
Kevin Rue-Albrecht (04:28:12): > make sense - we should throw those things into issues, to track them better, i’ve got a full day today, probably won’t get to it until tomorrow
Kevin Rue-Albrecht (18:01:32): > iSEEu 1.1.8 pushed upstream
Aaron Lun (22:56:23): > I’ve got this and another issue to post.
2020-10-15
Aaron Lun (00:44:57): > Filed. See if you can tackle them soonish while I bang my head on other parts of the stack.
2020-10-17
Aaron Lun (04:10:12): > iSEEu is failing.
Charlotte Soneson (14:17:43): > ****iSEE 2.1.25******** seems to have fixed it (I also can’t reproduce the error locally with the latest ********iSEE******** version)****
Aaron Lun (14:24:49): > yeah, probably just a sycn
Aaron Lun (14:38:56): > working on the two issues now.
Aaron Lun (14:50:02): > PRs made. Can someone test them out and merge. I’m going off for lunch.
Aaron Lun (20:11:24): > Push to BioC needed.
2020-10-18
Kevin Rue-Albrecht (15:48:01): > done
2020-10-19
Avi Srivastava (14:30:23): > Quick thought, what if we process all the 10x dataset which they have made public, quantify it through alevin (spatial, protein, RNA — can help with ATAC/chromatin) and upload it to a central place along with bunch of iSEE plots ? Basically I am thinking exactly what 10x has on their website however processed through alevin, visualized through iSEE and potentially we can annotate the cell types for major dataset like PBMC. It’ll help people to just pull the SingleCellExperiment for these object (or Seurat objects:stuck_out_tongue_winking_eye:) with the necessary information. I am happy to help on the quantification side if someone is ready to help on the visualization.
Avi Srivastava (14:32:55): > Thinking of probably we’d need a website also to upload all these things.
Charlotte Soneson (14:38:07): > Data sets (SCEs) could probably go in an ExperimentHub package, would make them easily accessible and also include documentation.@Federico Marinimay have something to say about the iSEE instances:slightly_smiling_face:
Federico Marini (14:50:47): > @Avi Srivastava, I can’t express how much I would appreciate such a thing
Federico Marini (14:51:41): > The idea would be to haveiSEEpower something like a SingleCellPortal, with less breadth in terms of datasets but more depth in capabilities
Federico Marini (14:52:31): > The website for this would entail some variation on the functionality automatically provided by thecreateLandingPagefunction
Federico Marini (14:53:35): > Where some kind of catalogue is there, and ideally even the possibility to upload the own sce object
Federico Marini (14:55:12): > hardware & software-wise: We ordered some powerful servers for general crunching, but one nice machine will be kinda mine to play with for these purposes:smile:Plus, we got a license for RStudio Connect to handle all the fun of shiny in a more pro way
Federico Marini (14:57:43): > My dream scenario would be that we snap the .data domain once that becomes available, and have iSEE.data
Avi Srivastava (14:59:59): > Lovely ! Thanks@Federico Marini@Charlotte Sonesonfor the fast replies. > A couple of thoughts: > 1.) Scalability in terms of dataset should not be a big problem, processing wise, as almost all the dataset on EMBL has been processed by alevin and have mtx on their website. Probably we can start with 10x dataset, which are probably 20-30 ish and scale when stable. > 2.) We might have to take some early decisions before hand for example what reference to use etc. > 3.) I think ExperimentHub would be great but I’d prefer somewhere where we have clickable link for the SCE (seurat) objects for the download, not sure if it’s already possible through Ehub. > > I am excited to know you have powerful machines to process the data and indeediSEE.datawould be awesome to have. I am new to R world but happy to help in any meaningful way as possible (apart from processing the raw data from alevin).
Federico Marini (15:01:22): > Awesome, nice to see this “topic” recognized by others and not somehow only lingering in my mind:smile:
Federico Marini (15:02:29): > Adding on top of this, I just had an idea of what to do with the metadata
Federico Marini (15:03:51): > I already threw in the thought some time ago to have something like a list-like class added in an element of the metadata slot, and have one function in iSEE that picks that, and displays that aas a modal. > Digging deeper, seemed we can just use any MIAME-like oriented class.
Federico Marini (15:04:37): > Pluuuuuus: > we could place an url in the metadata slot and that would link to the vignette that goes from raw-like data to the sce object we are providing
Aaron Lun (15:05:01): > I use the landingPage function to pull stuff from S3 for visualization.
Avi Srivastava (15:06:38): > I think it’s going to be helpful to many people, even people outside bioC. just to motivate it further SatijaLab also released a shiny app which does something similar although with totally different motivation. I think there is space for multiple motivations and use case and I look forward to contributing in it.http://azimuth.satijalab.org/app/azimuth
Federico Marini (15:07:02): > yep, I have seen that!
Federico Marini (15:10:53): > Aaron, did you already give a spin to some fully pimped landing page? Not that one needs it if the functionality one needs is just to load something and you’re good to go
Federico Marini (15:11:09): > but more for this case of showcasing that Avi also mentioned
Aaron Lun (15:11:56): > I have created some very complex landing pages, some which have a level of complexity equivalent to Shiny apps in their own right. However, these are all internal so you won’t be seeing them anytime soon.
Aaron Lun (15:14:49): > But it’s not particularly difficult. In this case, just populate a DataTable with all available datasets, and create an observer that triggers iSEE initialization when a particular dataset is chosen.
Federico Marini (15:16:10): > > you won’t be seeing them anytime soon > Could have smelled it:smile:But yes, indeed a complex/fully fledged landing page becomes a shiny app itself
Federico Marini (15:17:09): > Have you played around withreactable? I had the impression it has the capabilities of DT but a nicer SCHMUCK-effect on it
Federico Marini (15:17:24): > dont know how to define it, but it is just… sassy?
Kevin Rue-Albrecht (16:27:34): > Joining in the bandwagon: I absolutely support the idea. > > I’ve thought many times about hooking iSEE onto the ExperimentHub, but those are still too “raw” and need some more post-processing to generate reduced dimensions, markers, etc. > A “catalogue” of 10x datasets processed sufficiently far would be great. > I’ve always pictured an iSEE landing page with clickable links for the various datasets, something roughly likehttp://imlspenticton.uzh.ch:3838/conquer/Anyway.. lots of idea already brainstormed around here, worth a call to draft a plan. > > bunch of iSEE plots > iSEEappsorpanels, “plots” sound so… static:stuck_out_tongue: > > (or Seurat objects:stuck_out_tongue_winking_eye:) > if the iSEE launch script includesas.SingleCellExperiment()we can talk:stuck_out_tongue:We have thehttps://github.com/iSEEorganisation, where we can add repo(s) for that project. Again, we should probably organise a call to chat and break down the work, and talk strategy (e.g. reference(s) to use). I’d imagine at least one repo to process the datasets, and one repo for the launch scripts of the iSEE app(s). > > Great reading the conversation so far!
2020-10-20
Kevin Rue-Albrecht (07:19:20): > iSEEu 1.1.9 pushed upstream
2020-10-21
Kevin Blighe (13:34:50): > @Kevin Blighe has joined the channel
2020-10-22
ImranF (11:54:22): > @ImranF has joined the channel
2020-10-23
ImranF (11:40:10): > @ImranF has joined the channel
2020-10-25
Aaron Lun (14:04:53): > @Kevin Rue-AlbrechtI don’t have write access to the book?
Aaron Lun (14:05:26): > oh wait, is this protected?
Aaron Lun (14:06:32): > Ah damn. It’s because we moved the repo and you forked but my remote was still pointing at your old address.
2020-10-26
Kevin Rue-Albrecht (05:52:47): > iSEE 2.1.27 pushed upstream
Federico Marini (07:40:39): > https://www.biorxiv.org/content/10.1101/2020.10.25.354100v1
Federico Marini (07:40:55): > Not even we got cited in here:disappointed:
Federico Marini (07:41:35): > repo not found (private?)
Kevin Rue-Albrecht (07:57:24): > probably private yes
Kevin Rue-Albrecht (07:57:35) (in thread): > https://github.com/SGDDNB/ShinyCell
Federico Marini (08:44:36) (in thread): > yeah that one
Federico Marini (08:44:45) (in thread): > From the format they seem to go to Bioinformatics
Kevin Rue-Albrecht (08:44:48) (in thread): > hopeless
Kevin Rue-Albrecht (08:45:00) (in thread): > (getting fair citations)
Federico Marini (08:45:30) (in thread): > Yes, especially there
Federico Marini (08:45:41) (in thread): > Huh-. Maybe we get it to review it:smile:
Kevin Rue-Albrecht (08:48:53) (in thread): > it’d be nice to have a second review for Bioinformatics:slightly_smiling_face:AFAIC
2020-11-12
Federico Marini (10:04:37): > Could this become relevant iSEE-wise?https://twitter.com/lazappi/status/1326901725777825801?s=20 - Attachment (twitter): Attachment > Quick benchmark of geom_scattermore (from the scattermore :package:) vs regular geom_point with ~100K points. > > (Note the speedup is on rendering so you have to save the plot to see it, just creating the object is about the same) https://pbs.twimg.com/media/EmoZeofWMAI-WT2.png
Kevin Rue-Albrecht (12:22:55): > @Aaron Lunany idea why this error would happen in DockerHub when building the vignette forhttps://github.com/iSEE/iSEEWorkshopEuroBioc2020? > > E> ***** caught segfault ***** > E> address 0x4, cause 'memory not mapped' > > https://hub.docker.com/repository/registry-1.docker.io/iseedevelopers/iseeworkshopeurobioc2020/builds/43f5302d-3733-4bf2-9638-193ceead6ab8
Kevin Rue-Albrecht (12:23:34): > looks like it’s happening insidescran::computeSumFactors(sce, min.mean = 0.1)
Aaron Lun (12:23:50): > ¯*(ツ)*/¯
Kevin Rue-Albrecht (12:23:57): > thought so
Kevin Rue-Albrecht (12:24:42): > it’s reproducible though - i’ve tried on the bioconductor_devel container as well as the iseedevelopers/isee one
Kevin Rue-Albrecht (12:25:20): > i wonder whether it could be a memory limit on Docker Hub, though I’m not sure whether that could cause a “memory not mapped” error
Aaron Lun (12:26:12): > dense or sparse?
Kevin Rue-Albrecht (12:27:04): > sparse , I think, it’s using the TENxPBMCData
Aaron Lun (12:27:19): > that’s actually file-backed.
Kevin Rue-Albrecht (12:30:06): > oh .. did we run into that same issue for bioc2020 then? i can’t remember:confused:
Kevin Rue-Albrecht (12:30:29): > im trying to build the container locally now, haven’t tried that either yet
Aaron Lun (13:35:06): > I’ve got a container running, give me a MRE.
Aaron Lun (13:37:03): > > library(TENxPBMCData) > sce <- TENxPBMCData('pbmc4k') > > library(scran) > sce <- computeSumFactors(sce) > > works for me
Kevin Rue-Albrecht (14:15:44): > problem solved withas(..., "sparseMatrix")https://hub.docker.com/repository/registry-1.docker.io/iseedevelopers/iseeworkshopeurobioc2020/builds/caa30519-bbeb-4f36-936a-08ea91da7a9e
Kevin Rue-Albrecht (14:16:42): > the MRE is in the vignette of the workshop package I linked above
Aaron Lun (14:16:43): > well, you shouldn’t have to do taht
Aaron Lun (14:16:53): > can you just post some lines here
Kevin Rue-Albrecht (14:17:19): > > library(scran) > sce <- scran::computeSumFactors(sce, min.mean = 0.1) >
Kevin Rue-Albrecht (14:17:34): > which is what you had above
Aaron Lun (14:17:38): > how isscegenerated?
Kevin Rue-Albrecht (14:18:23): > > library(TENxPBMCData) > sce <- TENxPBMCData(dataset = "pbmc3k") > > library(scater) > colnames(sce) <- paste0("Cell", seq_len(ncol(sce))) > rownames(sce) <- scater::uniquifyFeatureNames( > ID = rowData(sce)$ENSEMBL_ID, > names = rowData(sce)$Symbol_TENx > ) > > MT <- rownames(sce)[grep("^MT-", rownames(sce))] > sce <- scater::addPerCellQC(sce, subsets = list(MT = MT)) > sce <- scater::addPerFeatureQC(sce) > sce$log10_total <- log10(sce$total) > rowData(sce)$n_cells <- as.integer(rowData(sce)$detected * ncol(sce)) > rowData(sce)$log10_total <- log10(rowSums(assay(sce, "counts")) + 1) > > (sce <- sce[, sce$subsets_MT_percent < 5]) > > library(scran) > sce <- scran::computeSumFactors(sce, min.mean = 0.1) >
Kevin Rue-Albrecht (14:19:18): > from my reading of the DockerHub log file, that’s where it crashes, but then the vignette continues with
Kevin Rue-Albrecht (14:20:01): > > sce <- scater::logNormCounts(sce) > > dec <- scran::modelGeneVar(sce) > top.dec <- dec[order(dec$bio, decreasing = TRUE), ] > > library(BiocSingular) > set.seed(1000) > sce <- scran::denoisePCA(sce, technical = dec) > > set.seed(1000) > sce <- scater::runTSNE(sce, dimred = "PCA", perplexity = 30) > sce <- scater::runUMAP(sce, dimred = "PCA") > > snn.gr <- scran::buildSNNGraph(sce, use.dimred = "PCA") > clusters <- igraph::cluster_walktrap(snn.gr) > sce$Cluster <- factor(clusters$membership) > table(sce$Cluster) > markers <- scran::findMarkers(sce, groups = sce$Cluster, > test.type = "t", > direction = "up", pval.type = "all") > for (i in names(markers)) { > rowData(sce)[, paste0("FDR_cluster", i)] <- > markers[[i]]$FDR[match(rownames(sce), > rownames(markers[[i]]))] > } >
2020-11-19
ImranF (07:45:20): > Hi folks, within any panel whenever I select “Feature” I get the following error: > > Warning: Error in intI: invalid character indexing > [No stack trace available] > Warning: Error in intI: invalid character indexing > [No stack trace available] > > Myrownames(sce)looks something like: > > [1] "WASH7P_ENSG00000227232.5" "AL627309.5_ENSG00000241860.7" "WASH9P_ENSG00000279457.4" "AP006222.1_ENSG00000228463.10" > > and I am using iSEEiSEE_2.0.0Thoughts?
Kevin Rue-Albrecht (07:52:00): > Any chance you can update to iSEE 2.2.1 and confirm that the issue is still valid in that latest release?
ImranF (07:57:34) (in thread): > naive question, I did aBiocManager::install("iSEE")but still ended up getting 2.0. Any specific command?
Kevin Rue-Albrecht (07:58:16) (in thread): > You probably don’t use the latest (stable) version of R then.
ImranF (07:58:56) (in thread): > R version 4.0.2 (2020-06-22)
Kevin Rue-Albrecht (07:59:18) (in thread): > BiocManager considers the version of R you’re running and gives you the latest version of Bioc packagesfor that version of R
ImranF (07:59:31) (in thread): > I see
ImranF (08:00:00) (in thread): > Sow aht should my R version be in order to get version 2.2.1 ?
ImranF (08:00:13) (in thread): > Because the docs just says “To install this package, start R (version "4.0") and enter:
Kevin Rue-Albrecht (08:00:29) (in thread): > I just sanity-checked: > > Bioconductor 3.12 is compatible with R 4.0.3 > https://bioconductor.org/news/bioc_3_12_release/
ImranF (08:02:56) (in thread): > So basically I need R 4.0.3 > Bioconductor 3.12 > iSEE 2.2.1
Kevin Rue-Albrecht (08:03:02): > To be clear, iSEE has supported - even required - character rownames since the early days. One thing that I’m not familiar with in this error message is the part that mentions > > Error in intI >
Kevin Rue-Albrecht (08:03:24) (in thread): > Yep
Kevin Rue-Albrecht (08:04:03) (in thread): > Though I just made a point in the main channel that I’m still surprised by this error.
Kevin Rue-Albrecht (08:04:27) (in thread): > Perhaps before you do the whole new installation you could simply check that BiocManager::valid() returns TRUE
Kevin Rue-Albrecht (08:04:37) (in thread): > (with your current version of R)
Kevin Rue-Albrecht (08:07:13): > I just found a similar issue on another packagehttps://github.com/satijalab/seurat/issues/1138#issuecomment-464190304My guess would be that the rownames are somehow changed at some point, and when you select “Feature”, it searches for a row name that doesn’t exist anymore
Kevin Rue-Albrecht (08:10:27) (in thread): > however, from a brief look, the only reason why we would touch rownames is if they don’t exist in the first place:https://github.com/iSEE/iSEE/blob/master/R/iSEE-main.R#L674
Kevin Rue-Albrecht (08:16:15): > I’ve just used the iSEE man page example, suffixing rownames with “.1” and “_1”, then I’ve colored the Red. Dim. plot panel by Feature name, and I’ve also changed the selected Y-axis feature for the Feat. assay plot panel, all working
Kevin Rue-Albrecht (08:18:39): > Perhaps send a screenshot as well, to show what you’re selecting, because you mentioned “Feature” when I can only see “Feature name”, for instance under “Visual Parameters” > “Color” > “Color by”
Kevin Blighe (08:28:48): > @Kevin Blighe has joined the channel
ImranF (09:32:26) (in thread): > Here you go KEvin
ImranF (09:32:30) (in thread): - File (PNG): image.png
ImranF (09:37:01) (in thread): > > > is.null(rownames(sceB)) > [1] FALSE >
ImranF (09:38:34) (in thread): > > Warning message: > 16 packages out-of-date; 5 packages too new > > Is that a bad thing? I’d prefer to not update things for at least a couple days (upcoming deadline)
Kevin Rue-Albrecht (10:04:47) (in thread): > yep, sorry, I just pointed that out of general interest, I saw that your data set had rownames
Federico Marini (11:08:00) (in thread): > the too new thing is if you install e.g. from github
Federico Marini (11:08:18) (in thread): > the old is just… well, there’s a newer released version
Federico Marini (11:08:33) (in thread): > if you have anyway the latest bioc release, it can not be THAT old
Federico Marini (11:09:52): > Can that be the usage of dots that screws it up?
Federico Marini (11:10:12): > Although, that is expected to be somewhat common if one uses Gencode
Kevin Rue-Albrecht (11:12:42) (in thread): > So it crashes when you select “Feature name”?
Kevin Rue-Albrecht (11:14:42) (in thread): > And another question, do you have the same issue when you run the example of the ?iSEE man page?
Aaron Lun (11:18:03): > Smells like a sparse matrix problem.
Kevin Rue-Albrecht (11:20:15) (in thread): > In one of the threads above, I tried adding dots to row names if the example ma page, still worked
Aaron Lun (11:27:49): > Basically it’s the error you get when you try to extract a row based on a non-existent row name: > > library(Matrix) > m <- rsparsematrix(10, 10, 0.1) > rownames(m) <- LETTERS[1:10] > m["Z",] > ## Error in intI(i, n = x@Dim[1], dn[[1]], give.dn = FALSE) : > ## invalid character indexing >
Aaron Lun (11:29:30): > Now, the question is how did this non-existing row name possibly get generated. Double-check that you don’t have any non-ASCII characters in the names.
Federico Marini (11:50:09): > or some column-name-making functions are tricking us on that
Federico Marini (11:50:51): > I am thinking ofmake.namesor similar that get called unadvertedly
Federico Marini (11:51:22): > What is if you try to replace the part after the dot (dot included) with just nothing@ImranF?
ImranF (12:10:55): > In that case, here’s the “full” error msg: > > Warning in .coerce_type(v, f, as_numeric = !.is_groupable(v), df = df) : > coloring covariate has too many unique values, coercing to numeric > Warning: Error in intI: invalid character indexing > [No stack trace available] > Warning: Error in intI: invalid character indexing > [No stack trace available] >
ImranF (12:11:18): > the.coerce_type()give you any hints?
ImranF (12:13:00): > Also, I dont think its a rownames() issue. Because, I didrownames(sce) <- NULLand still got the same error
ImranF (12:13:21): > So@Federico Marini, you might be onto something
ImranF (12:13:54): > Though, I should just update my R -> BioC -> iSEE before you debug further
Kevin Rue-Albrecht (12:19:03): > As a sanity check, do you have the same issue when you run the example of the?iSEEman page?
Kevin Rue-Albrecht (12:20:37): > Easiest way to check if it’s something to do with your dataset or your package library
ImranF (12:23:14): > Ok, I see ran the example snippet under ?iSEE. No problems. iSEE didnt crash.
Kevin Rue-Albrecht (12:24:22): > Ok, so that points to an issue with your dataset then. I think Aaron was onto something when he mentioned > > Now, the question is how did this non-existing row name possibly get generated. Double-check that you don’t have any non-ASCII characters in the names.
Kevin Rue-Albrecht (12:25:34): > Though I also saw that you mentioned trying setting the rownames toNULL, which would have forced iSEE to generate valid dummy rownames on the fly, so I’m not sure what the next move is.
Kevin Rue-Albrecht (12:26:10): > How aboutrownames(sce) <- paste0("Gene", seq_len(nrow(sce)))?
Kevin Rue-Albrecht (12:26:43): > I typed it on the fly, but basically rename your genes “Gene1”, “Gene2”, … and see if that works
Kevin Rue-Albrecht (12:27:14): > if so, then it would suggest that one of your original gene name contains a non-ASCII character that trips iSEE at runtime
ImranF (12:28:11): > I tried `rownames(sce) <- paste0("Gene", seq_len(nrow(sce))), still crashes. So it cant be non-ASCII character
Aaron Lun (13:11:25): > You’ll need to give us a MWE to debug this. Maybe take a 1:100,1:100 subset of the SCE and post that on dropbox?
ImranF (13:30:18): > Ok, but I have a feeling that its a version issue. I’ve shared this same sce with a collaborator and he hasnt reported this problem (but maybe he didnt use this feature, not sure). Nonetheless, here’s a mini-sce (that also crashes):
ImranF (13:30:36): - File (Gzip): buggy_sce.rds
ImranF (15:36:55): > Update: So updating to iSEE version 2.2.1 has solved the aforementioned issue
Aaron Lun (15:39:50): > yeah, I can’t repro it, so I guess it’s fixed.
2020-11-25
Kevin Rue-Albrecht (10:09:22): > It was clearly not the point of the photo, but all I can see is the sticker:rolling_on_the_floor_laughing: - File (JPEG): File from iOS
Federico Marini (10:24:07): > just like the gorilla experiment
Kevin Rue-Albrecht (10:41:15) (in thread): > had to google that one ^^
2020-12-01
Kevin Stachelek (22:24:04): > @Kevin Stachelek has joined the channel
2020-12-02
Konstantinos Geles (Constantinos Yeles) (05:42:59): > @Konstantinos Geles (Constantinos Yeles) has joined the channel
2020-12-06
Federico Marini (16:01:42): > Happy birthday iSEE! - File (PNG): image.png
2020-12-12
Huipeng Li (00:39:56): > @Huipeng Li has joined the channel
2020-12-13
Kelly Eckenrode (13:41:38): > @Kelly Eckenrode has joined the channel
2020-12-14
Nick Owen (13:21:20): > @Nick Owen has joined the channel
2020-12-15
Jeroen Gilis (06:04:54): > @Jeroen Gilis has joined the channel
2020-12-16
Assa (12:56:10): > @Assa has joined the channel
Thomas Naake (15:04:17): > @Thomas Naake has joined the channel
Peter Hickey (22:22:24): > is it possible to visualize values from analtExpof aSingleCellExperimentin iSEE? E.g., overlaying CITE-seq / FACS data stored in analtExpon a UMAP plot or in the Feature assay plot?
2020-12-17
Aaron Lun (01:51:36): > latest SCE has anunsplitAltExps()function that is probably the way to go here. Easier than modifying the UI to support more possible selections from the altexps.
Peter Hickey (02:21:25): > ah cool, that’ll do the trick from the looks of it
Kevin Rue-Albrecht (03:42:02): > I just wonder about the compatibility of colormaps across multiple altExps with potentially different dynamic ranges
Kevin Rue-Albrecht (03:44:30): > if that becomes a problem let me know, I have an update of the ExperimentColorMap class lined up, which I’ll probably get to after the Xmas break
Aaron Lun (03:46:12): > doesn’t really matter, I guess, if the colormap rescales for each feature.
2020-12-18
Anne-Maud Ferreira (12:47:18): > @Anne-Maud Ferreira has joined the channel
2020-12-31
Aaron Lun (04:11:13): - File (PNG): Screenshot from 2020-12-31 01-10-53.png
Aaron Lun (20:20:02): - File (MPEG 4 Video): iSEE_screencast.mp4
2021-01-01
Federico Marini (09:29:47): > Oh this is very nice!
Federico Marini (09:30:12): > could even be used to store something like a rendered vignette in it
Federico Marini (09:30:26): > Or, well, anime - that is prio 1
Federico Marini (09:30:35): > Happy new year to you all!
Bernd (14:05:34): > @Bernd has joined the channel
Aaron Lun (14:15:41): > I see you didn’t notice the other feature I added.
Aaron Lun (14:15:50): > Which makes sense, vignette really is quite cute.
2021-01-02
Aaron Lun (00:42:34): > Anyway@Kevin Rue-Albrechtboth packages need pushes to BioC.
Federico Marini (04:45:53) (in thread): > gotta catch up in the fam-free time:smile:
Kevin Rue-Albrecht (12:49:55): > cool - i saw the flood of email notifications in my mailbox, I just had other fires to put out elsewhere. Thanks for the green light
Kevin Rue-Albrecht (12:51:53): > Also, I’ve got a feeling that there was a recent PR that you wanted me to push to release, but I can’t remember which one, and I’ve got issues switching between R 4.1 and 4.0, so I can’t safely test stuff before pushing anything to release. I need to sort that out too. It’s going to be a busy January
2021-01-03
Federico Marini (08:46:46) (in thread): > WUT
Federico Marini (08:46:51) (in thread): > Lasso?
Federico Marini (08:46:53) (in thread): > :smile:
Federico Marini (08:47:08) (in thread): > :kissing:
Federico Marini (08:47:38) (in thread): > Now the RStudio team can really have one less excuse to not push it
Aaron Lun (14:51:21) (in thread): > no, the feature requires you to not watch the gifs and focus on what happens when I make a selection on the plot.
2021-01-04
Aaron Lun (16:44:42): > iSEEu alsoneeds apush.
2021-01-05
Kevin Rue-Albrecht (05:45:14): > done
2021-01-07
Federico Marini (16:54:48) (in thread): > shit it is a 2021 version of the gorilla experiment
Federico Marini (16:56:28) (in thread): > omg, this is goodgood!
Federico Marini (16:56:40) (in thread): > Jeez I am slow these days
2021-01-13
USLACKBOT (10:43:05): > This message was deleted.
Kevin Rue-Albrecht (11:41:52) (in thread): > answered, thanks!
Kevin Rue-Albrecht (11:42:20) (in thread): > I need to check again why I don’t see email notifications even though I follow the isee tag
Kevin Rue-Albrecht (11:42:39) (in thread): > might be redirected to a folder instead of the inbox
Kevin Rue-Albrecht (11:44:06) (in thread): > Uh… weird. I got an email notification about my answer, but none when the question was posted
2021-01-22
Annajiat Alim Rasel (15:44:17): > @Annajiat Alim Rasel has joined the channel
2021-01-26
Aaron Lun (00:15:02): > damn. Took me ages to figure out why the hell mytooltipwasn’t working.
2021-01-28
Aaron Lun (12:36:13): > @Kevin Rue-AlbrechtiSEE tests are borked, probably ude to update to shiny
Kevin Rue-Albrecht (12:46:02): > arf
Aaron Lun (12:46:49): > just one test. We probably missed asession=NULLsomewhere.
2021-01-29
Federico Marini (11:32:32): > I’m having a look at that now but it looks that our session is simply “not given as a ShinySession object”
2021-01-31
Aaron Lun (15:53:12): > so, is someone going to fix this, or what?
2021-02-01
Kevin Rue-Albrecht (04:39:35): > looking at it now (and updating the renv), indeed it looks like my hack of passing an environment forsessionwon’t work anymore, if Shiny has started checking the class of the session object
Kevin Rue-Albrecht (04:40:00): > weird that it break only in one place, i guess we should expect a bunch of other places to break soon too
Federico Marini (04:48:34): > This is what I meant with my former comment - is the ShinySession object a new thing then?
Kevin Rue-Albrecht (04:54:21): > yes, this seems to be the way to go now (I don’t know for how long it’s been)https://shiny.rstudio.com/reference/shiny/1.5.0/MockShinySession.html
Kevin Rue-Albrecht (04:55:05): > also the test that fails didn’t have anexpect_statement, which means that I don’t know if the current output (FALSE) is the one originally intended
Kevin Rue-Albrecht (04:57:32): > i’ve been trying to read through the code, to figure out what is supposed to happen… no idea so far
Kevin Rue-Albrecht (05:13:49): > ok i got it #FamousLastWords
Federico Marini (05:22:34): > shit I was getting there as well
Federico Marini (05:22:55): > I find R6 classes too much c++-like to really like them:smile:
Kevin Rue-Albrecht (05:23:09): > I still have it, kid:yum:
Federico Marini (05:23:17): > Merci:wink:
Kevin Rue-Albrecht (05:23:20): > :face_with_cowboy_hat:
Kevin Rue-Albrecht (05:25:22): > Emojis have gone so safe that there isn’t a revolver to play cowboy, barely a water pistol:gun:
Kevin Rue-Albrecht (05:27:00): > Anyway, in the end I resolved what the test was supposed to do and added a proper explicit check
Kevin Rue-Albrecht (05:27:45): > Don’t make me write guidelines for writing unit tests… you know I will:sweat_smile:
Kevin Rue-Albrecht (05:47:13): > > rcmdcheck::rcmdcheck() > 0 errors ✓ | 1 warning x | 2 notes x > > on my laptop, waiting for GHA to merge and push upstream
Federico Marini (05:50:14): > > ══ Results ══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ > Duration: 31.3 s > > ── Skipped tests ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── > ● empty test (1) > ● waiting for a fix to the SCE itself (1) > > [ FAIL 0 | WARN 3 | SKIP 2 | PASS 1398 ] > > local from mine for testing
Federico Marini (05:56:21): > check’S all good here as well, 0|1|2
Kevin Rue-Albrecht (06:22:36): > FYI, I’m porting the fix to release right now, testing locally on 3.12
Kevin Rue-Albrecht (06:23:24): > I’ve been thinking of replacing our GHA workflow by Leo’s one, while tweaking the GHA on release branches to test it on the relevant docker image of bioc
Kevin Rue-Albrecht (06:24:59): > because our time saving hack of predeploying an iSEE docker image to speed up our regular GHA workflow looks like the release branch will use that devel-based image too
Kevin Rue-Albrecht (06:25:15): > :face_with_head_bandage:
2021-02-02
Kevin Rue-Albrecht (06:58:18): > team
Federico Marini (07:29:25): > Presente. > I’d say that we should probably switch to a less customized one then
Aaron Lun (11:28:51): > what does leo’s one do?
Aaron Lun (11:28:56): > because the time saving is considerable.
2021-02-04
Aaron Lun (01:06:46): > Click on the little question mark - File (PNG): Screenshot from 2021-02-03 22-06-16.png
Aaron Lun (01:07:22): > and you get a tour for that part of the app - File (PNG): Screenshot from 2021-02-03 22-06-54.png
2021-02-20
Aaron Lun (22:20:13): > @Kevin Rue-Albrechtare you back in the land of the living this week?
2021-02-21
Kevin Rue-Albrecht (03:44:34): > Teaching the next 3 weeks. I’m looking at velociraptor this weekend
Aaron Lun (05:17:38): > because the distributed documentation needs to be added to the heatmap as well.
Kevin Rue-Albrecht (09:49:09): > right
2021-02-22
Aaron Lun (03:57:51): > finally, all the dot plot-specific help pages are done.
Kevin Rue-Albrecht (04:36:34): > Cool - I’m almost done with velociraptor, damn GHA takes its sweet time. I want to try out Leo’s biocthis alternative, I can’t remember if it uses a smarter caching mecanism. Although I do like the fact that it tests multiple platforms already on GH before it gets pushed to the BBS
2021-03-10
Federico Marini (17:00:30): > ICYMI:http://beta.vitessce.io/ - Attachment (beta.vitessce.io): Vitessce > Vitessce is a visual integration tool for exploration of spatial single-cell experiments.
Aaron Lun (17:01:16): > oh, that’s from the gehlenborg lab
Aaron Lun (17:01:35): > the upsetR guys
Aaron Lun (17:05:35): > Anything useful that we could steal?
Federico Marini (18:12:14): > R-side, they go heavy on the htmlwidgets way
Federico Marini (18:13:41): > I’d have to try the full set of frameworks, but here’s naptime:stuck_out_tongue:
2021-03-11
Kevin Stachelek (00:22:51): > Hi, I would like to make use ofInteractiveComplexHeatmapin iSEE. I thought first, that I might modify ComplexHeatmap code to manage this. I am daunted by the class-based construction of iSEE. Does it make sense to use InteractiveComplexHeatmap in iSEE? is iSEEu the best place to do this? Do you have a walkthrough for creating a new panel?
Federico Marini (05:10:28): > Hello there@Kevin Stachelek! > You might start looking at the vignettes for theiSEEu(niverse) package, and also this onehttp://www.bioconductor.org/packages/devel/bioc/vignettes/iSEE/inst/doc/custom.html
Kevin Stachelek (08:46:51) (in thread): > Thanks! I now see the iSEE book. That looks the introduction that I was hoping for.
Federico Marini (10:52:46) (in thread): > Yes, that is another excellent candidate venue.
Federico Marini (10:53:29) (in thread): > Feel free to drop by again if you have questions, or if you want to showcase it once it is up & running!
Kevin Stachelek (11:13:56) (in thread): > After checking the book and going through some of the example panels I have to ask one more thing. iSEE makes use of pObjects and rObjects for sharing application state. InteractiveComplexHeatmap uses the functionmakeInteractiveComplexHeatmapthat takes as arguments theinput``outputandsessionrather than arender*function. Is it possible to adapt to iSEE where these are not directly accessible? Maybe I don’t understand the observer creation well enough. Will dive deeper if this doesn’t sound problematic to you.
Aaron Lun (11:22:02) (in thread): > Unfortunately, there’s a very explicit demarcation betweeninputandoutputiniSEE. Rendering expressions do not have direct access toinput; any changes toinputget “laundered” through changes to the class parameters. This is necessary to ensure that we can reproduce the application state from the inputPanelclasses; if a panel talked directly withinput, it would be evading our ability to capture user interactions.
Aaron Lun (11:23:02) (in thread): > We could also have this discussion athttps://github.com/iSEE/iSEE/issues/546. - Attachment: #546 [Suggestion] link ComplexHeatmapPanel to other panels > iSEE can transmit information (selected rows and columns) between different panels. Currently ComplexHeatmap panel is the end of the transmission (https://github.com/iSEE/iSEE/blob/master/R/observers_heatmap.R#L139|https://github.com/iSEE/iSEE/blob/master/R/observers_heatmap.R#L139) and there is no further actions other panels can play with ComplexHeatmap panel. > > I am currently development a package InteractiveComplexHeatmap which can convert static complex heatmaps to interactive heatmaps. I’ve read how to implement new panel in iSEE in the vignettes, but it seems it only supports to output a plot or a table, while InteractiveComplexHeatmap outputs a complex widget, or maybe I was lost in the vignette because there are too much information there :). > > However, there are following functions in InteractiveComplexHeatmap that might be helpful for you to continue the transmission starting from ComplexHeatmap panel to other panels. I think the key thing here is to capture the row indices and column indices when users brush or click on the heatmap. > > Basically three functions from InteractiveComplexHeatmap are to be used. First, the positions of all heatmap slices are retrieved by function htPositionsOnDevice(). I think in ComplexHeatmap panel in iSEE, there is only one heatmap with no splitting, so here this functions returns the position of the single heatmap body. Note this function should be called inside a plotOutput because the heatmap should be generating and htPositionsOnDevice() needs to access various viewports to get the positions. > > > ht_list = draw(ht_list) > ht_pos = htPositionsOnDevice(ht_list) > > > Second, the position of the brush (if users brush on the heatmap) can be obtained by InteractiveComplexHeatmap:::get_pos_from_brush(). The argument for this argument is the brush ID on the UI. This function simply returns a list of two unit objects, which are the positions of the bottom left and top right of the brush rectangle. > > > lt = InteractiveComplexHeatmap:::get_pos_from_brush(input$heatmap_brush_id) > pos1 = lt[[1]] > pos2 = lt[[2]] > > > Last, send the brush position and heatmap position to function selectArea(). There are several arguments there: > > • ht_list: the heatmap object. Because the rows and columns might be reordered due to clustering, the reordering informationn is stored in the heatmap object. > • mark: always set to FALSE. > • pos1 and pos2: position of the brush. > • verbose set to FALSE to turn off messages. > • ht_pos: position of the heatmap. > > > df_brush = selectArea(ht_list, mark = FALSE, pos1 = pos1, pos2 = pos2, verbose = FALSE, ht_pos = ht_pos) > > > The function returns a data frame of the row indices and column indices that users selected. An example output is as follows.
> In the example, the heatmap is split by k-means and the brush was overlap heatmap slices, so you can see the data frame contains multiple rows and each row corresponds to one row slice and one column slice. > > > ## DataFrame with 4 rows and 6 columns > ## heatmap slice row_slice column_slice row_index > ## <character> <character> <numeric> <numeric> <IntegerList> > ## 1 mat_a mat_a_heatmap_body_1_2 1 2 7,5,2,... > ## 2 mat_a mat_a_heatmap_body_2_2 2 2 6,3 > ## 3 mat_b mat_b_heatmap_body_1_1 1 1 7,5,2,... > ## 4 mat_b mat_b_heatmap_body_2_1 2 1 6,3 > ## column_index > ## <IntegerList> > ## 1 2,4,1,... > ## 2 2,4,1,... > ## 3 1,2,3,... > ## 4 1,2,3,... > > > > It is similar to capture the row/column index for the single cell that user clicked (or hovered/double clicked). Just change to the functions InteractiveComplexHeatmap:::get_pos_from_click() and selectPosition(). > > > pos1 = InteractiveComplexHeatmap:::get_pos_from_click(input$heatmap_click_id) > df_click = selectPosition(ht_list, mark = FALSE, pos = pos1, verbose = FALSE, ht_pos = ht_pos) > > > The returned object df_click always have one row if a cell was clicked. > > > ## DataFrame with 1 row and 6 columns > ## heatmap slice row_slice column_slice row_index > ## <character> <character> <numeric> <numeric> <integer> > ## 1 mat_a mat_a_heatmap_body_1_2 1 2 4 > ## column_index > ## <integer> > ## 1 2 > > > > If users didn’t brush or click into the heatmap, selectArea() and selectPosition() return NULL. > > Then, I guess the captured row indices and column indices can be sent to other panels. > > Hope it helps!
Kevin Stachelek (12:34:19) (in thread): > Thank you. This looks like something a little beyond my scope. I’ll follow the issue. I’m interested because I appreciate functions inInteractiveComplexHeatmapthat allow me zoom to heatmap subregions, search, modify and export images/tables.
2021-03-18
Aaron Lun (03:10:38): > @Kevin Rue-Albrechtdid the latest stuff get pushed to BioC-devel?
Aaron Lun (03:11:20): > also, something’s borked with our actions.
Kevin Rue-Albrecht (11:16:25): > Yeah, I’ve pushed the distributed tour as a new ‘devel’ release. Looks ok
Kevin Rue-Albrecht (11:17:02): > actions are a WIP, I’m on a Carpentries instructor course this week, but I’ve started looking into an update to GHA
2021-03-22
Martin Morgan (14:23:51): > @Martin Morgan has joined the channel
Martin Morgan (14:28:56): > Is there a script that i could follow (literal step by step) to demo the voice commands inhttps://www.bioconductor.org/packages/devel/bioc/vignettes/iSEE/inst/doc/voice.html#5_Vocal_commands_available? I’m supposed to talk about bioc at the end of the week, and like to ‘live code’ but unfortunately broke my arm! Would be a great demo to reveal the cast then do something worth while anyway. Also would there be challenges doing this over a zoom call, e.g. competition for the mic or background noise?
Kevin Rue-Albrecht (14:32:50): > Hi Martin, > So sorry to hear about your arm, I hope you get better soon. > You should be able to run the example in?iSEEonly addingvoice=TRUEin the call toiSEE(..., voice=TRUE). > I haven’t tested the feature recently, and it’s not something that can be unit tested, so I’ll give it a whirl in a moment and I welcome feedback as well!
Kevin Rue-Albrecht (15:10:38): > actually one more tweak > > app <- iSEE(sce, voice = TRUE) > if (interactive()) { > shiny::runApp(app, port=1234, launch.browser = TRUE) > } >
Aaron Lun (15:11:11): > don’t forget to speak with a french accent.
Kevin Rue-Albrecht (15:11:12): > better to launch in a web browser usinglaunch.browser = TRUEas it allows you to enable the microphone
Charlotte Soneson (15:12:03): > Oh no, sorry about your arm Martin! Just to say that I did try the voice control features over Zoom, and it was able to pick up my voice fine, at least with headphones (I don’t actually know what would happen if someone else was speaking too, but I don’t think that should be an issue). If I remember correctly, I had some problems getting it to understand “Colour using column data”, not sure if it’s my accent…
Federico Marini (15:28:19): > As a sometimes-wanna-trier of that feature, I can highly recommend > * Chrome > * “start silent” - it is not Alexa, so you cannot really filter out what/when the mic starts picking up > * my classical ride-through is > > * “control reduced dimension panel 1” > * “color using column data” > * “color by driver 1 s” > * “color by secondary type”
Federico Marini (15:29:00): > Get better soon first of all!:hugging_face:
Martin Morgan (16:36:09): > Thanks! I won’t have a chance to try this until Thursday or Friday but it sounds promising I’ll get back my experience then. Thanks again for the awesome support
Kevin Rue-Albrecht (16:38:20): > The positive “Good boy”:dog:still works for me. I got interrupted before I could test the other commands:slightly_smiling_face:
Charlotte Soneson (16:39:11): > That one worked for me too!
Kevin Rue-Albrecht (16:40:47): > btw, I seem to remember that Google Chrome is theonlybrowser supported, which is a limitation of the JavaScript library that we use (annyang)
Federico Marini (16:43:41): > Oh yes, the “Good boy” is always a nice way to close:smile:
Federico Marini (16:44:09): > Given that we don’t have the “Dracarys” way of wrapping up the analysis session…
2021-03-23
Zizhen Song (15:54:39): > @Zizhen Song has joined the channel
2021-03-25
Zizhen Song (11:15:28): > Hello! I am a bit new to iSEE. I was wondering if there are there options/ panels that offer dynamic loading of panel settings from the output of the “display panel settings” button. I am trying to offer the option for users to be able to leave and comeback to their work with the same settings that they left with. I was envisioning a large text box that the users can paste their old settings into to load the layout that they want. Has anyone done something like that in the past?
Kevin Rue-Albrecht (11:20:10): > Currently, the output of the “display panel settings” button is meant as a read-only output only, not as an input. It might be possible to (re)set those settings at runtime, but I need to think about it more. Anything that is done at runtime has the potential to crash the app and needs to be carefully considered, which is why most users interactions are expected through UI elements rather than code
Kevin Rue-Albrecht (11:21:42): > in this particular case, the app would probably need to ‘dry-run’ the new settings that users would paste, to check whether they are valid and safe, which is a really difficult problem to tackle, as invalid and unsafe code can take many forms
Kevin Rue-Albrecht (11:22:56): > That said, thanks for your suggestion, we might look into it in due time. It won’t happen before the next release (3.13) as there is very little time left to work on any new functionality
Zizhen Song (11:23:43): > Thank you so much Kevin, that makes a lot of sense.
2021-03-26
Martin Morgan (16:07:52): > My voice iSEE presentation went without a hitch. It took about 20 minutes to install stuff and to walk through the example on?iSEE. And then using the tweaked command provided by Kevin to launch the app, an initial run-through of Federico’s script, a run through on a zoom call with myself with Charlotte’s hints in mind. And then showtime! You could see the zoom call participants light up with delight! > > Thanks for the amazing support! > > My only request would be for a ‘dog’ flavor of the ‘good’ command:dog:!
Aaron Lun (16:09:30) (in thread): > if we were to do this, they would need to be uploading a striclty controlle format, e.g., JSON or YAML. Evaluation of arbitrary user code would be extremely dangerous in typical Shiny deployment environments.
2021-03-27
Aaron Lun (03:15:28): > @Kevin Rue-AlbrechtiSEEu needs a push to BioC-devel.
Charlotte Soneson (05:01:11) (in thread): > Good to hear that it went well!
Kevin Rue-Albrecht (13:23:39): > will do now
Kevin Rue-Albrecht (13:23:56): > in the meantime, I just stumbled onto this by chance - File (PNG): image.png
Aaron Lun (20:15:13): > ha
Aaron Lun (20:15:15): > haha
Aaron Lun (20:15:16): > hahahaha
Aaron Lun (21:20:42): > iSEE itself will also need a push. And our CI is broken.
2021-03-28
Kevin Rue-Albrecht (05:22:42): > CI was broken because of GenomicFeatures again a few days ago. I was waiting that things settled before trying again I did run rcmdtest locally though, before i pushed upstream
2021-04-02
Tim Howes (18:01:37): > @Tim Howes has joined the channel
2021-04-03
Aaron Lun (03:07:48): > Another day, another few pushes needed to BioC.
2021-04-05
Kevin Rue-Albrecht (13:38:55): > i was going to ask, you added one commitafterthe one were you version-bumped, if I pushed as it is I suspect that the extra commit would be included in the release, shall i push the commit of the version bump first, and only push the next one one the release is out?
Aaron Lun (16:52:03): > no, that’s fine, just pusht hem all.
Aaron Lun (16:53:29): > I just couldn’t be boethered to do another version bump.masteris always valid anyway.
2021-04-06
Lindsay Hayes (00:12:07): > @Lindsay Hayes has joined the channel
Kevin Rue-Albrecht (05:06:10): > ok it should be up to date now
Kevin Rue-Albrecht (05:49:38): > @Aaron Lunwhile answering an email for someone who’s interested in writing a custom iSEE panel, I noticed that in the iSEE book, chapters ‘Dynamic reduced dimensions’ and ‘Dynamic differential expression’ don’t have asetClasscall. They go straight intosetValidity2. The last two chapters have a chunk that doessetClassbefore the chunk that doessetValidity2. Is it an omission? Somehow the book still seems to compile fine, but I don’t get how the chapters can work on classes that are not defined.
Aaron Lun (11:24:11): > ¯*(ツ)*/¯
Aaron Lun (11:24:13): > probably
Kevin Rue-Albrecht (12:08:48): > also, it looks like bookdown doesn’t cope well with identically-named sections in separate chapters, all the ‘In action’ headers point to the first occurence in the hexagonal reduced dimension plot chapter.
Aaron Lun (12:09:37): > Maybe it needs to be rebuilt. This might be fixed. Observed for OSCA earlier but isn’t an issue anymore.
2021-04-09
Aaron Lun (01:28:43): > hey, what happened to the push?
Aaron Lun (01:40:12): > for iSEEu
Kevin Rue-Albrecht (05:17:00): > done - sorry i only spotted the iSEE update
2021-04-11
Aaron Lun (18:54:59): > another week, another iSEEu bump.
2021-04-12
Jayaram Kancherla (19:05:40): > I had to make a smaller, single-line version of the iSEE logo, hopefully everyone here approves:slightly_smiling_face: - File (PNG): isee_hori.png
2021-04-13
Federico Marini (02:43:08): > I do like it:slightly_smiling_face:Do you need the hi-res version for the goggles part?
Kevin Rue-Albrecht (03:22:30): > hell yeah I approve this one:smile:
Jayaram Kancherla (09:18:12) (in thread): > I don’t but would be great to replace that with a high res version
Federico Marini (10:24:29) (in thread): > I’ll look it up later:wink:
Federico Marini (15:23:40) (in thread): > I think you can safely use this:https://github.com/Bioconductor/BiocStickers/blob/master/iSEE/glassescc0_full_opaque.png - Attachment: iSEE/glassescc0_full_opaque.png
Federico Marini (15:24:33): > or, well, the one from the sticker as a pdf:https://github.com/Bioconductor/BiocStickers/blob/master/iSEE/iSEE.pdf
Federico Marini (15:24:44): > (I noticed the original goggles are a little squished)
Aaron Lun (15:27:38): > @Kevin Rue-Albrechtcan you push iSEEumaster?
2021-04-14
Kevin Rue-Albrecht (04:41:17) (in thread): > sure
Kevin Rue-Albrecht (05:35:46) (in thread): > done
2021-04-17
Jayaram Kancherla (00:40:15): > @Aaron Lundoes any method expose the session object ?
Aaron Lun (00:40:27): > Some of them do
Aaron Lun (00:40:42): > hm. I thinkcreateObserversdoes.
Aaron Lun (00:43:45): > See if you can sum up any blockers re. the interaction with custom JS plugins
Jayaram Kancherla (01:23:22): > iSEEuhas installation errors > > **** byte-compile and prepare package for lazy loading > Error in setMethod(".definePanelTour", "AggregatedDotPlot", function(x) { : > no existing definition for function '.definePanelTour' > Error: unable to load R code in package 'iSEEu' > Execution halted > ERROR: lazy loading failed for package 'iSEEu' > * removing '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/iSEEu' > Error: Failed to install 'iSEEu' from GitHub: > (converted from warning) installation of package '/var/folders/qt/2sv1dfl5249gsvlbwg7p2sf40000gp/T//RtmpOvXlLh/file7db3118234fe/iSEEu_1.3.5.tar.gz' had non-zero exit status >
Aaron Lun (01:23:47): > i guess you’re pullingmaster, which would be BioC-devel
Aaron Lun (01:24:28): > it shows green for me on the devel build machines and on CI
Jayaram Kancherla (01:34:23): > alright POC - File (PNG): image.png
Aaron Lun (01:34:32): > holy shit
Aaron Lun (01:35:01): > very nice.
Jayaram Kancherla (01:35:21): > that wasn’t too bad, but now comes the real part of adding custom tracks, reactivity and receiving selections
Jayaram Kancherla (01:35:47) (in thread): > lets tag@hcorrada:slightly_smiling_face:
Aaron Lun (01:39:05): > yeah, see if you can get those in, and then we’ll see if anything needs to be modified in iSEE core to accommodate it.
2021-04-19
Federico Marini (06:27:46): > Wowzer browzer!
Kevin Rue-Albrecht (07:21:40): - File (PNG): Untitled-1.png
2021-05-07
Aaron Lun (20:48:39): > can someone show me what this renku thing is all about?
2021-05-08
Federico Marini (05:14:30): > Re: centralized place -iSEEhqas a repo to put all the “design & co”?
Kevin Rue-Albrecht (10:09:37) (in thread): > here’s a 2018 presentation to get you started, it gets more illustrated toward 10:40https://youtu.be/5kFhAAnv2D0?t=639 - Attachment (YouTube): Reproducible science with the Renku platform- Sandra Savchenko-de Jong (Swiss Data Science Center)
Kevin Rue-Albrecht (10:12:57) (in thread): > hold on, that’s more recenthttps://www.youtube.com/watch?v=8T-adH9HbJs&ab_channel=SIB-SwissInstituteofBioinformatics - Attachment (YouTube): Advanced Single Cell - Enabling Practical Reproducibility with RENKU platform
Kevin Rue-Albrecht (10:20:31) (in thread): > briefly: > * uses gitlab > * each repository builds a Docker image > * spins up a Docker container to work and commit > * renkuCLI can be used to build a knowledge graph (KG) which declares where input data comes from (data repositories, output of other renku repositories) and what the output files of this repository are (so that they can picked up by downstream projects, and so on)
Kevin Rue-Albrecht (10:22:15) (in thread): > Check out Charlotte’shttps://renkulab.io/projects/charlottesoneson/isee-dashboard-data/files/blob/create_datasets_code.shto see how she uses renku to create data sets
Kevin Rue-Albrecht (10:23:16) (in thread): > and myhttps://renkulab.io/projects/kevinrue67/isee-dashboard/files/blob/src/utils/release_datasets.shto see how I fetch data sets tagged with ‘iseedata’ (case-insensitive) in the knowledge graph to create a landing page that lists all of the available datasets for the iSEE dashboard
Aaron Lun (14:20:00) (in thread): > Oh wait, I still need to code if I’m an end-user? I though it was all point and click once it was set up.
Aaron Lun (14:20:39): > well, maybe something with a more explicit name, e.g., iSEE-logos.
Charlotte Soneson (15:38:12) (in thread): > No, it should be point and click. We set up the application, and the user chooses a dataset from a dropdown.
2021-05-09
Aaron Lun (14:32:24) (in thread): > is there a URL to the app?
2021-05-10
Kevin Rue-Albrecht (03:22:15) (in thread): > not yet - the Renku platform is working on setting up a Shiny Server - at the moment, you clone the repo with the app (mine), and your run the script named “app-release.R”, you’ll get the usual RStudio popup with the app
2021-05-11
Megha Lal (16:44:19): > @Megha Lal has joined the channel
2021-05-17
Federico Marini (12:04:11): > I was thinking of the “headquarters” idea to be also a kind of first place to check, as a syllabus
2021-05-20
Aaron Lun (23:49:40): > @Nitesh Turagado you reckon our azure agreement can run an instance of a shiny server? The idea would be to use iSEE as a front-end to explore datasets on ExperimentHub
Aaron Lun (23:50:37): > Minimal code is taken from?createLandingPagein ****iSEE****: > > # Alternative approach, to create a landing page > # that opens one of the datasets from the scRNAseq package. > library(scRNAseq) > all.data <- ls("package:scRNAseq") > all.data <- all.data[grep("Data$", all.data)] > > lpfun <- createLandingPage( > seUI=function(id) selectInput(id, "Dataset:", choices=all.data), > seLoad=function(x) get(x, as.environment("package:scRNAseq"))() > ) > > app <- iSEE(landingPage=lpfun) > shiny::runApp(app, port=1234) > > which will give access to all datasets in thescRNAseqpackage.
2021-05-21
Aaron Lun (16:53:47): > oops, wrong@Nitesh Turaga
Nitesh Turaga (17:24:46): > @Nitesh Turaga has joined the channel
Nitesh Turaga (17:26:03): > Hi@Aaron LunIt might be possible. How big an instance are we talking about ? We have a limited budget at the moment.
Nitesh Turaga (17:29:28): > Maybe, a few more details can help. Like, instance size, how many people we expect on there, etc….
Aaron Lun (17:59:58): > uh, let’s say, 8 GB instance, single core. Dunno how many people would be on it, kind of depends on how popular it becomes. I was thinking it could be a BioC-wide interface to inspect any EHub-derived SE without people having to boot up R themselves.
2021-05-23
Vince Carey (05:34:34): > We could start this off atshinyapps.io, where i have a paying account and some configurability. I tried your script@Aaron Lunand it ran, but for the first option (Aztekin) saw > > shiny::runApp(app, port=1234) > Loading required package: shiny > > Listening on[http://127.0.0.1:1234](http://127.0.0.1:1234)snapshotDate(): 2021-05-05 > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > loading from cache > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > loading from cache > Error in readRDS(x$datapath) : bad 'file' argument > Warning in .refineParameters(initial[[idx]], se) : > no valid y-axis 'rowData' fields for 'RowDataPlot' > Warning in .refineParameters(extra[[idx]], se) : > no valid y-axis 'rowData' fields for 'RowDataPlot' > Warning in .coerce_type(v, f, max_levels = max_levels, df = df) : > covariate has too many unique values, coercing to numeric > Warning in .coerce_type(v, f, max_levels = max_levels, df = df) : > covariate has too many unique values, coercing to numeric > snapshotDate(): 2021-05-05 > see ?scRNAseq and browseVignettes('scRNAseq') for documentation > downloading 1 resources >
Vince Carey (05:41:25): > for bach-mammary there were quite a few cache actions needed and a significant wait. i use shinytoastr to notify users about what is going in during long pauses. i didn’t see a way of going back to the landing page but maybe that is early in development. i just refreshed the browser and went back. it would be good for the iSEE app to tell the user what dataset it is looking at. i have a feeling that if i try to put it inshinyapps.ioas is, the errors i’ve noted (which also occurred with buettnerESC but not with bach-mammary) will be a problem. but l will give it a shot and report back.
Jonas Schulte-Schrepping (08:10:33): > @Jonas Schulte-Schrepping has joined the channel
Aaron Lun (15:19:00): > thex$datapatherror doesn’t really matter, it’ll still load.
Aaron Lun (15:20:09): > we could probably silence it better, but it doesn’t matter, it’ll justtryand move on.
Aaron Lun (15:21:14): > a loading screen can be prepared easily by modifying the landing page; this can be handled later once the deployment path is clear.
Aaron Lun (15:21:38): > similarly, an information panel can be easily created, e.g., usingiSEEu::MarkdownBoard.
2021-05-25
Peter Hickey (19:35:46): > is there any facility for creating annotations in iSEE a lahttps://chanzuckerberg.github.io/cellxgene/posts/annotations? If not, is it due to lack of interest thus far or difficulty/impossibility of implementing in the existing framework? - Attachment (cellxgene): annotations > Creating annotations
Aaron Lun (19:37:21): > kind of, the selection allows you to save selections, but the support there is pretty primitive; first in, last out.
Aaron Lun (19:38:00): > is it modifying the colData inside the app?
Peter Hickey (19:52:08): > cellxgene doesn’t modify the input (a h5ad file) but rather creates a CSV of the user-created annotations
Peter Hickey (19:52:34): > i haven’t thought through pros/cons of modifying the colData directly vs. separate annotation file(s)
2021-05-26
Kevin Rue-Albrecht (03:53:45): > the main issue about editing the colData in iSEE, is that we preprocess/inspects all the slots (assays, metadata, redDims) on the app startup, to cache some information frequently needed for decision-making in the app (e.g. which factors have too many levels and should be processed as numeric). > Adding more colData at runtime would require to update/add that cached information, which now that I write it out loud, might be possible (I can’t picture all the consequences right now).
Aaron Lun (11:29:57): > it would be a fairly big change and I’m not clear on the use-case. If people are adding annotations on the flyinside the running app instance, there must be a method for how they’re adding annotations, e.g., a lasso or a brush, and I would prefer to save that and give it a name or something. I can’t imagine people are going through and writing down, “oh yes, this cell is a B cell. And this next cell, that’s a monocyte.” etc. etc.
Peter Hickey (16:53:34): > yes, the cellxgene approach uses a lasso (unsure if it implements a brush approach, i.e. cluster 1 is B cell)
Aaron Lun (17:50:33): > i just realized that this could be easily accommodated by having the “save selections” work on a name rather than positional index. This would also permit a deletion scheme other than FILO.
2021-06-16
Chris Vanderaa (09:09:50): > @Chris Vanderaa has joined the channel
2021-07-10
Chris Vanderaa (13:17:08): > Hi there! I really love the app! I was wondering how you create the panel hierarchy overview (after clicking theExamine panel chart**** ****button)? From the source code, you seem to call to a function callediSEE_INTERNAL_link_graph_plotbut I can’t find the source code of that function. It would be interested to reuse this function to plot the hierarchical structure of the assays in aQFeaturesobject (extension ofMultiAssayExperiment)
Kevin Rue-Albrecht (13:18:36): > I think it’s to do with the igraph package.@Federico Marinimight remember better unless it was Aaron who did that part
Kevin Rue-Albrecht (13:23:34) (in thread): > There’s some code in utils_links.R to manage the graph > Still looking for the function that you mentioned for the modal
Kevin Rue-Albrecht (13:25:53) (in thread): > What you mentioned might be the reactive variable that we watch to trigger the event. I found the function “.snapshot_graph_linkedpanels” in utils_trackers.R which does the plotting
Kevin Rue-Albrecht (13:27:57) (in thread): > Found the variable herehttps://github.com/iSEE/iSEE/blob/70a793cc867a142deedd6e37131d23c9e4605a78/R/observers_general.R#L2
Kevin Rue-Albrecht (13:29:06) (in thread): > Which then links tohttps://github.com/iSEE/iSEE/blob/70a793cc867a142deedd6e37131d23c9e4605a78/R/observers_general.R#L112
Kevin Rue-Albrecht (13:29:47) (in thread): > And herehttps://github.com/iSEE/iSEE/blob/70a793cc867a142deedd6e37131d23c9e4605a78/R/outputs_general.R#L22
Kevin Rue-Albrecht (13:30:22) (in thread): > Sorry I can’t browse and write more extensively,I’m on my phone right now. Back at the office on Monday
Chris Vanderaa (13:30:57) (in thread): > Ah it’s already excellent! Thanks a lot!:pray:
Kevin Rue-Albrecht (13:31:39) (in thread): > Basically we build and maintain an igraph when users change the links between panels, and then we watch a button that triggers the plotting
2021-07-12
Federico Marini (03:42:22) (in thread): > very much what Kev said:wink:thanks!
2021-07-17
mariadermit (15:00:32): > @mariadermit has joined the channel
2021-08-17
Tanya Grancharova (19:15:49): > Hi, is it possible to add a custom button to the Additional information? I want to link to a glossary of terms used in my sample metadata and was wondering if it could be incorporated into the app itself via Additional information. - File (PNG): image.png
2021-08-18
Kevin Rue-Albrecht (07:24:48): > great idea! can you open an issue on the github repository please? that’ll help us track progress and act as a sticky note:slightly_smiling_face:https://github.com/iSEE/iSEE/issues/new/choose
Tanya Grancharova (13:05:51) (in thread): > thanks, I opened an issue
2021-09-08
Rene Welch (16:15:21): > @Rene Welch has joined the channel
2021-09-09
Julien Roux (01:58:02): > @Julien Roux has joined the channel
2021-09-25
Haichao Wang (07:21:00): > @Haichao Wang has joined the channel
2021-10-07
Aaron Lun (12:59:48): > someone should work with@Vince Careyto get an ISEE app deployed as an ExperimentHub browser
Vince Carey (13:30:24): > Right, if you have code that does this in a local shiny app just point me to it and I will see about publishing it atshinyapps.iofor which I am paying $39/mo and can get the necessary RAM
Kevin Rue-Albrecht (13:39:41): > I’m all booked up this month. Might be able to look into it Nov/Dec. I had a vision of a landing page where users could browse from the EHub and choose an object to load and explore in iSEE. Just never gathered the motivation driven by any particular demand of the feature yet.
Kevin Rue-Albrecht (13:41:24): > one other factor being the fact that most EHub objects come without dimred, which is normal of course, but don’t have that sought-after UMAP panel:stuck_out_tongue:
2021-10-21
Emily Collins (14:28:43): > @Emily Collins has joined the channel
2021-11-08
Paula Nieto García (03:27:45): > @Paula Nieto García has joined the channel
2021-11-22
Gavin Rhys Lloyd (03:28:31): > @Gavin Rhys Lloyd has joined the channel
2022-01-28
Megha Lal (11:13:26): > @Megha Lal has left the channel
2022-02-01
Stephanie Hicks (20:25:21): > @Stephanie Hicks has left the channel
2022-03-31
Leo Lahti (11:49:20): > @Leo Lahti has joined the channel
2022-04-01
Nitesh Turaga (13:56:24): > @Nitesh Turaga has left the channel
2022-07-13
Simón(e) Sun (12:12:13): > @Simón(e) Sun has joined the channel
2022-07-15
Ashley Robbins (15:20:30): > @Ashley Robbins has joined the channel
2022-08-30
Lisa Breckels (07:24:10): > @Lisa Breckels has joined the channel
2022-09-30
Ludwig Geistlinger (12:01:52): > @Ludwig Geistlinger has joined the channel
Tyrone Lee (12:02:29): > @Tyrone Lee has joined the channel
2022-10-06
Devika Agarwal (05:37:16): > @Devika Agarwal has joined the channel
2022-12-06
Kevin Rue-Albrecht (05:28:42): > It’s that time of the year again:joy: - File (PNG): image.png
Federico Marini (06:39:59): > Oh god the kid is already in kindergarten age
Federico Marini (06:40:14): > and it is raised truly multilingual
2022-12-07
Kevin Rue-Albrecht (16:11:18): > I just realise that I’ve mentioned it a lot of places but here. This wish/challenge/thought/nudge has been a reality for a while now:eyes:https://isee.github.io/iSEEhub/:innocent: - Attachment: Attachment > someone should work with @Vince Carey to get an ISEE app deployed as an ExperimentHub browser - Attachment (isee.github.io): iSEE for the Bioconductor ExperimentHub > This package defines a custom landing page for an iSEE app > interfacing with the Bioconductor ExperimentHub. The landing page allows > users to browse the ExperimentHub, select a data set, download and cache it, > and import it directly into a Bioconductor iSEE app.
Kevin Rue-Albrecht (16:14:14) (in thread): > maybe not as a live publicly hosted free-for-all web page, but at least as a package that anyone can install to run the app on their own machine:slightly_smiling_face:
2022-12-08
Federico Marini (09:30:26) (in thread): > well, there’s a running demo herehttp://shiny.imbei.uni-mainz.de:3838/iSEEhub-demo/
Federico Marini (09:30:46) (in thread): > which right now annoys me with some packages that are there but are seen as missing
Kevin Rue-Albrecht (09:30:47) (in thread): > ah right i forgot!
Federico Marini (09:31:06) (in thread): > i.e., the usability is quite cranked up right now:smile:
Federico Marini (09:31:22) (in thread): > but can be taken as a template to start with
Federico Marini (09:32:01) (in thread): > maybe I could simply put an example of it in the iSEE_instances repo
2022-12-09
Kevin Rue-Albrecht (12:02:55): > Hi all, > Not submitted to Bioconductor yet, but getting in a pretty good shape:https://isee.github.io/iSEEde/index.htmlLooking for beta-testers interested in adding some differential expression to their iSEE apps:slightly_smiling_face: - Attachment (isee.github.io): iSEE extension for panels related to differential expression analysis > This package contains diverse functionality to extend the usage > of the iSEE package, including additional classes for the panels or modes > facilitating the analysis of differential expression results.
Kevin Rue-Albrecht (12:03:29) (in thread): > I can’t get it to crash anymore. I need help:smile:
2023-03-10
Federico Marini (04:27:02): > iSEEu failing - is it because of the KeggREST service?
Federico Marini (04:29:01): > oh. they did change the names of things
Federico Marini (04:30:06): > I think it was starting withpath:map, now seems to be justmap, and that annoys our
Federico Marini (04:30:09): - File (PNG): image.png
Federico Marini (04:30:41): > is it worth making it more “generic” by saying “anything before the numbers”, or shall we just change thatsubcall?
Kevin Rue-Albrecht (06:38:15): > All things iSEE (and velociraptor) have failed recently. I was giving it some time to see if it’s something upstream that was temporarily failing - File (PNG): image.png
Kevin Rue-Albrecht (06:38:57): > here’s what I’m seeing today - File (PNG): image.png
Federico Marini (10:33:15): > it was a pandoc citeproc thing, it is now a thing to fix on our side
Federico Marini (10:33:16): > on it
Federico Marini (10:46:28): > fix is in as PR, if someone wants to check before pushing@Kevin Rue-Albrecht
Vince Carey (11:04:06) (in thread): > We ran into this with AnnotationDbi …https://github.com/Bioconductor/AnnotationDbi/pull/10#event-8691540995
Federico Marini (11:06:22) (in thread): > ok, took the chance to answer@Hervé Pagès’s curiosity then:smile:
Kevin Rue-Albrecht (11:18:24): > Thanks will do this evening I hope
Federico Marini (11:23:00) (in thread): > side thought: could be made stronger by a regexp capturing the expected pattern? but I am fine with the exact match sub
Vince Carey (11:25:36) (in thread): > It would be nice to have a connection to the KEGG folk that would help us anticipate changes of this kind. The change is IIRC indicated in the KEGG API release notes. We need to “watch” those with some intensity or we scramble to respond. Might they “revert” if enough people complained about the effects of this change? After all, why remove information that lots of code undoubtedly took for granted?
Federico Marini (11:26:05) (in thread): > :shrug:I’m on your line
2023-07-07
Kevin Rue-Albrecht (09:31:05): - Attachment: Attachment > :raised_back_of_hand: :microphone: :boom: https://rehwinkellab.shinyapps.io/ifnresource/ > First application of iSEEindex, just accepted a few days ago in Bioconductor. In the meantime, see here https://isee.github.io/iSEEindex/ :blush:
2023-07-27
Louise Morlot (07:35:40): > @Louise Morlot has joined the channel
2023-07-28
Benjamin Yang (15:57:45): > @Benjamin Yang has joined the channel
2023-08-02
Jamin Liu (14:44:16): > @Jamin Liu has joined the channel
2023-08-04
Ray Su (10:48:50): > @Ray Su has joined the channel
2023-08-29
Sean Davis (10:37:44): > Re: NSF/UKRI funding for iSEE - Attachment: Attachment > I’ll enquire to my Uni’s grant team about what it would take for me to get involved. > > I’d be keen to try my luck at this to fund further iSEE-related development, under “The implementation of, scaling of, or major improvements to research tools, products, and services for biology applicable to a wide range of researchers.” > > Since, it’s about allowing “U.S. and UK researchers to submit a single collaborative proposal that will undergo a single review process”, I’m happy to chat with whoever else would like to work on this together
Kevin Rue-Albrecht (12:01:26): > FYI. As a postdoc I can’t be principal investigator, but I can be co-investigator named in the grant with my group leader submitting for me. > Still, need to find US collaborators for this grant, any taker?:grimacing:
2023-09-15
Leo Lahti (04:55:09): > @Leo Lahti has joined the channel
2023-09-21
Giulio Benedetti (09:12:00): > @Giulio Benedetti has joined the channel
2023-09-25
Anna (05:43:09): > @Anna has joined the channel
2023-09-26
Anna (11:00:18): > I just found out this youtube channel and I love it, it explains everything super clearly, I shared it with my collaborators straigh away. Hope there will be more episodes soon!:slightly_smiling_face:https://www.youtube.com/watch?v=R72Tbb5vxmw - Attachment (YouTube): 1. Welcome to the iSEE-verse !
2023-09-27
Kevin Rue-Albrecht (04:05:08) (in thread): > Thanks!:heart_hands:I’ve recorded a sub-par episode 3, so it’s parked until I plan it better:wink:Meanwhile, I’ve started a new book (https://isee.github.io/iSEETheBook/) rather than fix the old one (https://isee.github.io/iSEE-book/) - Attachment (isee.github.io): Extending iSEE > This book describes how to use the Bioconductor iSEE package to create web-applications for exploring data stored in SummarizedExperiment objects.
2023-09-28
Kevin Rue-Albrecht (06:58:00) (in thread): > Heads up though: I’m focusing on the new book more than videos at the moment. Largely because my videos are scripted anyway (taking me several hours to plan and write each of them). It’s just more natural to write book chapters first, and then turn those into videos.
2023-09-29
Kevin Rue-Albrecht (14:06:57): > Hi all, > I’ve started to collect links to deployments of iSEE apps in the wild, in this READMEhttps://github.com/iSEE/awesome-isee-apps/blob/main/README.mdI would greatly appreciate if anyone here aware of any other deployment could either send inpull requestsor open anissuewith at least URLs, but optional any additional information about the contents of the app would be welcome too! > Thanks!
Kevin Rue-Albrecht (15:25:48) (in thread): > FWIW i’m sorting by version of iSEE (latest to oldest)
2023-10-11
Riccardo L. Rossi (03:49:09): > @Riccardo L. Rossi has joined the channel
2023-10-28
Giulio Benedetti (10:17:42): > Hi everyone! I’m happy to share with you thatiSEEtree(iSEE extension for TreeSummarizedExperiment) is in early stage of development, but its current version is already viable. Any feedback/contribution is welcome!
2023-10-30
Kevin Rue-Albrecht (09:54:21) (in thread): > I just managed to find the time and test the example of the man page?iSEE-TreeSEIt looks very nice!
Kevin Rue-Albrecht (09:56:37) (in thread): > I’ll need more time for a deeper study of the questions you listed athttps://github.com/iSEE/iSEE/issues/636#issuecomment-1778974771 - Attachment: Comment on #636 Extend iSEE visual toolkit to TreeSummarizedExperiment-specific features > Hi! > > I finally started working on this idea. An initial working package is available here: iSEEtree. For now, there is a new panel which shows a phylogenetic tree for the rows: RowTreePlot. > > As you can see from the issues, there are a few things about RowTreePlot that I’d like to improve before moving to other (possible) panels. It would be great if you could give me some hint how to go about the following issues: > > 1. Add RowTreePlot to default panels for TreeSummarizedExperiment objects > 2. RowTreePlot always coloured by rowData (it should be black when Node/Line color = None) > 3. RowTreePlot should also have a collapsible window for visual parameters > > Thank you!
Giulio Benedetti (09:58:46) (in thread): > Thank you! I found a quick fix for those issues, but still a general check from your side would be valuable
Kevin Rue-Albrecht (16:35:28) (in thread): > My first reaction (I’m spoiled) was to test if the brush functionality was supported (sadly no, if anyone wonders). I totally understand why it doesn’t make sense for some panels (e.g. circular tree). Do you think there is any reasonable use case that would motivate the implementation of the brush?
Giulio Benedetti (18:14:00) (in thread): > Thank you for the point! The package is still missing any form of data selection, but there are certainly some places where it could make sense to brush data for example from a reduced dimension
2023-11-28
Kevin Rue-Albrecht (16:43:18): > iSEE is now on LinkedIn:blush: - File (JPEG): File
2024-03-11
Federico Marini (12:10:46): > reviving this a bit -https://github.com/iSEE/iSEEu/actions/runs/8235700412/job/22520416495
Federico Marini (12:10:56): > pending PR for some extra documentation in iSEEu
2024-03-12
Kevin Rue-Albrecht (07:56:20): > OK - thanks - teaching right now, but let me know whether you want a thorough review of the PR or a basic nod of approval. > If it’s just documentation, I don’t mind merging and releasing without a thorough read of the PR:wink:
2024-03-14
Federico Marini (11:14:41) (in thread): > more the nod of approval. no breaking changes are in, just some description in the vignette
2024-04-18
Weston Elison (15:54:24): > @Weston Elison has joined the channel
2024-05-13
Kevin Rue-Albrecht (10:40:38): > Aside from iSEEtree, is anyone here working on any iSEE extension? I’ve lost track of fellow developers contacting us to help them in their endeavours. Not that there were that many, but I haven’t kept a list of related projects out there:slightly_smiling_face:
Aaron Lun (10:44:35): > that reminds me; feel free to rename the volcano plot etc. all our internal shiny apps will be refactored anyway
Kevin Rue-Albrecht (10:46:59): > Does that mean you won’t use it anymore? In which case, I would deprecate it in favour of my version in iSEEde, rather than maintain the two of them
Aaron Lun (10:49:57): > yes, go ahead
Aaron Lun (10:50:15): > [Unsupported block type: image] > > [Unsupported block type: context]
2024-05-15
Sunil Nahata (08:31:00): > @Sunil Nahata has joined the channel
2024-05-30
Tuomas Borman (01:56:37): > @Tuomas Borman has joined the channel
2024-08-07
Giulio Benedetti (09:42:47): > iSEEtree is on Bioconductor devel now:https://bioconductor.org/packages/devel/bioc/html/iSEEtree.html - Attachment (Bioconductor): iSEEtree (development version) > iSEEtree is an extension of iSEE for the TreeSummarizedExperiment. It leverages the functionality from the miaViz package for microbiome data visualisation to create panels that are specific for TreeSummarizedExperiment objects. Not surprisingly, it also depends on the generic panels from iSEE.
2024-08-19
Rema Gesaka (09:41:01): > @Rema Gesaka has joined the channel
2024-09-19
Aaron Lun (17:52:15): > note that if you change the class definitions in iSEE you need to rebump packages that define derived subclasses, as the S4 class definitions are baked in at install time; this means that someone updating e.g. iSEE but not iSEEu will get failures like > > Warning: Error in .nextMethod: invalid names for slots of class “VolcanoPlot”: FixAspectRatio, ViolinAdd > 1: shiny::runApp > > I assume that this is in part due tohttps://github.com/iSEE/iSEE/commit/b9c22da13ab4791446332f304c7be2827828d360
2024-10-01
Kevin Rue-Albrecht (10:13:36) (in thread): > Did I misunderstand? > Why am I getting the error AFTER the version bump?https://github.com/iSEE/iSEEu/actions/runs/11126533591/job/30916671660?pr=119#step:22:398
Kevin Rue-Albrecht (10:14:12) (in thread): > well at least Windows is happyhttps://github.com/iSEE/iSEEu/actions/runs/11126533591/job/30916672688?pr=119
Kevin Rue-Albrecht (10:22:15) (in thread): > nevermind - it’s because in this instance, iSEEu is calling the iSEEhex function which hasn’t propagated the version bump yet, exactly the error you reported in the first place
2025-02-05
Dominik Burri (07:28:59): > @Dominik Burri has joined the channel
2025-02-10
Tyrone Lee (13:30:07): > Hi all, > We’re running into an issue in which one of our reduced dimension plots has a lot of cell types and its forcing our categorical labels into numeric ones. > > Warning in .coerce_type(v, f, max_levels = max_levels, df = df) : > covariate has too many unique values, coercing to numeric >
Tyrone Lee (13:33:01): > The result are dimension reduction plots with numeric values instead of types (see theReduced dimension plot 1). Our collaborator wishes to remedy this for another app. Is there a method to override this check in plotting.R or does anyone know of a good workaround? - File (PNG): image.png
Charlotte Soneson (13:41:04) (in thread): > You could useregisterAppOptionsto increase the threshold - e.g. > > sce <- registerAppOptions(sce, list(color.maxlevels = 84)) >
2025-02-12
Ario Moniri (08:11:13): > @Ario Moniri has joined the channel
2025-02-20
Tyrone Lee (14:17:49) (in thread): > Another follow-up, > We managed to get the numeric issue worked but now there’s a problem with the legends being very dense. To try and just remove the legend I thought we could simply use the option for the panelLegendPosition = "none"but that only accepts “Right” and “Below”. - File (PNG): image.png
2025-02-25
Tyrone Lee (13:49:56): > Is there a way to remove or suppress legends fromdotplotclass panels? It seems the option only takes right and bottom but not LegendPosition = “none”
2025-02-26
Kevin Rue-Albrecht (05:57:17) (in thread): > That seems like an easy tweak (adding that one option). > Can you please open an issue on theiSEE/iSEErepo to keep track of it?
Tyrone Lee (11:14:39) (in thread): > Made the issue, thanks KevinAnother option we could try is making a custom panel that inherits everything but the setting from reducedDimsPanel but that might take up more work than a patch for the class itself. - Attachment: #688 Remove or suppress legends from dotplot class panels > Right now the dotplot class has an option that accepts only 2 options ( “Bottom”,“Right”) for LegendPosition. We have a case in one of our apps that the resulting ReducedDimPlot has over 50 discrete labels; this makes the legend take up too much of the space. We would like to be able to use LegendPosition = none which ggplot will simply not create the legend.
Federico Marini (12:36:05) (in thread): > I did see this now, and give ourselves a pointer on how to proceed
Federico Marini (12:36:17) (in thread): > might need to simply add one option AND check it is picked up correctly
Tyrone Lee (15:17:42) (in thread): > wow. that was a nasty slack outage. I think the checks still passed on the pull request.
Kevin Rue-Albrecht (15:23:15) (in thread): > Wowdidn’tthink Fede was so eager to tackle that one:smile:
Kevin Rue-Albrecht (15:23:40) (in thread): > But I agree, should be one extra option and a switch in the code
2025-02-27
Federico Marini (12:10:05) (in thread): > I was faster than slack
Federico Marini (12:10:09) (in thread): > (I think)
2025-03-17
Giulio Benedetti (10:34:57): > Hi! Does iSEE have a limit size for the input SummarizedExperiment object? I vaguely remember seeing a warning some time ago, but recently it hasn’t appeared.
Charlotte Soneson (11:04:25) (in thread): > There is a limit (that can be increased) for uploading a SummarizedExperiment object (e.g. if launching withiSEE()only), but apart from that I don’t think there are limits.
2025-03-18
Federico Marini (11:05:39) (in thread): > IIRC not iSEE but shiny has one
Federico Marini (11:06:03) (in thread): > So in the end you could simply set the correspondingoptions(...)item for that
2025-03-20
Kevin Rue-Albrecht (05:10:19): > Hi good people! > Just to put it out there that I’ve just spotted this funding opportunity and that I’ll be looking into it asaphttps://www.software.ac.uk/research-software-maintenance-fund/round-1Similar to the CZI EOSS where we passed the pre-selection but failed the full application, this one also focuses on software thatalready has an established community of users, so… keep those downloads up!:stuck_out_tongue:More seriously, you can get involved by: > * adding any public deployment of iSEE app you have / know about onhttps://isee.github.io/apps.html > * getting in touch if you’re interested in collaborating. I’ve only just started looking at the information, but I’m open to any ideas that might strengthen the grant, which in turn would be mutually beneficial to fund dedicated time on the project > * keeping in touch with any other idea! - Attachment (software.ac.uk): Research Software Maintenance Fund - Round 1 | Software Sustainability Institute > The first round of the Software Sustainability Institute Research Software Maintenance Fund will take place from April to September 2025. - Attachment (software.ac.uk): UKRI awards the Software Sustainability Institute £4.8m to strengthen research software maintenance in the UK | Software Sustainability Institute > The SSI has launched the Research Software Maintenance Fund, a new funding initiative offering £4.8 million to support existing research software.
2025-04-18
Kevin Rue-Albrecht (12:29:09): > @Giulio BenedettimiaDash made it into Stephen Turner’s blog post on the latest R/BioC release:wink:https://www.r-bloggers.com/2025/04/repost-r-4-5-0-and-bioconductor-3-21/?utm_source=ph[…]medium=email&utm_campaign=R-bloggers-daily&utm_content=HTML - Attachment (R-bloggers): Repost: R 4.5.0 and Bioconductor 3.21 | R-bloggers > Reposted from the original at https://blog.stephenturner.us/p/r-450-bioconductor-321.Faster package installation, import only the functions you want with use(), built-in Palmer penguins data, grep values shortcut, and lots of new bioinformatics pa…
Bioconductor Slack Export Bioconductor Slack Export Home /index.html General /general.html Random /random.html Introductions /introductions.html Jobs /jobs.html Papers & Preprints /papersandpreprints.html Welcome Material /FCF05URJ0P021Welcome_material.html Development Developers Forum /developers-forum.html Bioc Git /bioc_git.html Bioc Builds /bioc-builds.html Package Submissions /package_submissions.html R7 Object System /r7.html GitHub Actions /github-actions.html WASM /wasm.html CPP /cpp.html GPUs /gpus.html Cloud WG /cloud-working-group.html Cloud GitHub Actions Subgroup /cloud-github-actions-subgroup.html Containers /containers.html Singularity /singularity.html ARM Linux /arm-linux.html BiocCheck-a-thon /bioccheck-a-thon.html BiocVerse /biocverse.html Bioc2u /bioc2u.html GitDevTest /gitdevtest.html New Packages (Archived?) /new_packages.html Null Ranges /nullranges.html OMA Developers /oma-developers.html Other Distros /other-distros.html Package Submission (Old?) /package-submission.html Palm Tree /palmtree.html RCWL /rcwl.html RHDF5 Client /rhdf5client.html RowDataFix /rowdatafix.html Conferences & Events BioC Workshops /biocworkshops.html BioC 2018 /bioc2018.html BioC 2018 Workshops /bioc2018-workshops.html BioC 2019 /bioc2019.html BioC 2020 /bioc2020.html BioC 2020 Annotation Wksh /bioc2020-bioconductor-100_introduction-to-bioconductor-annotation-resources.html BioC 2020 BoF 10sr /bioc2020-bof-10sr.html BioC 2020 Workshops /bioc2020-workshops.html BioC 2021 /bioc2021.html BioC 2021 Microbiome Wksh /microbiome_experiment_bioc2021.html BioC 2022 /bioc2022.html BioC 2024 /bioc2024.html BioC Conf Everyone /bioc-conference-everyone.html BioC Conf Non-Competing /bioc-conference-noncompeting.html BioC Conf Sponsorship /bioc-conference-sponsorship.html BioC Conf Web /bioc-conference-web.html BioCAsia /biocasia.html BioCAsia 2020 /biocasia2020.html BioCAsia 2021 /biocasia2021.html BioCAsia 2022 /biocasia2022.html BioCAsia 2023 /biocasia2023.html Calendar Events /calendar-events.html CSAMA 2023 /csama2023.html EuroBioC 2019 /eurobioc2019.html EuroBioC 2020 /eurobioc2020.html EuroBioC 2020 Chairs /eurobioc2020-chairs.html EuroBioC 2020 iSEE Transcripts /eurobioc2020-isee-transcripts.html EuroBioC Everyone /eurobioc-conference-everyone.html Events /events.html Hacktoberfest /hacktoberfest.html IBT2025 KushCentreGroup1 /ibt2025-kushcentregroup1.html ISMB OSCA /ismb-osca.html Meetups /meetups.html Community & Outreach Accessible Vis /accessible-vis.html Beginners & Schools /beginners_and_schools.html Bioc Africa /bioc_africa.html Carpentries /carpentries.html Community Advisory Board /community-advisory-board.html Developers Mentorship /developers-mentorship.html DiverseBioc /diversebioc.html Education & Training /education-and-training.html Humble Brag /humble-brag.html Mentorship Program Microstasis /mentorship_program_microstasis.html Outreachy /outreachy.html Queer Bioc /queer-bioc.html R-Ladies /r-ladies.html Social Media /social-media.html Support Site /support-site.html Technical Advisory Board /tech-advisory-board.html Projects & Topics Alpha Missense /alpha-missense.html Annotation Roadmap /annotation-roadmap.html Anvil /anvil.html ArtifactDB /artifactdb.html Big Data Rep /bigdata-rep.html BioC Books /biocbooks.html BioC Challenges /biocchallenges.html BioC Classes /biocclasses.html BioC Connect /bioc-connect.html BioC Hubs /biochubs.html BioC Python /biocpython.html BioC Website /biocwebsite.html BugSigDB /bugsigdb.html Cancer Data /cancerdata.html Cancer Related /cancer-related.html cBioPortal Data R /cbioportal-data-r.html Channel Requests /channel-requests.html Clinical /clinical.html Cool Vis /cool-vis.html Cyto Spec Book /cyto-spec-book.html Database /database.html Delayed Array /delayed_array.html Doc Translation /documentation-translation.html EasyLift /easylift.html EasyPlot /easyplot.html EDAM Collaboration /edam-collaboration.html Elixir /elixir.html Funding /funding.html GenAI Bioc /genai-bioc.html GSEABase /gseabase.html GWAS Data /gwas-data.html Hash Multi Species /hash-multi-species.html HCA Clustering /hca_clustering.html HCA Data Insights /hca-data-insights.html HCA RFA /hca_rfa.html Histopathology Image Analysis /histopathology-image-analysis.html HuBMAPr /hubmapr.html Image Analysis /image-analysis.html Industry /industry.html iSEE /isee.html LLMs for Bioc /llms-for-bioc.html Long Read Study Group /long-read-study-group.html Longread /longread.html Machine Learning /machine-learning.html MAE Discussion /mae-discussion.html MAE Updates /mae-updates.html Mass Spectrometry /mass-spectrometry.html Metabolomics /metabolomics.html MiaVerse /miaverse.html MiaVerse Admins /miaverse-admins.html Microbiome/Metagenome /microbiome_metagenome.html Microsoft Azure Collaboration /microsoft-azure-collaboration.html MolEvolvR /molevolvr.html Multiomics /multiomics.html OmicIdx /omicidx.html OSCA Book /osca-book.html OSCA Book Club /osca-book-club.html Pangenome /pangenome.html Pharmacogenomics /pharmacogenomics.html Proteomics /proteomics.html Ragged Experiment /raggedexperiment.html Reproducibility /reproducibility.html Salmon2Bioc /salmon2bioc.html Scalability /scalability.html SC Batch Correction /sc-batch-correction.html SC Repertoires /sc-repertoires.html SC Signature /sc-signature.html SC Velocity /sc-velocity.html Shiny /shiny.html Single Cell Experiment /singlecellexperiment.html Single Cell Queries /singlecell-queries.html Spatial /spatial.html Spatial Data Devel /spatialdata-devel.html Spatial Experiment /spatialexperiment.html Spatial Experiment Devel /spatialexperiment-devel.html Spatial Interoperability /spatial-interoperability.html Sweave2Rmd /sweave2rmd.html Terra /terra.html TeSS Integration /tess-integration.html Tidiness in Bioc /tidiness_in_bioc.html Translation FR /translation-fr.html Tree-like SE /tree-like-se.html Voyager /voyager.html Workflows /workflows.html Zarr /zarr.html
#isee – Bioconductor Slack Export #isee – Bioconductor Slack Export #isee – Bioconductor Slack Export Bioconductor Slack Export
http://r.research.att.com↩︎
https://github.com/seandavi/Bioc2019-iSEE-workshop/commits/master↩︎
