⚠️ This repository is archived and no longer maintained. Workshop materials from 2022. For current DataJoint development, see datajoint-python.
The DataJoint team and the Johns Hopkins University Applied Physics Lab will hold an open brainstorm workshop to define the roadmap for automated research workflows in neuroscience. We will review the current landscape critically and define paths for implementing effective research workflows in neuroscience.
- JHU/APL: Daniel Xenes, Erik Johnson
- Columbia University: Taiga Abe, John Cunningham
- Berlin Institute of Health, Charité University EBRAINS (remote): Michael Schirner, Petra Ritter
- University of Utah, Visus: Amy Gooch, Valerio Pascucci
- Catalyst Neuro: Ben Dichter
- Ontologic: Dallas McCall
- Allen Institute - Neural Dynamics (remote): David Feng
- DataJoint: Thinh Nguyen, Raphael Guzman, Monty Kosma, Kabi Gunalan, Joseph Burling, Chris Brozdowski, Tolga Dincer, Sidharth Hulyalkar, Jaerong Ahn, Kushal Bakshi, Monty Kosma, Anu Pansare, Dimitri Yatsenko
The workshop will be held at DataJoint in Houston, TX.
Address: 4265 San Felipe St, Ste 1025 (10th Floor), Houston, TX 77027. Plenty of parking is available in the garage attached on the south side of the building.
Aim:
- to vastly increase the speed, scale, and validity of neuroscience research
- to provide a clear competitive advantage to researchers
- time to launch
- high-performance computing
- latest analysis tools
- ability to collaborate
- time to publish
Approach: Implement Research Workflow Automation
- reduce barriers to participation / collaboration
- expand capabilities -> tackle new problems
- ease access to cyberinfrastructure
- transparency and reproducibility
- integration of diverse tools and resources
- proper incentives and credit assignment
What is needed to implement automated workflows in neuroscience
- framework + tools = a language for formal workflows = DataJoint Core
- reference implementations + integrations = DataJoint Elements
- community + support + resources + services = DataJoint Works
Breakfast and lunch will be provided.
The workshop will identify general directions for addressing the challenges in team science and data-driven research workflows.
Day 1 and Day 2 will map the current trends and challenges in the field.
This session will be open to interested researchers -- please notify Dimitri Yatsenko August 31, 2022 if you would like to attend.
Day 3 will focus on the roadmap for the emerging DataJoint platform and will be limited to the DataJoint and JHU/APL teams.
7.30 am - Breakfast
8 am - Automated Workflows for Neuroscience - Dimitri
9 am - Current Challanges - Erik, Dimitri, Thinh
- Major friction points in data-driven research
- The structure of typical vs highly successful projects
- Advanced neurotechnologies and multi-modal experiments
- Data acquisition and aggregation
- Team organization, collaboration process
- IT Infrastructure - Use of cloud
- Collaborative analysis
- Credit assignment
- Human in the loop -- curation
- AI in the loop
- Reproducibility, integrity, continuity
- Budgeting and cost controls
10 am - Virtual Research Environments - Michael Schirner
- The Virtual Brain – EBRAINS Cloud woorkflows
- Virtual Research Environment (VRE) for research on sensitive health data at Charité Berlin
11 am - BRAIN Initiative Neuroinformatics Developments - Erik
- A critical review of the current and emerging neuroinformatics initiatives
- Global efforts - INCF
12 noon - lunch
1 pm - Visus. Data streaming, HPC infrastructure - Valerio
2 pm - collaborative analysis tools, platforms, and environments. DANDI, PanNeuro - Ben ,Erik
- PanNeuro - Ben
3 pm - ML integration. Human-in-the loop - Dimitri, Erik
7.30 am breakfast
8 am: An overview of Formal Workflow management tools - Raphael, Erik
- Prefect, Apache Airflow, Flyte, Argo -> Common Workflow Language
Most widely used bioinformatics workflow systems (from Reiter-2020):
| Workflow system | Documentation | Example workflow | Tutorial |
|---|---|---|---|
| Snakemake | Docs | ChipSeq | Tutorial |
| Nextflow | Docs | Sarek | Get Started |
| CWL | commonwl.org | EBI-Metagenomics | User Guide Example |
| WDL | openwdl.org | gatk4 | HOWTO |
9 am Infrastructure access, orchestration - Erik, Raphael, Joseph, Taiga
- NeuroCaas - Taiga Abe
- Job orchestration - Raphael
10 am Data Management, the role of databases - Dimitri, David Feng
- workflows + data management
- data models: structured vs self-describing data
- data consistency, integrity
- query speed,indexing
- Data lakes, workflows/structure: Benchling, Code Ocean
11 am - Accessibility, provenance, versioning - Erik, Ben, Dimitri, David Feng
- association between code, data
- Joint management of code, environment, and data
12 noon - lunch
1 pm - Neuroinformatics resources - Erik, Dimitri, Ben
- Allen and Janelia atlases
- Catalogs, nomenclatures, ontologies
- Data standards, archives, public data repositories and databases.
- Analysis tools
- Collaboration platforms
2 pm - The DataJoint experience - Thinh, Dimitri
- DataJoint Core - differentiators, performance, gap analysis
- DataJoint Elements - User experience
- DataJoint Works - user experience
- Interfaces: SciViz, LabBook, Codebook
- Platform for tool developers - dissemination, tracking, credit assignment
3 pm - Teamflows - Erik, Thinh, Dimitri
- Roles, team structure
- Software Engineering in Research
- incentives, credit assignment
- informal collaborations
- integration of community development
- funding, support
4 pm - Online research platforms - Erik
- Examples: Galaxy, HubMap
7.30 am - breakfast
8 am - DataJoint Platform Workshop -- internal. DataJoint and JHU APL only.
- Aims for DataJoint SciOps
- Key differentiators
- SciOps user experience
- Framework development
- Integration of DataJoint Elements
- Integrations - acquisition software, atlases, archives
- Aim 1
- Aim 2
- Aim 3
- Milestones, Statement of work, Roles and responsibilities
1 pm - lunch, adjourn