Skip to content

Conversation

@dimitri-yatsenko
Copy link
Member

@dimitri-yatsenko dimitri-yatsenko commented Jan 24, 2026

Summary

Code refactoring for better organization and import efficiency. No functional changes.

  • Extract VirtualModule to separate virtual_module.py (from schemas.py)
  • Split migrate.py into a package with submodules for each migration version
  • Extract table operations into mixins (table_insert.py, table_delete.py)
  • Lazy-load migrate module to improve import time (heavy pandas dependency)
  • Add unit tests for heading, condition, and declare modules (+101 tests)

Motivation

  • Reduce cognitive load by splitting large files into focused modules
  • Improve import performance by deferring heavy dependencies
  • Better test coverage for core parsing and data structure modules

Test plan

  • All existing tests pass (846 passed, 4 skipped)
  • New unit tests for heading.py, condition.py, declare.py
  • Lazy import tests verify deferred loading

🤖 Generated with Claude Code

Move imports inside methods where they're used to avoid loading
these dependencies at module import time. This speeds up `import datajoint`
for users who don't call populate().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions github-actions bot added the enhancement Indicates new improvements label Jan 24, 2026
Dead code - class was defined but never instantiated anywhere.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dimitri-yatsenko dimitri-yatsenko changed the title perf: lazy-load deepdiff and tqdm in autopopulate perf: lazy-load autopopulate deps + remove dead code Jan 24, 2026
@dimitri-yatsenko dimitri-yatsenko marked this pull request as draft January 24, 2026 16:27
Split the 1,286-line builtin_codecs.py into a builtin_codecs/ package
with separate modules for each codec:

- blob.py: BlobCodec (Python object serialization)
- hash.py: HashCodec (hash-addressed storage)
- schema.py: SchemaCodec (base class for schema-addressed codecs)
- object.py: ObjectCodec (files/folders)
- attach.py: AttachCodec (file attachments)
- filepath.py: FilepathCodec (references to existing files)
- npy.py: NpyCodec + NpyRef (numpy arrays)

Benefits:
- Each codec is now self-contained and easier to understand
- Easier to maintain and test individual codecs
- Clearer organization for contributors
- No change to public API (same imports work)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dimitri-yatsenko dimitri-yatsenko changed the title perf: lazy-load autopopulate deps + remove dead code refactor: code cleanup and organization improvements Jan 24, 2026
@dimitri-yatsenko dimitri-yatsenko changed the title refactor: code cleanup and organization improvements refactor: code organization and lazy imports for better modularity Jan 24, 2026
@dimitri-yatsenko dimitri-yatsenko marked this pull request as ready for review January 24, 2026 18:23
@dimitri-yatsenko dimitri-yatsenko merged commit 68300f2 into pre/v2.1 Jan 24, 2026
7 of 8 checks passed
@dimitri-yatsenko dimitri-yatsenko deleted the perf/lazy-load-autopopulate branch January 24, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Indicates new improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants