-
-
Notifications
You must be signed in to change notification settings - Fork 34k
gh-142518: Document thread-safety guarantees of dict operations #144184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5508,6 +5508,146 @@ can be used interchangeably to index the same dictionary entry. | |
| .. versionchanged:: 3.8 | ||
| Dictionaries are now reversible. | ||
|
|
||
| .. admonition:: Thread safety | ||
|
|
||
| Creating a dictionary with the :class:`dict` constructor is atomic when the | ||
| argument to it is a :class:`dict` or a :class:`tuple`. When using the | ||
| :meth:`dict.fromkeys` method, dictionary creation is atomic when the | ||
| argument is a :class:`dict`, :class:`tuple`, :class:`set` or | ||
| :class:`frozenset`. | ||
|
|
||
| The following operations and function are lock-free and | ||
| :term:`atomic <atomic operation>`. | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| d[key] # dict.__getitem__ | ||
| d.get(key) # dict.get | ||
| key in d # dict.__contains__ | ||
| len(d) # dict.__len__ | ||
|
|
||
| These operations may compare keys using :meth:`~object.__eq__`, which can | ||
| execute arbitrary Python code. During such comparisons, the dictionary may | ||
| be modified by another thread. For built-in types like :class:`str`, | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IIRC devguide recommenda avoiding double spaces after periods for new documentation but I am not entirely sure. |
||
| :class:`int`, and :class:`float`, that implement :meth:`~object.__eq__` in C, | ||
| the underlying lock is not released during comparisons and this is not a | ||
| concern. | ||
|
|
||
| All other operations from here on hold the per-object lock. | ||
|
|
||
| Writing or removing a single item is safe to call from multiple threads | ||
| and will not corrupt the dictionary: | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| d[key] = value # write | ||
| del d[key] # delete | ||
| d.pop(key) # remove and return | ||
| d.popitem() # remove and return last item | ||
| d.setdefault(key, v) # insert if missing | ||
|
|
||
| These operations also compare keys, so the same :meth:`~object.__eq__` | ||
| considerations as above apply. | ||
|
|
||
| The following operations return new objects and hold the per-object lock | ||
| for the duration: | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| d.copy() # returns a shallow copy of the dictionary | ||
| d | other # merges two dicts into a new dict | ||
| d.keys() # returns a new dict_keys view object | ||
| d.values() # returns a new dict_values view object | ||
| d.items() # returns a new dict_items view object | ||
|
|
||
| The :meth:`~dict.clear` method holds the lock for its duration. Other | ||
| threads cannot observe elements being removed. | ||
|
|
||
| The following operations lock both dictionaries. For :meth:`~dict.update` | ||
| and ``|=``, this applies only when the other operand is a :class:`dict` | ||
| that uses the standard dict iterator (but not subclasses that override | ||
| iteration). For equality comparison, this applies to :class:`dict` and | ||
| its subclasses: | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| d.update(other_dict) # both locked when other_dict is a dict | ||
| d |= other_dict # both locked when other_dict is a dict | ||
| d == other_dict # both locked for dict and subclasses | ||
|
|
||
| The equality comparison also compares values using :meth:`~object.__eq__`, | ||
| so for non-built-in types the lock may be released during comparison. | ||
|
|
||
| :meth:`~dict.fromkeys` locks both the new dictionary and the iterable | ||
| when the iterable is exactly a :class:`dict`, :class:`set`, or | ||
| :class:`frozenset` (not subclasses): | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| dict.fromkeys(a_dict) # locks both | ||
| dict.fromkeys(a_set) # locks both | ||
| dict.fromkeys(a_frozenset) # locks both | ||
|
|
||
| When updating from a non-dict iterable, only the target dictionary is | ||
| locked. The iterable may be concurrently modified by another thread: | ||
|
|
||
| .. code-block:: | ||
| :class: maybe | ||
|
|
||
| d.update(iterable) # iterable is not a dict | ||
| d |= iterable # iterable is not a dict | ||
| dict.fromkeys(iterable) # iterable is not a dict/set/frozenset | ||
|
|
||
| Operations that involve multiple accesses, as well as iteration, are never | ||
| atomic: | ||
|
|
||
| .. code-block:: | ||
| :class: bad | ||
|
|
||
| # NOT atomic: read-modify-write | ||
| d[key] = d[key] + 1 | ||
|
|
||
| # NOT atomic: check-then-act (TOCTOU) | ||
| if key in d: | ||
| del d[key] | ||
|
|
||
| # NOT thread-safe: iteration while modifying | ||
| for key, value in d.items(): | ||
| process(key) # another thread may modify d | ||
|
|
||
| To avoid time-of-check to time-of-use (TOCTOU) issues, use atomic | ||
| operations or handle exceptions: | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| # Use pop() with default instead of check-then-delete | ||
| d.pop(key, None) | ||
|
|
||
| # Or handle the exception | ||
| try: | ||
| del d[key] | ||
| except KeyError: | ||
| pass | ||
|
|
||
| To safely iterate over a dictionary that may be modified by another | ||
| thread, iterate over a copy: | ||
|
|
||
| .. code-block:: | ||
| :class: good | ||
|
|
||
| # Make a copy to iterate safely | ||
| for key, value in d.copy().items(): | ||
| process(key) | ||
|
|
||
| Consider external synchronization when sharing :class:`dict` instances | ||
| across threads. See :ref:`freethreading-python-howto` for more information. | ||
|
|
||
|
|
||
| .. seealso:: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would suggest having this before the thread safety. |
||
| :class:`types.MappingProxyType` can be used to create a read-only view | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I
Admonition will create a big box which will not be good considering the size of the text. A rubric is like a title but without being it in the sidebar and a label allows us to link it.
Alternatively we can have a real section.