Repository navigation
fix: skip matched extensions in match_unmapped_resources - #2339
Open
AradhyaTiwari10 wants to merge 1 commit into
Open
AradhyaTiwari10 wants to merge 1 commit into
AradhyaTiwari10 wants to merge 1 commit into
Conversation
…g#2338 The result of to_unmapped.exclude(extension__in=matched_extensions) was discarded, so files with an extension already looked up in PurlDB were sent to PurlDB a second time. Signed-off-by: Aradhya Tiwari <82563152+AradhyaTiwari10@users.noreply.github.com>
tdruez
marked this pull request as ready for review
November 17, 2026 08:08
tdruez
reviewed
Nov 17, 2026
tdruez
left a comment
Contributor
There was a problem hiding this comment.
@AradhyaTiwari10 Looks good, see my two minor comments on the test.
| self.assertEqual(flag.IGNORED_MEDIA_FILE, media_file.status) | ||
|
|
||
| @mock.patch("scanpipe.pipes.d2d._match_purldb_resources") | ||
| def test_match_unmapped_resources_skips_matched_extensions( |
Contributor
There was a problem hiding this comment.
Test names in this module use the test_scanpipe_pipes_d2d_ prefix.
Suggested change
| def test_match_unmapped_resources_skips_matched_extensions( | |
| def test_scanpipe_pipes_d2d_match_unmapped_resources_skips_matched_extensions( |
Comment on lines
+1518
to
+1522
| to_resources = mock_match_purldb_resources.call_args.kwargs["to_resources"] | ||
| self.assertEqual( | ||
| ["to/project.tar.zst-extract/expected.json"], | ||
| [resource.path for resource in to_resources], | ||
| ) |
Contributor
There was a problem hiding this comment.
This works only because if to_unmapped: evaluated and cached the QuerySet before the call. Re-evaluated, it would be empty since all resources are flagged at this point, so a switch to .exists() in the pipe would break this test for the wrong reason. The log assertion above already covers the exclusion, I'd drop this one.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issues
Changes
match_unmapped_resources()calledto_unmapped.exclude(extension__in=matched_extensions)without assigning the result, so thematched_extensionsfilter was a no-op and files already looked up in PurlDB bymatch_resources_to_purldb(for example.classfiles) were sent to PurlDB again. The result is now assigned back toto_unmapped.Those files still end up with the
requires-reviewstatus at the end of the step, as before; only the second PurlDB lookup is skipped.Added
test_match_unmapped_resources_skips_matched_extensions, which fails onmain(Mapping 2 to/ resources ...) and passes with the fix.manage.py test scanpipe.tests.pipes.test_d2d scanpipe.tests.test_pipelines: 186 tests OK.ruff checkandruff format --checkpass.AI usage: I used Claude Code to help locate the bug and write the fix and the test; I reproduced the failure and ran the tests locally.
Checklist