Repository navigation
FIX: Handle seed prompts with metadata=None in SeedGroup - #3509
Open
ni (nniiovoo) wants to merge 1 commit into
Open
ni (nniiovoo) wants to merge 1 commit into
ni (nniiovoo) wants to merge 1 commit into
Conversation
SeedPrompt.metadata can be None (the PKU-SafeRLHF loader sets it for prompts without a harm category), but _prompts_to_messages passed it to MessagePiece(prompt_metadata=...), which requires a dict. Building next_message, user_messages or prepended_conversation then raised a ValidationError. Use an empty dict when the seed has no metadata. Signed-off-by: NNIIOVO <haimingni44@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
SeedPrompt.metadatacan beNone, butSeedGroup._prompts_to_messagespassed it toMessagePiece(prompt_metadata=...), which requires a dict. So a seed group with such a prompt failed innext_message,user_messages,prepended_conversationandAttackParameters.from_seed_group_async. The PKU-SafeRLHF loader produces these seeds for every prompt without a harm category.This passes
prompt.metadata or {}, so those prompts get empty metadata like a prompt with no metadata set.Fixes #3508
Tests and Documentation
Added
test_seed_prompts_without_metadata_convert_to_messages, which covers history, the next message and user messages. It fails on main and passes with the fix. The models, executor, datasets and memory unit tests and pre-commit pass. No doc changes.