Conversation
henchaves
left a comment
There was a problem hiding this comment.
I think we are still giving much attention to the usage of InteractionRecord as a Scenario component. We need to focus on using only Interaction wherever it's possible.
The manual instantiation of InteractionRecord should be limited for when creating a Trace:
trace = Trace(interactions=[InteractionRecord(inputs="test", outputs="result")])
Having scenarios with mixed InteractionRecord and Interaction is not nice, and it's kind of the same thing of having InteractionSpec/Interaction - so what is the goal of this PR?
# ❌ BAD
scenario = Scenario(sequence=[
InteractionRecord(inputs="test", outputs="result"),
Interaction(inputs="hello", outputs="word")
])
If you think it's appropriate, I would make only Interaction as a Scenario component (and Check, of course), and if needed, users could map InteractionRecord to Interaction using an API such as Scenario.from_records()
Overall, the implementation is well done. Did many tests with different scenarios and all of them worked well.
libs/giskard-checks/src/giskard/checks/core/interaction/interaction.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Henrique Chaves <44180294+henchaves@users.noreply.github.com>
Co-authored-by: Henrique Chaves <44180294+henchaves@users.noreply.github.com>
|
@henchaves should have addressed all comments, except for the union which didn't feel necessary at this stage. |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a significant and valuable refactoring of the interaction APIs, clarifying the distinction between dynamic specifications (Interact) and static records (Interaction). The code changes are well-executed and the new structure is more intuitive. However, the documentation (README, CODEMAP) and some docstrings have not been fully updated to reflect these changes, leading to inconsistencies and incorrect examples. I've pointed out several places where class names like Interact, InteractionSpec, and Interaction are used incorrectly. Addressing these documentation issues will be crucial for users to understand and adopt the new API.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a major and beneficial refactoring of the interaction API, clarifying the distinction between dynamic interaction specifications (Interact) and static interaction records (Interaction). The renames from InteractionSpec to Interact and BaseInteractionSpec to InteractionSpec improve clarity. The code reorganization into the checks.core.interaction submodule is logical. The changes are extensive but appear to be consistently applied across the codebase, including documentation and tests. I've found a couple of minor issues in the documentation that should be addressed.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Major renaming and API change:
InteractionSpecis now simplyInteractionInteract. This is the main dynamic interface. It extendsInteractionSpecwhich is its abstract definition (withgeneratemethod).The oldInteractionbecomesInteractionRecordto signal that it is an immutable realization of theInteractionInteractionremains a static representation of a single exchange, but it is now only used internally and not exposed. E.g.Scenarioonly acceptsInteractionSpec/Interactobjects and noInteraction.Scenarios and testcases now only see
InteractionSpec, i.e. the dynamic version. However, theChecklogic receives a staticTracecomposed ofInteractions which are guaranteed to be complete (no dynamic values, no mutable fields).I also regrouped the interactions and trace objects under the
checks.core.interactionsubmodule. Before, definitions were split acrosschecks.core.interaction,checks.interaction, andchecks.core.trace.This replaces #2269 by maintaining the two distinct classes (dynamic vs frozen).