U h@sdZddlmZddlZddlmZmZmZddlm Z ddl m Z ddl m Z ddl m Z Gd d d eZGd d d ejZGd ddee jZGdddee jZddddZddddZdddddZdddddZd d!dd"d#ZdS)$z2Utilities for working with post-processing tokens.) annotationsN)AnyCallableSequence)py_utils) llm_function)llmfn_output_row)llmfn_post_processc@seZdZdZdS)PostProcessParseErrorz,An error parsing the post-processing tokens.N)__name__ __module__ __qualname____doc__rrS/tmp/pip-unpacked-wheel-doshhd5e/google/generativeai/notebook/post_process_utils.pyr sr c@s:eZdZdZejddddZejddddd Zd S) ParsedPostProcessExprz:A post-processing expression parsed from the command line.strreturncCsdS)z$Returns the name of this expression.Nrselfrrrname"szParsedPostProcessExpr.namellm_function.LLMFunctionllm_fnrcCsdS)zEAdds this parsed expression to `llm_fn` as a post-processing command.Nrrrrrradd_to_llm_function&sz)ParsedPostProcessExpr.add_to_llm_functionN)r r r rabcabstractmethodrrrrrrrs rc@sNeZdZdZdddddZdddd Zd d d d dZdddddZdS)_ParsedPostProcessAddExprzEAn expression that returns the value of a new column to add to a row.rCallable[[str], Any]rfncCs||_||_dS)zConstructor. Args: name: The name of the expression. The name of the new column will be derived from this. fn: A function that takes the result of a row and returns a new value to add as a new column in the row. N_name_fnrrr"rrr__init__2s z"_ParsedPostProcessAddExpr.__init__rcCs|jSNr$rrrrr>sz_ParsedPostProcessAddExpr.name-Sequence[llmfn_output_row.LLMFnOutputRowView]z Sequence[Any]rowsrcsfdd|DS)Ncsg|]}|qSrr%Z result_value.0rowrrr Dsz6_ParsedPostProcessAddExpr.__call__..rrr,rrr__call__Asz"_ParsedPostProcessAddExpr.__call__rrcCs|j|j|dSNr!)Zadd_post_process_add_fnr$rrrrrFsz-_ParsedPostProcessAddExpr.add_to_llm_functionNr r r rr'rr3rrrrrr-s  rc@sNeZdZdZdddddZdddd Zd d d d dZdddddZdS)_ParsedPostProcessReplaceExprz:An expression that returns the new result value for a row.rCallable[[str], str]r!cCs||_||_dS)zConstructor. Args: name: The name of the expression. fn: A function that takes the result of a row and returns the new result. Nr#r&rrrr'Qsz&_ParsedPostProcessReplaceExpr.__init__rcCs|jSr(r)rrrrr[sz"_ParsedPostProcessReplaceExpr.namer* Sequence[str]r+csfdd|DS)Ncsg|]}|qSrr-r.rrrr1asz:_ParsedPostProcessReplaceExpr.__call__..rr2rrrr3^sz&_ParsedPostProcessReplaceExpr.__call__rrcCs|j|j|dSr4)Zadd_post_process_replace_fnr$rrrrrcsz1_ParsedPostProcessReplaceExpr.add_to_llm_functionNr5rrrrr6Ls  r6r r"cCst|j|dSr4)rr r9rrrpost_process_add_fnjsr:r7cCst|j|dSr4)r6r r9rrrpost_process_replace_fnnsr;r8None)tokensrcCs$|s tdt|dkr tddS)Nz,Cannot have empty post-processing expressionz3Post-processing expression should be a single token)r len)r=rrr'validate_one_post_processing_expressionrs  r@ztuple[str, Any]c Csvt||dd}t}t|D]@\}}t|}||kr^tdd|d|d||}q&d||fS)z)Returns name and the resolved expression.r.zUnable to resolve "{}"Nr> ) r@splitrZget_main_module enumeratevarsr formatjoin)r=Z token_partsZcurrent_moduleZpart_numpartZcurrent_module_varsrrr'_resolve_one_post_processing_expression}s rIzSequence[Sequence[str]]zSequence[ParsedPostProcessExpr]cCs`g}|D]R}t|\}}t|tr.||qt|trL|t||dqtd|q|S)aResolves post-processing tokens into ParsedPostProcessExprs. E.g. Given [["add_length"], ["to_upper"]] as input, this function will return a sequence of ParsedPostProcessExprs that will execute add_length() and to_upper() on each entry of the LLM output as post-processing operations. Raises: PostProcessParseError: An error parsing or resolving the tokens. Args: tokens: A sequence of post-processing tokens after splitting. Returns: A sequence of ParsedPostProcessExprs. r!z{} is not callable)rI isinstancerappendrrr rF)r=resultsZ expressionZ expr_nameZ expr_valuerrrresolve_post_processing_tokenss    rM)r __future__rrtypingrrrZgoogle.generativeai.notebookrZ google.generativeai.notebook.librrr RuntimeErrorr ABCrZLLMFnPostProcessBatchAddFnrZLLMFnPostProcessBatchReplaceFnr6r:r;r@rIrMrrrrs,