wfh5ddlmZddlZddlmZmZddlmZddlmZGddejZ Gdd ejZ y) ) annotationsN)Tensornn) CrossEncoder)fullnamec.eZdZdZddfd ZddZxZS)PListMLELambdaWeightzMBase class for implementing weighting schemes in Position-Aware ListMLE Loss.c0t|||_y)z Initialize a lambda weight for PListMLE loss. Args: rank_discount_fn: Function that computes a discount for each rank position. If None, uses default discount of 2^(num_docs - rank) - 1. N)super__init__rank_discount_fn)selfr __class__s u/home/chris/cleankitchens-env/lib/python3.12/site-packages/sentence_transformers/cross_encoder/losses/PListMLELoss.pyr zPListMLELambdaWeight.__init__ s  0c&|j|j|S|jdd}tj|j d|j j |}tjd||z dz }||z}|S)a Calculate position-aware weights for the PListMLE loss. Args: mask: A boolean mask indicating valid positions [batch_size, num_docs] Returns: Tensor: Weights for each position [batch_size, num_docs] Tdimkeepdimdeviceg@g?)r sumtorcharangesizer expand_aspow)rmasknum_docs_per_queryranksweightss rforwardzPListMLELambdaWeight.forwards  ,((. ."XX!TX: TYYq\$++>HHN))C!3e!; Training Examples > MS MARCO <../../../examples/cross_encoder/training/ms_marco/README.html>`_ Requirements: 1. Query with multiple documents (listwise approach) 2. Documents must have relevance scores/labels. Both binary and continuous labels are supported. 3. Documents must be sorted in a defined rank order. Inputs: +----------------------------------------+--------------------------------+-------------------------------+ | Texts | Labels | Number of Model Output Labels | +========================================+================================+===============================+ | (query, [doc1, doc2, ..., docN]) | [score1, score2, ..., scoreN] | 1 | +----------------------------------------+--------------------------------+-------------------------------+ Recommendations: - Use :class:`~sentence_transformers.util.mine_hard_negatives` with ``output_format="labeled-list"`` to convert question-answer pairs to the required input format with hard negatives. Relations: - The :class:`~sentence_transformers.cross_encoder.losses.PListMLELoss` is an extension of the :class:`~sentence_transformers.cross_encoder.losses.ListMLELoss` and allows for positional weighting of the loss. :class:`~sentence_transformers.cross_encoder.losses.PListMLELoss` generally outperforms :class:`~sentence_transformers.cross_encoder.losses.ListMLELoss` and is recommended over it. - :class:`~sentence_transformers.cross_encoder.losses.LambdaLoss` takes the same inputs, and generally outperforms this loss. Example: :: from sentence_transformers.cross_encoder import CrossEncoder, CrossEncoderTrainer, losses from datasets import Dataset model = CrossEncoder("microsoft/mpnet-base") train_dataset = Dataset.from_dict({ "query": ["What are pandas?", "What is the capital of France?"], "docs": [ ["Pandas are a kind of bear.", "Pandas are kind of like fish."], ["The capital of France is Paris.", "Paris is the capital of France.", "Paris is quite large."], ], "labels": [[1, 0], [1, 1, 0]], }) # Either: Position-Aware ListMLE with default weighting lambda_weight = losses.PListMLELambdaWeight() loss = losses.PListMLELoss(model, lambda_weight=lambda_weight) # or: Position-Aware ListMLE with custom weighting function def custom_discount(ranks): # e.g. ranks: [1, 2, 3, 4, 5] return 1.0 / torch.log1p(ranks) lambda_weight = losses.PListMLELambdaWeight(rank_discount_fn=custom_discount) loss = losses.PListMLELoss(model, lambda_weight=lambda_weight) trainer = CrossEncoderTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() g|=rz< supports a model with 1 output label, but got a model with z output labels.N)r r model lambda_weightrIdentity activation_fnmini_batch_sizerespect_input_ordereps num_labels ValueErrorrr&)rr/r0r2r3r4rs rr zPListMLELoss.__init__.sH  **;bkkm.#6  :: A %>>**+,((, (=(='>oO  &rc  t|tr tdt|dk7rtdt|d|\}}|Dcgc] }t|}}t |}t|}||Dcgc] }t|c}k7r)td|dDcgc] }t|c}dt ||D  cgc]\} }|D]} | | f} }} } | s,t jd|jjd S|jxs|} | d kr t| } g} td t| | D]}| ||| z}|jj|d d d }|j|jj}|jdi|d jd}| j|t j | d }|j#|}t j$||fd|jj}t j |Dcgc] }t j&t|"c}d }t j(t j&|t j|}||||f<t j*|t j,}d |||f<t j.|t1d }t j d j1|||f<t j2|s,t jd|jjd S|j4s.|j7d d\}}t j8|d|}n|}|j;}t j<t j>t j<|dgddg}|t j@||jBzz }|jD9|jE|}||jGdd |jBzz }||z}d||<t jF|d }t j2|s,t jd|jjd St jH|Scc}wcc}wcc}wcc} }} wcc}w)a5 Compute PListMLE loss for a batch of queries and their documents. Args: inputs: List of (queries, documents_list) labels: Ground truth relevance scores, shape (batch_size, num_documents) Returns: Tensor: Mean PListMLE loss over the batch z^PListMLELoss expects a list of labels for each sample, but got a single value for each sample.zCPListMLELoss expects two inputs (queries, documents_list), but got z inputs.z)Number of documents per query in inputs (z-) does not match number of labels per query (z).gT)r requires_gradrpt)padding truncationreturn_tensors)rgؗҜ->1d->-SVZV^V^-^_M!M1I 4%!IIiQ77yy)*<<DJJ,=,=TR Rzz*++k;@z\d@!Ps S :SS S8%S c|jdnt|jt|j|j|jdS)z Get configuration parameters for this loss function. Returns: Dictionary containing the configuration parameters N)r0r2r3r4)r0rr2r3r4rs rget_config_dictzPListMLELoss.get_config_dict sI&*%7%7%?TXdN`N`Ea%d&8&89#33#'#;#;   rcy)Na  @inproceedings{lan2014position, title={Position-Aware ListMLE: A Sequential Learning Process for Ranking}, author={Lan, Yanyan and Zhu, Yadong and Guo, Jiafeng and Niu, Shuzi and Cheng, Xueqi}, booktitle={UAI}, volume={14}, pages={449--458}, year={2014} } rCrys rcitationzPListMLELoss.citations r) r/rr0zPListMLELambdaWeight | Noner2znn.Module | Noner3z int | Noner4rRr$r%)r]z list[list[str], list[list[str]]]r^z list[Tensor]r$r)r$z#dict[str, float | int | str | None])r$str) r&r'r(r rr1r r#rzpropertyr|r*r+s@rr-r--s6J5K*5"++-&*$( pp3p( p $ p " p pdk,Z    rr-) __future__rrrr#sentence_transformers.cross_encoderrsentence_transformers.utilrModuler r-rCrrrs7" </ 299 Fy299yr