wfh)#jddlmZddlZddlmZmZddlmZddlmZGddejZ y)) annotationsN)Tensornn) CrossEncoder)fullnamecteZdZejdf dfd ZddZddZed dZ xZ S) ListNetLossNcNt|||_|xstj|_||_tj|_|jjdk7r9t|jjd|jjdy)a ListNet loss for learning to rank. This loss function implements the ListNet ranking algorithm which uses a list-wise approach to learn ranking models. It minimizes the cross entropy between the predicted ranking distribution and the ground truth ranking distribution. The implementation is optimized to handle padded documents efficiently by only processing valid documents during model inference. .. note:: The number of documents per query can vary between samples with the ``ListNetLoss``. Args: model (CrossEncoder): CrossEncoder model to be trained activation_fn (:class:`~torch.nn.Module`): Activation function applied to the logits before computing the loss. Defaults to :class:`~torch.nn.Identity`. mini_batch_size (int, optional): Number of samples to process in each forward pass. This has a significant impact on the memory consumption and speed of the training process. Three cases are possible: - If ``mini_batch_size`` is None, the ``mini_batch_size`` is set to the batch size. - If ``mini_batch_size`` is greater than 0, the batch is split into mini-batches of size ``mini_batch_size``. - If ``mini_batch_size`` is <= 0, the entire batch is processed at once. Defaults to None. References: - Learning to Rank: From Pairwise Approach to Listwise Approach: https://www.microsoft.com/en-us/research/publication/learning-to-rank-from-pairwise-approach-to-listwise-approach/ - Context-Aware Learning to Rank with Self-Attention: https://arxiv.org/abs/2005.10084 - `Cross Encoder > Training Examples > MS MARCO <../../../examples/cross_encoder/training/ms_marco/README.html>`_ Requirements: 1. Query with multiple documents (listwise approach) 2. Documents must have relevance scores/labels. Both binary and continuous labels are supported. Inputs: +----------------------------------------+--------------------------------+-------------------------------+ | Texts | Labels | Number of Model Output Labels | +========================================+================================+===============================+ | (query, [doc1, doc2, ..., docN]) | [score1, score2, ..., scoreN] | 1 | +----------------------------------------+--------------------------------+-------------------------------+ Recommendations: - Use :class:`~sentence_transformers.util.mine_hard_negatives` with ``output_format="labeled-list"`` to convert question-answer pairs to the required input format with hard negatives. Relations: - :class:`~sentence_transformers.cross_encoder.losses.LambdaLoss` takes the same inputs, and generally outperforms this loss. Example: :: from sentence_transformers.cross_encoder import CrossEncoder, CrossEncoderTrainer, losses from datasets import Dataset model = CrossEncoder("microsoft/mpnet-base") train_dataset = Dataset.from_dict({ "query": ["What are pandas?", "What is the capital of France?"], "docs": [ ["Pandas are a kind of bear.", "Pandas are kind of like fish."], ["The capital of France is Paris.", "Paris is the capital of France.", "Paris is quite large."], ], "labels": [[1, 0], [1, 1, 0]], }) loss = losses.ListNetLoss(model) trainer = CrossEncoderTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() z< supports a model with 1 output label, but got a model with z output labels.N) super__init__modelrIdentity activation_fnmini_batch_sizeCrossEntropyLosscross_entropy_loss num_labels ValueError __class____name__)selfrrrrs t/home/chris/cleankitchens-env/lib/python3.12/site-packages/sentence_transformers/cross_encoder/losses/ListNetLoss.pyr zListNetLoss.__init__ s\  *;bkkm."$"5"5"7 :: A %>>**+,((, (=(='>oO  &c t|tr tdt|dk7rtdt|d|\}}|Dcgc] }t|}}t |}t|}||Dcgc] }t|c}k7r)td|dDcgc] }t|c}dt ||D  cgc]\} }|D]} | | f} }} } | s,t jd|jjd S|jxs|} | d kr t| } g} td t| | D]}| ||| z}|jj|d d d }|j|jj}|jdi|d jd}| j|t j | d }|j#|}t j$||fd|jj}t j |Dcgc] }t j&t|"c}d }t j(t j&|t j|}||||f<t j*|t-d}t j d j-|||f<|j|jj}|j/||j1d}|Scc}wcc}wcc}wcc} }} wcc}w)a3 Compute ListNet loss for a batch of queries and their documents. Args: inputs: List of (queries, documents_list) labels: Ground truth relevance scores, shape (batch_size, num_documents) Returns: Tensor: Mean ListNet loss over the batch z]ListNetLoss expects a list of labels for each sample, but got a single value for each sample.zBListNetLoss expects two inputs (queries, documents_list), but got z inputs.z)Number of documents per query in inputs (z-) does not match number of labels per query (z).gT)device requires_gradrpt)padding truncationreturn_tensors)dimg7yA)rz-infr ) isinstancerrlenmaxziptorchtensorrrrrange tokenizertoviewappendcatrfullarangerepeat_interleave full_likefloatrsoftmax)rinputslabelsqueries docs_listdocsdocs_per_querymax_docs batch_sizequerydocumentpairsr logits_listimini_batch_pairstokenslogits logits_matrix doc_indices batch_indices labels_matrixlosss rforwardzListNetLoss.forwardesO ff %o  v;! TUXY_U`Taaij $09:#d)::~&\ ?fc&k? ?;N;KKxU[z\KQz}E{Fz\y]]_` 7:'96Mcc{ud^bcRZ%"c"cc<<DJJ,=,=TR R..<* a !%jO q#e*o6 'A$Q_)<= ZZ)) # *F YYtzz001FTZZ)&)!,11"5F   v & ';A.##F+ J#95IZIZ[ iiY OTc$i!8 OUVW // Z0H%,,WeJfg 4: m[01 uV}E 49IIf!4L4R4R4T m[01%(():):; &&}m6K6KPQ6K6RS k;@z\d@!Ps L;:MM M 8%McFt|j|jdS)z Get configuration parameters for this loss function. Returns: Dictionary containing the configuration parameters )rr)rrrrs rget_config_dictzListNetLoss.get_config_dicts!"*$*<*r^s&" </{")){r