|
|
|
|
@@ -0,0 +1,138 @@
|
|
|
|
|
---
|
|
|
|
|
pipeline_tag: text-ranking
|
|
|
|
|
---
|
|
|
|
|
Quantization made by Richard Erkhov.
|
|
|
|
|
|
|
|
|
|
[Github](https://github.com/RichardErkhov)
|
|
|
|
|
|
|
|
|
|
[Discord](https://discord.gg/pvy7H8DZMG)
|
|
|
|
|
|
|
|
|
|
[Request more models](https://github.com/RichardErkhov/quant_request)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
TinyLlama-ContextQuestionPair-Classifier-Reranker - GGUF
|
|
|
|
|
- Model creator: https://huggingface.co/cnmoro/
|
|
|
|
|
- Original model: https://huggingface.co/cnmoro/TinyLlama-ContextQuestionPair-Classifier-Reranker/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Name | Quant method | Size |
|
|
|
|
|
| ---- | ---- | ---- |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q2_K.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q2_K.gguf) | Q2_K | 0.4GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_XS.gguf) | IQ3_XS | 0.44GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_S.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_S.gguf) | IQ3_S | 0.47GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_S.gguf) | Q3_K_S | 0.47GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_M.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ3_M.gguf) | IQ3_M | 0.48GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K.gguf) | Q3_K | 0.51GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_M.gguf) | Q3_K_M | 0.51GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q3_K_L.gguf) | Q3_K_L | 0.55GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ4_XS.gguf) | IQ4_XS | 0.57GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_0.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_0.gguf) | Q4_0 | 0.59GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.IQ4_NL.gguf) | IQ4_NL | 0.6GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K_S.gguf) | Q4_K_S | 0.6GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K.gguf) | Q4_K | 0.62GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_K_M.gguf) | Q4_K_M | 0.62GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_1.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q4_1.gguf) | Q4_1 | 0.65GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_0.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_0.gguf) | Q5_0 | 0.71GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K_S.gguf) | Q5_K_S | 0.71GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K.gguf) | Q5_K | 0.73GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_K_M.gguf) | Q5_K_M | 0.73GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_1.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q5_1.gguf) | Q5_1 | 0.77GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q6_K.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q6_K.gguf) | Q6_K | 0.84GB |
|
|
|
|
|
| [TinyLlama-ContextQuestionPair-Classifier-Reranker.Q8_0.gguf](https://huggingface.co/RichardErkhov/cnmoro_-_TinyLlama-ContextQuestionPair-Classifier-Reranker-gguf/blob/main/TinyLlama-ContextQuestionPair-Classifier-Reranker.Q8_0.gguf) | Q8_0 | 1.09GB |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Original model description:
|
|
|
|
|
---
|
|
|
|
|
license: cc-by-nc-2.0
|
|
|
|
|
language:
|
|
|
|
|
- en
|
|
|
|
|
- pt
|
|
|
|
|
tags:
|
|
|
|
|
- classification
|
|
|
|
|
- llama
|
|
|
|
|
- tinyllama
|
|
|
|
|
- rag
|
|
|
|
|
- rerank
|
|
|
|
|
---
|
|
|
|
|
```python
|
|
|
|
|
template = """<s><|system|>
|
|
|
|
|
You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
|
|
|
|
|
<|user|>
|
|
|
|
|
Context:
|
|
|
|
|
{Text}
|
|
|
|
|
|
|
|
|
|
Question:
|
|
|
|
|
{Prompt}</s>
|
|
|
|
|
<|assistant|>
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
# Output should be:
|
|
|
|
|
|
|
|
|
|
{"relevant": true}
|
|
|
|
|
|
|
|
|
|
# or
|
|
|
|
|
|
|
|
|
|
{"relevant": false}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
```text
|
|
|
|
|
<s><|system|>
|
|
|
|
|
You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
|
|
|
|
|
<|user|>
|
|
|
|
|
Context:
|
|
|
|
|
old. NFT were observed in almost all patients over 60 years of age, but the incidence was low.
|
|
|
|
|
Many ubiquitin-positive small-sized granules were observed in the second and third layer of the parahippocampal gyrus of aged patients,
|
|
|
|
|
and the incidence rose with increasing age. On the other hand, few of these granules were in patients with Alzheimer\'s type dementia.
|
|
|
|
|
Granulovacuolar degeneration was examined. Many centrally-located granules were positive for ubiquitin. Based on electron microscopic
|
|
|
|
|
observation of these granules at several stages, the granules were thought to be a type of autophagosome. During the first stage of
|
|
|
|
|
granulovacuolar degeneration, electron-dense materials appeared in the cytoplasm, following which they were surrounded by smooth cytoplasm,
|
|
|
|
|
following which they were surrounded by smooth endoplasmic reticulum. Analytical electron microscopy disclosed that the granules contained
|
|
|
|
|
some aluminium. Several senile changes in the central nervous system in cadavers were examined. The pattern of extension of Alzheimer\'s
|
|
|
|
|
neurofibrillary tangles (NFT) and senile plaques (SP) in the olfactory bulbs of 100 specimens was examined during routine autopsy by
|
|
|
|
|
immunohistochemical staining. NFT were first observed in the anterior olfactory nucleus after the age of 60, and incidence rose with
|
|
|
|
|
increasing age. Senile plaques were found in the nucleus when there were many SP in the cerebral cortex. Of 25 non-demented amyotrophic
|
|
|
|
|
lateral sclerosis patients, SP were found in the cerebral cortices of 10, and 9 of 10 were over 60 years old. NFT were observed in almost
|
|
|
|
|
all patients over
|
|
|
|
|
|
|
|
|
|
Question:
|
|
|
|
|
What is granulovacuolar degeneration and what was its observation on electron microscopy?</s>
|
|
|
|
|
<|assistant|>
|
|
|
|
|
{"relevant": true}</s>
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
vLLM recommended request parameters:
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
prompt = "<s><|system|>\nYou are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>\n<|user|>\nContext:\nConhecida como missão de imagem de raios-x e espectroscopia (da sigla em inglês XRISM), a estratégia é utilizar o telescópio para ampliar os estudos da humanidade a níveis celestiais com uma fração dos pixels da tela de um Gameboy original, lançado em 1989. Isso é possível por meio de uma ferramenta chamada “Resolve”. Apesar de utilizar a medição em pixels, a tecnologia é bastante diferente de uma câmera. Com um conjunto de microcalorímetros de seis pixels quadrados que mede 0,5 cm², ela detecta a temperatura de cada raio-x que o atinge. Como funciona o Resolve do telescópio XRISM? Cientista do projeto XRISM da NASA, Brian Williams explicou em um comunicado o funcionamento do telescópio. “Chamamos o Resolve de espectrômetro de microcalorímetros porque cada um de seus 36 pixels está medindo pequenas quantidades de calor entregues por cada raio-x recebido, nos permitindo ver as impressões digitais químicas dos elementos que compõem as fontes com detalhes sem precedentes”.\n\nQuestion:\nQual é a sigla em alemão mencionada?</s>\n<|assistant|>\n{\"relevant\":"
|
|
|
|
|
|
|
|
|
|
headers = {
|
|
|
|
|
"Accept": "text/event-stream",
|
|
|
|
|
"Authorization": "Bearer EMPTY"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
body = {
|
|
|
|
|
"model": model,
|
|
|
|
|
"prompt": [prompt],
|
|
|
|
|
"best_of": 5,
|
|
|
|
|
"max_tokens": 1,
|
|
|
|
|
"temperature": 0,
|
|
|
|
|
"top_p": 1,
|
|
|
|
|
"use_beam_search": True,
|
|
|
|
|
"top_k": -1,
|
|
|
|
|
"min_p": 0,
|
|
|
|
|
"repetition_penalty": 1,
|
|
|
|
|
"length_penalty": 1,
|
|
|
|
|
"min_tokens": 1,
|
|
|
|
|
"logprobs": 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
result = requests.post(base_uri, headers=headers, json=body)
|
|
|
|
|
result = result.json()
|
|
|
|
|
|
|
|
|
|
boolean_response = bool(eval(json_result['choices'][0]['text'].strip().title()))
|
|
|
|
|
print(boolean_response)
|
|
|
|
|
```
|
|
|
|
|
|
