初始化项目，由ModelHub XC社区提供模型

Model: llmware/bling-cerebras-1.3b-0.1
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,123 @@
+---
+license: apache-2.0  
+inference: false  
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+BLING-cerebras-1.3b-0.1 is part of the BLING ("Best Little Instruction-following No-GPU-required") model series, with instruct training on top of the cerebras/Cerebras-GPT-1.3B base.
+
+BLING models are fine-tuned with distilled high-quality custom instruct datasets, targeted at a specific subset of instruct tasks with 
+the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even 
+without using any advanced quantization optimizations.
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **Developed by:** llmware
+- **Model type:** Instruct-trained GPT decoder
+- **Language(s) (NLP):** English
+- **License:** Apache 2.0
+- **Finetuned from model [optional]:** cerebras/Cerebras-GPT-1.3B
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+The intended use of BLING models is two-fold:
+
+1.  Provide high-quality Instruct models that can run on a laptop for local testing.  We have found it extremely useful when building a
+   proof-of-concept, or working with sensitive enterprise data that must be closely guarded, especially in RAG use cases.
+
+2.  Push the state of the art for smaller Instruct-following models in the sub-7B parameter range, especially 1B-3B, as single-purpose
+    automation tools for specific tasks through targeted fine-tuning datasets and focused "instruction" tasks.
+
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+BLING is designed for enterprise automation use cases, especially in knowledge-intensive industries, such as financial services,
+legal and regulatory industries with complex information sources.  Rather than try to be "all things to all people," BLING models try to focus on a narrower set of Instructions more suitable to a ~1B parameter GPT model.
+
+BLING is ideal for rapid prototyping, testing, and the ability to perform an end-to-end workflow locally on a laptop without
+having to send sensitive information over an Internet-based API.
+
+The first BLING models have been trained for common RAG scenarios, specifically:   question-answering, key-value extraction, and basic summarization as the core instruction types
+without the need for a lot of complex instruction verbiage - provide a text passage context, ask questions, and get clear fact-based responses.
+
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+Any model can provide inaccurate or incomplete information, and should be used in conjunction with appropriate safeguards and fact-checking mechanisms.
+
+
+## How to Get Started with the Model
+
+The fastest way to get started with BLING is through direct import in transformers:
+
+    from transformers import AutoTokenizer, AutoModelForCausalLM  
+    tokenizer = AutoTokenizer.from_pretrained("llmware/bling-cerebras-1.3b-0.1")  
+    model = AutoModelForCausalLM.from_pretrained("llmware/bling-cerebras-1.3b-0.1")  
+
+Please refer to the generation_test .py files in the Files repository, which includes 200 samples and script to test the model.  The **generation_test_llmware_script.py** includes built-in llmware capabilities for fact-checking, as well as easy integration with document parsing and actual retrieval to swap out the test set for RAG workflow consisting of business documents.  
+
+
+The BLING model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
+
+    full_prompt = "\<human>\: " + my_prompt + "\n" + "\<bot>\:"
+
+The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
+
+1.  Text Passage Context, and
+2.  Specific question or instruction based on the text passage
+
+To get the best results, package "my_prompt" as follows:
+
+    my_prompt = {{text_passage}} + "\n" + {{question/instruction}}
+
+
+
+If you are using a HuggingFace generation script:
+
+    # prepare prompt packaging used in fine-tuning process
+    new_prompt = "<human>: " + entries["context"] + "\n" + entries["query"] + "\n" + "<bot>:"
+
+    inputs = tokenizer(new_prompt, return_tensors="pt")  
+    start_of_output = len(inputs.input_ids[0])
+
+    #   temperature: set at 0.3 for consistency of output
+    #   max_new_tokens:  set at 100 - may prematurely stop a few of the summaries
+
+    outputs = model.generate(
+            inputs.input_ids.to(device),
+            eos_token_id=tokenizer.eos_token_id,
+            pad_token_id=tokenizer.eos_token_id,
+            do_sample=True,
+            temperature=0.3,
+            max_new_tokens=100,
+            )
+
+    output_only = tokenizer.decode(outputs[0][start_of_output:],skip_special_tokens=True)  
+
+   
+## Citation [optional]
+
+This BLING model is built on top of a Cerebras base GPT trained model - for more information about the Cerebras GPT models, please see the following paper:
+
+{
+Title:  Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster
+Authors: Nolan Dey, Gurpreet Gosal, Zhiming (Charles) Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joe Hestness
+Publication:  April 6, 2023
+}
+
+
+## Model Card Contact
+
+Darren Oberst & llmware team
+

config.json

@@ -0,0 +1,36 @@
+{
+  "model_name": "bling-cerebras-1.3b-0.1",
+  "description": "Instruct train fine-tuning using distilled knowledge based critical reading tasks training dataset",
+  "training_timestamp": "Sat Oct 7 08:52:05 2023",
+  "training_comments": "cerebras-1.3b-base",
+    "_name_or_path": "cerebras/Cerebras-GPT-1.3B",
+    "architectures": [
+      "GPT2LMHeadModel"
+    ],
+    "activation_function": "gelu",
+    "attn_pdrop": 0.0,
+    "bos_token_id": 50256,
+    "embd_pdrop": 0.0,
+    "eos_token_id": 50256,
+    "initializer_range": 0.02,
+    "layer_norm_epsilon": 1e-05,
+    "model_type": "gpt2",
+    "n_embd": 2048,
+    "n_head": 16,
+    "n_inner": 8192,
+    "n_layer": 24,
+    "n_positions": 2048,
+    "reorder_and_upcast_attn": false,
+    "resid_pdrop": 0.0,
+    "scale_attn_by_inverse_layer_idx": false,
+    "scale_attn_weights": true,
+    "summary_activation": null,
+    "summary_first_dropout": 0.1,
+    "summary_proj_to_labels": true,
+    "summary_type": "cls_index",
+    "summary_use_proj": true,
+    "transformers_version": "4.33.2",
+    "use_cache": true,
+    "vocab_size": 50257
+  }
+

generation_test_hf_script.py

@@ -0,0 +1,89 @@
+
+
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+
+def load_rag_benchmark_tester_ds():
+
+    # pull 200 question rag benchmark test dataset from LLMWare HuggingFace repo
+    from datasets import load_dataset
+
+    ds_name = "llmware/rag_instruct_benchmark_tester"
+
+    dataset = load_dataset(ds_name)
+
+    print("update: loading RAG Benchmark test dataset - ", dataset)
+
+    test_set = []
+    for i, samples in enumerate(dataset["train"]):
+        test_set.append(samples)
+
+        # to view test set samples
+        # print("rag benchmark dataset test samples: ", i, samples)
+
+    return test_set
+
+
+def run_test(model_name, test_ds):
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+
+    print("\nRAG Performance Test - 200 questions")
+    print("update: model - ", model_name)    
+    print("update: device - ", device)
+
+    model = AutoModelForCausalLM.from_pretrained(model_name)
+    model.to(device)
+
+    tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+    for i, entries in enumerate(test_ds):
+
+        # prepare prompt packaging used in fine-tuning process
+        new_prompt = "<human>: " + entries["context"] + "\n" + entries["query"] + "\n" + "<bot>:"
+
+        inputs = tokenizer(new_prompt, return_tensors="pt")
+        start_of_output = len(inputs.input_ids[0])
+
+        #   temperature: set at 0.3 for consistency of output
+        #   max_new_tokens:  set at 100 - may prematurely stop a few of the summaries
+
+        outputs = model.generate(
+            inputs.input_ids.to(device),
+            eos_token_id=tokenizer.eos_token_id,
+            pad_token_id=tokenizer.eos_token_id,
+            do_sample=True,
+            temperature=0.3,
+            max_new_tokens=100,
+            )
+
+        output_only = tokenizer.decode(outputs[0][start_of_output:],skip_special_tokens=True)
+
+        # quick/optional post-processing clean-up of potential fine-tuning artifacts
+
+        eot = output_only.find("<|endoftext|>")
+        if eot > -1:
+            output_only = output_only[:eot]
+
+        bot = output_only.find("<bot>:")
+        if bot > -1:
+            output_only = output_only[bot+len("<bot>:"):]
+
+        # end - post-processing
+
+        print("\n")
+        print(i, "llm_response - ", output_only)
+        print(i, "gold_answer - ", entries["answer"])
+
+    return 0
+
+
+if __name__ == "__main__":
+
+    test_ds = load_rag_benchmark_tester_ds()
+
+    model_name = "llmware/bling-cerebras-1.3b-0.1"
+    output = run_test(model_name,test_ds)
+
+

generation_test_llmware_script.py

@@ -0,0 +1,64 @@
+
+from llmware.prompts import Prompt
+
+
+def load_rag_benchmark_tester_ds():
+
+    # pull 200 question rag benchmark test dataset from LLMWare HuggingFace repo
+    from datasets import load_dataset
+
+    ds_name = "llmware/rag_instruct_benchmark_tester"
+
+    dataset = load_dataset(ds_name)
+
+    print("update: loading test dataset - ", dataset)
+
+    test_set = []
+    for i, samples in enumerate(dataset["train"]):
+        test_set.append(samples)
+
+        # to view test set samples
+        # print("rag benchmark dataset test samples: ", i, samples)
+
+    return test_set
+
+
+def run_test(model_name, prompt_list):
+
+    print("\nupdate: Starting RAG Benchmark Inference Test")
+
+    prompter = Prompt().load_model(model_name,from_hf=True)
+
+    for i, entries in enumerate(prompt_list):
+
+        prompt = entries["query"]
+        context = entries["context"]
+
+        response = prompter.prompt_main(prompt,context=context,prompt_name="default_with_context", temperature=0.3)
+
+        fc = prompter.evidence_check_numbers(response)
+        sc = prompter.evidence_comparison_stats(response)
+        sr = prompter.evidence_check_sources(response)
+
+        print("\nupdate: model inference output - ", i, response["llm_response"])
+        print("update: gold_answer              - ", i, entries["answer"])
+
+        for entries in fc:
+            print("update: fact check - ", entries["fact_check"])
+
+        for entries in sc:
+            print("update: comparison stats - ", entries["comparison_stats"])
+
+        for entries in sr:
+            print("update: sources - ", entries["source_review"])
+
+    return 0
+
+
+if __name__ == "__main__":
+
+    core_test_set = load_rag_benchmark_tester_ds()
+
+    model_name = "llmware/bling-cerebras-1.3b-0.1"
+
+    output = run_test(model_name, core_test_set)

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18fc7e86391e3d51531015f317f9c62bbddd477b2f2b8b53ef3839433e51e3da
+size 5363683717