OCI Generative AI is a fully managed Oracle Cloud Infrastructure service that provides customizable large language models (LLMs) that cover a wide range of use cases, including chat, text generation, summarization, and creating text embeddings.
You can use APIs & scripts to embed OCI Gen AI to your applications after testing it out on the playground from the OCI console.
In the example below using python.
# coding: utf-8
# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved.
# This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl or Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. You may choose either license.
##########################################################################
# chat_demo.py
# Supports Python 3
##########################################################################
# Info:
# Get texts from LLM model for given prompts using OCI Generative AI Service.
##########################################################################
# Application Command line(no parameter needed)
# python chat_demo.py
##########################################################################
import oci
# Setup basic variables
# Auth Config
# TODO: Please update config profile name and use the compartmentId that has policies grant permissions for using Generative AI Service
compartment_id = "ocid1.compartment.oc1.."
CONFIG_PROFILE = "DEFAULT"
config = oci.config.from_file('~/.oci/config', CONFIG_PROFILE)
# Service endpoint
endpoint = "https://inference.generativeai...."
generative_ai_inference_client = oci.generative_ai_inference.GenerativeAiInferenceClient(config=config, service_endpoint=endpoint, retry_strategy=oci.retry.NoneRetryStrategy(), timeout=(10,240))
chat_detail = oci.generative_ai_inference.models.ChatDetails()
chat_request = oci.generative_ai_inference.models.CohereChatRequest()
#chat_request.message = "{input}"
chat_request.message = """
Instructions:
Here are customer feedback emails for a book retailer. Please
classify each of the emails in 2 categories: good feedback and bad feedback, don't give an explanation.
Write a response for each of the good & bad feedback emails. Be polite and proffesional.
Data:
Email 1:
Subject: A Bookworm's Paradise!
Dear BookHunter ,
Just wanted to drop a note and say how thrilled I am with your diverse collection of books. Every visit feels like an adventure, and I always find hidden gems that aren't available elsewhere. Keep up the fantastic work!
Email 2:
Subject: Shipping Delays - A Small Wrinkle in My Experience
Hello BookHunter ,
I've been a loyal customer for years and have always loved your book selection. However, my last order took longer than expected to arrive. Perhaps there's an opportunity to review your shipping partners or provide clearer delivery estimates?
Email 3:
Subject: Disheartened by My Recent Experience
Dear BookHunter ,
I hate to say this, but my recent visit to your store was quite disappointing. The staff seemed disinterested and didn't help when I asked for recommendations. Such a stark contrast from the warm, welcoming experience I used to love. I hope this was just a one-off.
"""
chat_request.max_tokens = 600
chat_request.temperature = 0.25
chat_request.frequency_penalty = 0
chat_request.top_p = 0.75
chat_request.top_k = 0
chat_detail.serving_mode = oci.generative_ai_inference.models.OnDemandServingMode(model_id="ocid1.generativeaimodel.oc1.uk-london-1.a....")
chat_detail.chat_request = chat_request
chat_detail.compartment_id = compartment_id
chat_response = generative_ai_inference_client.chat(chat_detail)
# Print result
print("**************************Chat Result**************************")
print(vars(chat_response))
Results:
Happy Coding!