上課筆記系列(3) — ChatGPT Prompt Engineering for Developers(4)擴充/問答機器人/結語
課程筆記索引：
(1) 簡介及互動基本指引
(2) 迭代和總結
(3) 推論和轉換
(4) 擴充和聊天機器人/結語 — 本篇
方便有興趣的朋友點閱。
擴充
就是文意擷取/下標題的反向操作，輸入指示只提供簡短的文字段落，甚至類似分類標籤，然後要求模型輸出一段較長的文字段落。
值得注意的是，有些人會利用它來製造垃圾郵件。所以吳恩達語重心長地提醒，請將這個功能用在正途，或能幫助人的地方。
課堂舉的例子是，利用AI助手的名義閱讀使用者回饋，如果發現回饋帶有正面或中立語氣，就回覆感謝他的使用和選擇；如果帶有負面，就道歉。
必須根據回饋的內容來寫回覆。
語氣必須專業。
必須署名「AI回饋助手」在最後面，以顯示透明性。
關於溫度(temperature)
是一個輸入指示的參數。簡單來說，他可以調整模型每次產生回答的重複性。由於模型升成文字段落是機率性的，所以理論上每次生成的文字段落，其內容多少有一些差異。然而，也有可能對於某個文字段落，其接續的文字有極高機率是特定的字，那這樣在生成時，模型大概率每次都是生成這個文字。例如：
經過統計，我最愛的食物是：70%披薩，20%米飯，5%麵。
那模型在生成文字時，如果提到「我最愛的食物是」時，70%的機率會接「披薩」。所以可能只有很少的機會，模型後面會接米飯或麵。如果你希望生成文字的時候有較高機率看到接的是米飯或麵，可以調高temperature，例如調到0.3、0.5甚至0.7。其原本是放在API裡面的參數，我們在建立get_completion包裝API函數時，已經一起包進來了，保留這個參數設定的空間。再看一次這個函數：(沒有程式設計背景的也沒關係，可以跳過這段，知道有一個這樣的東西就好)
def get_completion(prompt, model="gpt-3.5-turbo",temperature=0): # Andrew mentioned that the prompt/ completion paradigm is preferable for this class
messages = [{"role": "user", "content": prompt}]
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=temperature, # this is the degree of randomness of the model's output
)
return response.choices[0].message["content"]所以怎麼設定temperature？其實和我們的需求有關係。如果我們希望生成的文字段落盡量一致、穩定，那就讓temperature低一些，或者不設定(預設是0)。反之，如果希望回覆的多變一些，那就調高它。
聊天機器人
OpenAI本身有為ChatGPT設計互動的網路介面了，網址在此：
https://chat.openai.com/
這讓使用者和模型互動的時候，模型成為一個問答機器人。在這邊則是要讓我們客製化，打造自己的聊天機器人。這個應用領域很廣，例如作為智慧客服，或者初步篩選訊息的助手等等。如果有玩linebot，串接line也是可能的。
本段涉及較多程式設計內容，若無相關背景，可直接跳過。
和chatGPT互動的機制
我們先了解一下API和chatGPT互動的機制。
def get_completion(prompt, model="gpt-3.5-turbo"):
messages = [{"role": "user", "content": prompt}]
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=0, # this is the degree of randomness of the model's output
)
return response.choices[0].message["content"]這是課堂上設置為了方便運用API而包裹的函式。作為指示輸入的prompt被我們用巢狀結構：串列包裝字典(dict in list)的形式包裝起來，以符合API要求的輸入形式(即message參數項)。從這裡可以看出模型是根據輸入的messages來輸出結果。此外，這個字典裡有role和prompt兩個key。我們在第一篇的時候有稍微帶過這件事，還記得嗎?這邊要揭開伏筆了。
定義一個函數，幫助我們省去每次輸入和輸出都需要寫一次設定，以及呼叫API的程式碼。我們輸入prompt，get_completion會幫我們呼叫API和模型溝通，並把模型的回答回覆給我們。從這段程式碼可以看到：
模型使用的是gpt-3.5-turbo
可以更改發出訊息的角色(role)：實際上，除了user，還有system和assistant共三種角色
細節可以到chat completions觀看。
system是高階的角色，作為提示模型的訊息。意即，在模型開始接收輸入訊息前，它會先根據這段訊息來決定怎麼處理輸入訊息。user顧名思義，是使用者輸入的訊息，也就是模型要處理的。在openAI網頁介面輸入的訊息，都是user角色。assistant就是模型本身。
所以，作為開發者，如果想引導模型在回覆或處理訊息上有什麼方向，或者語氣、該注意的地方，可以用system的角色寫入訊息。這樣模型就可以不著痕跡地回覆訊息，而帶有你希望它具有的風格，使用者卻不自知。這表示開發者是可以引導資訊的。
我們改寫一下上面的函式，把巢狀結構取消，讓函式只能接受符合巢狀格式的輸入，而不是單純的指示輸入(原本的輸入格式是字串)：
def get_completion_from_messages(messages, model="gpt-3.5-turbo", temperature=0):
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=temperature, # this is the degree of randomness of the model's output
)
# print(str(response.choices[0].message))
return response.choices[0].message["content"]這樣的好處是，我們可以較有彈性的改寫message的內容，例如：
messages = [
{'role':'system', 'content':'You are an assistant that speaks like Shakespeare.'},
{'role':'user', 'content':'tell me a joke'},
{'role':'assistant', 'content':'Why did the chicken cross the road'},
{'role':'user', 'content':'I don\'t know'} ]在system角色，提示模型回答需帶有莎翁風格。同時我們放了一段前導對話，「請告訴我一個笑話」「為什麼雞過馬路」「我不知道」。然後帶入函式，預期模型要以assistant的角色回答：
response = get_completion_from_messages(messages, temperature=1)
print(response)把temperature改到1，希望增加回答的多樣性。得到的輸出如下：
To get to the other side, verily!
好吧，我果然不適合外文系，不知道笑點。我們把message完整顯示出來好了：
{
“content”: “It did so to get to the other side, good sir!”,
“role”: “assistant”
}
確認模型是以assistant的角色回覆訊息。注意，user是使用者，所以這樣的對話方式，message串列中的最後一個字典的role key，對應的值須為user(messages[-1][‘role’] == ‘user’)比較合理。雖然，最後一個是assitant也可以順利執行就是了。
再看一個例子。
messages = [
{'role':'system', 'content':'You are friendly chatbot.'},
{'role':'user', 'content':'Yes, can you remind me, What is my name?'} ]
response = get_completion_from_messages(messages, temperature=1)
print(response)在之前沒有告訴模型我的名字的情況下，請模型提醒我自己的名字。得到回應如下：
I’m sorry, but as an AI language model, I do not have access to your personal information like your name. Could you please tell me your name?
這表示模型不知道的會說沒有，至少忠於事實。另外還表示：模型確實會根據輸入的前綴對話來決定如何回覆。這個例子裡我們沒有任何前綴對話。如果我改成：
messages = [
{'role':'system', 'content':'You are friendly chatbot.'},
{'role':'user', 'content':'I\'m Martin.'},
{'role':'assistant', 'content':'Nice to meet you, Martin.'},
{'role':'user', 'content':'Yes, can you remind me, What is my name?'} ]
response = get_completion_from_messages(messages, temperature=1)
print(response)得到回覆
Your name is Martin.
過濾對話中的訊息
接下來要玩真的了。以下的例子，以我實作linebot的經驗，是要花個幾天的。這裡一個prompt搭配協助包裹API的自訂函式就能搞定，讓我有點不能接受…
這個例子的場景是：一間披薩餐廳想要利用ChatGPT建立一個線上訂單互動系統。餐廳把價目表交給ChatGPT，並訂出交涉流程：需確認披薩的大小，以及各品項的個數。在得到訂單之後，確認如何取餐(外送或外帶)，並最終確認一次訂單。然後告訴對方價格。
如果是用linebot，我想到的流程是：利用template設定價目表，讓客戶選擇之後再確認大小以及數量。價目表必須放在後台資料庫，讓linebot主程式可以串接。最後確認取餐方式、重複一次訂單以確認是否要修改，最後告知金額。整個過程比較難處理的是在最後確認訂單的時候，因為要列出品項，還要讓對方修改或刪除，以line的訊息設計限制，有時候會比較複雜。
但在chatGPT，通通放在prompt裡面就可以了。
先用一個執行核心函式：
def collect_messages(_):
prompt = inp.value_input
inp.value = ''
context.append({'role':'user', 'content':f"{prompt}"})
response = get_completion_from_messages(context)
context.append({'role':'assistant', 'content':f"{response}"})
panels.append(
pn.Row('User:', pn.pane.Markdown(prompt, width=600)))
panels.append(
pn.Row('Assistant:', pn.pane.Markdown(response, width=600, style={'background-color': '#F6F6F6'})))
return pn.Column(*panels)這個函式除了前面的get_completion_from_message和模型互動之外，還利用panel套件來產生介面，紀錄使用者和模型的訊息，並把他們都即時顯示出來。接下來是系統訊息。
context = [ {‘role’:’system’, ‘content’:”””
You are OrderBot, an automated service to collect orders for a pizza restaurant.
You first greet the customer, then collects the order,
and then asks if it’s a pickup or delivery.
You wait to collect the entire order, then summarize it and check for a final
time if the customer wants to add anything else.
If it’s a delivery, you ask for an address.
Finally you collect the payment.
Make sure to clarify all options, extras and sizes to uniquely
identify the item from the menu.
You respond in a short, very conversational friendly style.
The menu includes…(以下略)}]
我們提示模型的角色、和使用者互動的流程、語氣，最後附上價目表。值得一提的是，價目表不需要特別標示食物或飲料的大中小尺寸，只要按同樣順序列出價格，模型就懂了。這點蠻神奇的。
然後是介面的執行程式。
import panel as pn # GUI
pn.extension()
panels = [] # collect display
context = [...] #上面那串
inp = pn.widgets.TextInput(value="Hi", placeholder='Enter text here…')
button_conversation = pn.widgets.Button(name="Chat!")
interactive_conversation = pn.bind(collect_messages, button_conversation)
dashboard = pn.Column(
inp,
pn.Row(button_conversation),
pn.panel(interactive_conversation, loading_indicator=True, height=300),
)
dashboard利用context收錄全部的對話紀錄，等一下可以做後處理。執行之後的介面長這樣：
披薩店開張啦！
隨意的輸入都是可以的，但當然，它的主要任務是訂餐。我們還是專心玩訂餐的指令就好了。
User:
Assistant:
Hello! Welcome to our pizza restaurant. What can I get for you today?
User:
Is today Friday?
Assistant:
I’m sorry, but as an AI language model, I don’t have access to real-time information. However, I’m here to take your order if you’d like to place one!
User:
How to get to your restaruant?
Assistant:
I’m sorry, but as an AI language model, I don’t have access to real-time information about our restaurant’s location. However, you can search for our restaurant’s address on Google Maps or our website. Is there anything else I can help you with?
這邊可以看到有點小bug。一開始使用者還沒輸入，所以應該是從assitant開始才對。此外，如果問餐廳地址，因為是虛構的，所以模型沒辦法告訴你要怎麼去。
User:
2 eggplant pizza, with mushroom toppings and AI sauce, pepper. I also want 2 sprite
Assistant:
Great! So that’s 2 eggplant pizzas with mushroom toppings and AI sauce, and peppers. And 2 sprites. Is that correct?
User:
yes
Assistant:
Got it! Would you like to pick up your order or have it delivered?
User:
delivered
Assistant:
Sure thing! Can you please provide me with your delivery address?
User:
Taipei City, Thankyou Rd. Section 3, №20
Assistant:
Thank you for providing your delivery address. Your total comes out to be $39.90, including the delivery fee. Is there anything else you would like to add to your order?
這裡有比較大的問題。模型沒問我餐點的尺寸就直接計算價格了！這樣我就不知道實際上是算我哪種尺寸的價格。
User:
You didn’t ask me what kind of size of pizza and sprite I’d like to have.
Assistant:
I apologize for the confusion. What size of pizza and sprite would you like to order? We have small, medium, and large sizes available for pizzas, and small, medium, and large sizes available for drinks.
User:
Small pizza and small drinks.
Assistant:
Thank you for letting me know. So that’s 2 small eggplant pizzas with mushroom toppings and AI sauce, and peppers, and 2 small sprites. Your total is still $39.90, including the delivery fee. Is there anything else you would like to add to your order?
User:
I’d like large pizza but small drink
Assistant:
Sure thing! So that’s 2 large eggplant pizzas with mushroom toppings and AI sauce, and peppers, and 2 small sprites. Your new total is $47.90, including the delivery fee. Is there anything else you would like to add to your order?
User:
No. That’s all.
Assistant:
Great! Your order is 2 large eggplant pizzas with mushroom toppings and AI sauce, and peppers, and 2 small sprites, to be delivered to Taipei City, Thankyou Rd. Section 3, №20. Your total is $47.90. Is there anything else I can help you with?
提醒之後，模型補救了。其餘的步驟都做得很確實。就成效來說已經相當有效率了，如同前面所說，如果是用linebot，整個設計不花個幾天是很難完成的。最後，餐廳人員再把這個對話紀錄裡面的訂單萃取出來。
messages = context.copy()
messages.append(
{'role':'system', 'content':'create a json summary of the previous food order. Itemize the price for each item\
The fields should be 1) pizza, include size 2) list of toppings 3) list of drinks, include size 4) list of sides include size 5)total price '},
)
#The fields should be 1) pizza, price 2) list of toppings 3) list of drinks, include size include price 4) list of sides include size include price, 5)total price '},
response = get_completion_from_messages(messages, temperature=0)
print(response)結果如下：
Sure, here’s a JSON summary of the previous food order:
```
{
“pizza”: [
{
“type”: “eggplant”,
“size”: “large”,
“toppings”: [
“mushrooms”,
“AI sauce”,
“peppers”
],
“price”: 11.95
},
{
“type”: “eggplant”,
“size”: “large”,
“toppings”: [
“mushrooms”,
“AI sauce”,
“peppers”
],
“price”: 11.95
}
],
“drinks”: [
{
“type”: “sprite”,
“size”: “small”,
“price”: 1.00
},
{
“type”: “sprite”,
“size”: “small”,
“price”: 1.00
}
],
“sides”: [],
“total_price”: 47.90
}
```Note that the price for each item is included in the JSON summary.
我們可以讓前面的指示更確實，以確保在下訂單的時候，模型有照我們的流程走，不會漏掉要問的資訊。另外，如果顧客要問價目表，也可以根據顧客想看的項目作顯示。我修改如下：
You are OrderBot, an automated service to collect orders for a pizza restaurant. \
You have to follow the instructions as steps:
-Greet the customer. \
-Collect order. \
-Ask the size of items customer ordered, if they have different prices by size. \Make sure to clarify all options, extras and sizes to uniquely \
identify the item from the menu.\
You respond in a short, very conversational friendly style. \
When customer asks about prices, show the prices in table. \
The row of the table is the item customer concerns, and the columns \
are the sizes.-Ask if it’s a pickup or delivery. \
-Summarize order and check for a final \
time if the customer wants to add anything else. \
-If it’s a delivery, you ask for an address. \
-Finally you collect the payment.\The menu includes…(以下略)
互動如下：
Assistant:
Hello! Welcome to our pizza restaurant. How can I assist you today?
User:
2 cheese pizza, 3 coke.
Assistant:
Great! Would you like to know the different sizes and prices for the cheese pizza? We have small for $6.50, medium for $9.25, and large for $10.95.
User:
2 small cheese pizza
Assistant:
Got it! And would you like any toppings on your cheese pizzas? We have extra cheese for $2.00, mushrooms for $1.50, sausage for $3.00, Canadian bacon for $3.50, AI sauce for $1.50, and peppers for $1.00.
User:
no
Assistant:
Sure thing! And would you like to pick up your order or have it delivered?
User:
pickup
Assistant:
Great! So to confirm, you ordered 2 small cheese pizzas and 3 cokes for pickup. Is there anything else you would like to add to your order?
User:
what is the price of coke
Assistant:
Sure! Here is the price for coke based on size:
Small: $1.00
Medium: $2.00
Large: $3.00
You ordered 3 cokes, so the total cost for the drinks is $9.00.
User:
small coke, please
Assistant:
I apologize for the confusion earlier. So to confirm, you ordered 2 small cheese pizzas and 1 small coke for pickup. Is there anything else you would like to add to your order?
User:
3 small coke
Assistant:
Got it! So you ordered 2 small cheese pizzas and 3 small cokes for pickup. The total cost for your order is $19.50. Would you like to add anything else to your order?
User:
no
Assistant:
Alright then! Your order of 2 small cheese pizzas and 3 small cokes for pickup is all set. Please come to our restaurant to pick up your order. Thank you for choosing our pizza restaurant!
我同時把第一行還沒輸入時會顯示user也修改掉了。這個修改過的prompt，秉持按部就班原則，清楚指示模型該做什麼，也告訴模型，在客人詢問價格時，以表格列出對應價目。看起來比一開始好一些，但在飲料的部分，模型還是忘了問，就直接算最高價了(老闆：可以，你錄取了！)
結語
總結一下在這堂課學到的東西吧。
- 輸入指示的要訣：
(1) 清楚而明確的指示
(2) 給模型時間「思考」 - 用迭代的方式逐步修正模型的表現
- 核心能力：總結/推論/轉換/擴充
- 總結以上，建立一個問答機器人
千里之行始於足下，其實不見得是為了什麼目的才開始使用ChatGPT。相信模型訓練出來的時候，也不見得是為了什麼遠大的目標，就像寫程式一樣。玩玩看，或許會有不一樣的心得！
歡迎分享這些心得整理。
