前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >cloudlflare推理llama2 - plus studio

cloudlflare推理llama2 - plus studio

作者头像
plus sign
发布2024-02-29 08:26:05
820
发布2024-02-29 08:26:05
举报
文章被收录于专栏:个人博客个人博客

cloudlflare推理llama2

最近,cloudlfare悄悄上线了一项新功能,全球网络上的gpu加速推理,显然的,我们可以用它推理llama2,cloudflare也提供了一个库进行推理。

新建一个cloudflare,然后,代码改成下面的,就可以进行推理了

代码语言:text
复制
import { Ai } from './vendor/@cloudflare/ai.js';

export default {
  async fetch(request, env) {
    const tasks = [];
    const ai = new Ai(env.AI);

    // Get the request body
    const requestBody = await request.json();

    // messages - chat style input
    let chat = {
      messages: [
        { role: 'system', content: 'You are a helpful, kind, honest, friendly, good at writing and never fails to answer my requests immediately and with details and precision.'},
        { role: 'user', content: requestBody.prompt }
      ]
    };
    let response = await ai.run('@cf/meta/llama-2-7b-chat-int8', chat);
    tasks.push({ inputs: chat, response });

    return Response.json(tasks);
  }
};

我们可以测试一下,在命令行运行下面的命令,将https://example.workers.dev/ 换成你的网址。

代码语言:text
复制
curl -X POST https://example.workers.dev/ -d '{"prompt":"Write a poem that talks about the connectivity cloud"}' 

看到类似下面的返回值代表成功

代码语言:text
复制
[{"inputs":{"messages":[{"role":"system","content":"You are a helpful, kind, honest, friendly, good at writing and never fails to answer my requests immediately and with details and precision."},{"role":"user","content":"Write a poem that talks about the connectivity cloud"}]},"response":{"response":"In the realm of the digital sky,\nWhere information flows, and data fly,\nThere's a place that brings us all together,\nA connectivity cloud, a true forever.\n\nIt's a space that's vast and wide,\nWhere thoughts and ideas collide,\nA hub of communication and exchange,\nWhere the world's voices all combine and blend.\n\nIn this cloud of connectivity,\nWe find our voices, our identity,\nA platform for sharing and growth,\nWhere our stories are told and our dreams take flight.\n\nWith just a click or tap of a key,\nWe can connect with anyone, anywhere,\nSharing laughter, love, and tears,\nIn this digital embrace, we all share.\n\nSo let us cherish this cloud of connectivity,\nThis gift that brings us all sovereignty,\nFor in its depths, we find our tribe,\nAnd our voices, heard, can never be denied.\n\nIn this cloud of connectivity,\nWe are all connected, you see,\nA global community, united and free,\nIn this digital age, where we all can be"}}]
本文参与?腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2023-10-11,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客?前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与?腾讯云自媒体同步曝光计划? ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • cloudlflare推理llama2
相关产品与服务
腾讯云服务器利旧
云服务器(Cloud Virtual Machine,CVM)提供安全可靠的弹性计算服务。 您可以实时扩展或缩减计算资源,适应变化的业务需求,并只需按实际使用的资源计费。使用 CVM 可以极大降低您的软硬件采购成本,简化 IT 运维工作。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档
http://www.vxiaotou.com