五、帧图像人体识别

原创

alphaair

发布于 2024-05-09 09:28:19

730

发布于 2024-05-09 09:28:19

随着深度学习推理技术的不断发展，让小型设备运行深度学习成为可能，阿里体育等IT大厂，推出的“乐动力”、“天天跳绳”AI运动APP，让云上运动会、线上运动会、健身打卡、AI体育指导等概念空前火热。那么，能否将这些在APP成功应用的场景搬上微信小程序，分享这些概念的红利呢？本系列文章就带您一步一步从零开始开发一个AI运动小程序，本系列文章将使用“AI运动识别”小程序插件，插件详情可以前往微信服务市场搜索相应插件。

一、初始化人体识别功能

人体识别能力调用前需要初始化（可多次初始化），插件的具有双人体识别引擎，默认不指定引擎将自动选择最优引擎，也可指定引擎，识别引擎的差别请参考集成文档。

const that = this;
humanDetection.initialize({
	//ve: 2,
	callback(err) {
		uni.hideLoading();

		if (!err) {
			console.log('人体检测能力初始化成功。', humanDetection.getVe());
			return;
		}

		uni.showModal({
			content: `初始化失败，详细信息：${err.message}`,
			showCancel: false
		});
	}
});

二、调用人体识别功能

//接上篇抽帧
const context = wx.createCameraContext();
const listener = context.onCameraFrame((frame) => {
	//frame.data图像数组将用于后续的人体识别
	const image = {
		width: Number(frame.width),
		height: Number(frame.height),
		data: new Uint8Array(frame.data)
	};
	humanDetection.detectionAsync(image).then(human=>{
		console.log(human);
	});
});
listener.start();

三、人体识别结果

{
  "score": 0.7251838252824896,
  "width": 480,
  "height": 640,
  "timestamp": 1684325583506,
  "keypoints": [
    {
      "y": 103.56424502483996,
      "x": 256.0109768927012,
      "score": 0.4570312798023224,
      "name": "nose"
    },
    {
      "y": 90.71574411931093,
      "x": 270.50707531981544,
      "score": 0.58251953125,
      "name": "left_eye"
    },
    {
      "y": 90.09307140499885,
      "x": 244.60145098668588,
      "score": 0.552734375,
      "name": "right_eye"
    },
    {
      "y": 95.37442872749463,
      "x": 286.88540601464354,
      "score": 0.75244140625,
      "name": "left_ear"
    },
    {
      "y": 93.90985185987653,
      "x": 231.85045311881774,
      "score": 0.7392578125,
      "name": "right_ear"
    },
    {
      "y": 162.78484030178413,
      "x": 311.0556351088547,
      "score": 0.919921875,
      "name": "left_shoulder"
    },
    {
      "y": 161.2381417518693,
      "x": 203.38841526521654,
      "score": 0.77197265625,
      "name": "right_shoulder"
    },
    {
      "y": 240.78316880186404,
      "x": 310.24842737472574,
      "score": 0.84912109375,
      "name": "left_elbow"
    },
    {
      "y": 242.12329664294745,
      "x": 201.77353663841666,
      "score": 0.833984375,
      "name": "right_elbow"
    },
    {
      "y": 281.9979693520591,
      "x": 352.673287407275,
      "score": 0.59765625,
      "name": "left_wrist"
    },
    {
      "y": 286.2878520237733,
      "x": 173.8384814716242,
      "score": 0.63427734375,
      "name": "right_wrist"
    },
    {
      "y": 307.2371714929637,
      "x": 273.4654390815558,
      "score": 0.82177734375,
      "name": "left_hip"
    },
    {
      "y": 305.7596342955926,
      "x": 222.0470940485152,
      "score": 0.84912109375,
      "name": "right_hip"
    },
    {
      "y": 417.72422545441424,
      "x": 267.0201893540119,
      "score": 0.6103515625,
      "name": "left_knee"
    },
    {
      "y": 414.7774591985668,
      "x": 204.46947287192143,
      "score": 0.8701171875,
      "name": "right_knee"
    },
    {
      "y": 481.8868752974563,
      "x": 280.1792094531294,
      "score": 0.74072265625,
      "name": "left_ankle"
    },
    {
      "y": 467.38751675509405,
      "x": 247.29222232381437,
      "score": 0.7451171875,
      "name": "right_ankle"
    }
  ],
  "bodyRange": {
    "x": 173.8384814716242,
    "y": 90.09307140499885,
    "width": 178.8348059356508,
    "height": 391.79380389245745
  }
}