数字人项目中，有时用户上传的人脸素材不符合数字人的要求，项目中就需要对人脸进行检测，探测人脸姿态：左右、上下，头部倾斜角度，如果人脸的左右、上下，头部倾斜的角度大于一定阈值时，就提示用户重新上传素材，而不是等待人工审核后发现不符合要求，再驳回。
飘易这里采用的是开源项目 SynergyNet：
https://github.com/choyingw/SynergyNet
可探测人脸姿态：左右、上下，头部倾斜角度。
下载 https://github.com/choyingw/SynergyNet/archive/refs/heads/main.zip 并解压
conda create --name SynergyNet conda activate SynergyNet
PyTorch 1.9 (should also be compatiable with 1.0+ versions), Torchvision, Opencv, Scipy, Matplotlib, Cython
我安装的是：
torch 2.0.0+cu118（可先离线下载whl文件，再安装） torchaudio 2.0.1+cu118 torchvision 0.15.1+cu118
pip install torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
Download data：
https://drive.google.com/file/d/1YVBRcXmCeO1t5Bepv67KVr_QKcOur3Yy/view?usp=sharing
https://drive.google.com/file/d/1SQsMhvAmpD1O8Hm0yEGom0C0rXtA0qs8/view?usp=sharing
Extract these data under the repo root下载后解压到项目根目录.
These data are processed from [3DDFA] and [FSA-Net].
Download pretrained weights下载与训练权重文件：
https://drive.google.com/file/d/1BVHbiLTfX6iTeJcNbh-jgHjWDoemfrzG/view?usp=sharing
Put the model under 'pretrained/'
cd Sim3DR ./build_sim3dr.sh cd ../FaceBoxes ./build_cpu_nms.sh
【这里注意，windows下不能直接执行，需变通】：
【编译 Sim3DR】：
cd Sim3DR python setup.py build_ext --inplace 提示（下面有一些警告，不影响使用）： Compiling lib/rasterize.pyx because it changed. [1/1] Cythonizing lib/rasterize.pyx E:\conda\facecheck\lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: E:\conda\facecheck\Code\Sim3DR\lib\rasterize.pyx tree = Parsing.p_module(s, pxd, full_module_name) cl: 命令行 warning D9002 :忽略未知选项“-std=c++11” rasterize.cpp E:\conda\facecheck\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION cl: 命令行 warning D9002 :忽略未知选项“-std=c++11” rasterize_kernel.cpp lib/rasterize_kernel.cpp(110): warning C4305: “=”: 从“double”到“float”截断 lib/rasterize_kernel.cpp(148): warning C4305: “=”: 从“double”到“float”截断 lib/rasterize_kernel.cpp(257): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 lib/rasterize_kernel.cpp(258): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 lib/rasterize_kernel.cpp(330): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 lib/rasterize_kernel.cpp(331): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 lib/rasterize_kernel.cpp(416): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 lib/rasterize_kernel.cpp(417): warning C4244: “=”: 从“int”转换到“float”，可能丢失数据 正在创建库 build\temp.win-amd64-cpython-310\Release\lib\Sim3DR_Cython.cp310-win_amd64.lib 和对象 build\temp.win-amd64-cpython-310\Release\lib\Sim3DR_Cython.cp310-win_amd64.exp 正在生成代码 已完成代码的生成
【编译FaceBoxes】：
cd FaceBoxes/utils/ python build.py build_ext --inplace
编译前，必须要先改下文件，不然无法编译通过：
1）windows环境下，打开build.py文件，修改47行：
extra_compile_args=["-Wno-cpp", "-Wno-unused-function"], #"-Wno-cpp", "-Wno-unused-function" 改为： extra_compile_args=['std=c99'],
2）打开 FaceBoxes\utils\nms\cpu_nms.pyx ：
搜索 np.int_t 替换为 np.int64_t
一定改完之后再编译，不然运行项目时会报错：
ValueError: Buffer dtype mismatch, expected 'int_t' but got 'long long'
返回（下面有一些警告，不影响使用）：
Compiling nms/cpu_nms.pyx because it changed. [1/1] Cythonizing nms/cpu_nms.pyx E:\conda\facecheck\lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: E:\conda\facecheck\Code\FaceBoxes\utils\nms\cpu_nms.pyx tree = Parsing.p_module(s, pxd, full_module_name) cl: 命令行 warning D9024 :无法识别的源文件类型“std=c99”，假定为对象文件 cl: 命令行 warning D9027 :源文件“std=c99”被忽略 cpu_nms.c E:\conda\facecheck\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION nms/cpu_nms.c(5191): warning C4244: “=”: 从“npy_intp”转换到“int”，可能丢失数据 nms/cpu_nms.c(5281): warning C4244: “=”: 从“__pyx_t_5numpy_int64_t”转换到“int”，可能丢失数据 nms/cpu_nms.c(5457): warning C4244: “=”: 从“__pyx_t_5numpy_int64_t”转换到“int”，可能丢失数据 nms/cpu_nms.c(5584): warning C4244: “函数”: 从“double”转换到“__pyx_t_5numpy_float32_t”，可能丢失数据 nms/cpu_nms.c(5594): warning C4244: “函数”: 从“double”转换到“__pyx_t_5numpy_float32_t”，可能丢失数据 nms/cpu_nms.c(5995): warning C4244: “=”: 从“npy_intp”转换到“unsigned int”，可能丢失数据 nms/cpu_nms.c(6255): warning C4018: “<”: 有符号/无符号不匹配 nms/cpu_nms.c(6766): warning C4018: “<”: 有符号/无符号不匹配 nms/cpu_nms.c(6899): warning C4244: “=”: 从“double”转换到“float”，可能丢失数据 nms/cpu_nms.c(6910): warning C4244: “=”: 从“double”转换到“float”，可能丢失数据 nms/cpu_nms.c(6931): warning C4244: “=”: 从“double”转换到“float”，可能丢失数据 nms/cpu_nms.c(6950): warning C4244: “=”: 从“double”转换到“float”，可能丢失数据 nms/cpu_nms.c(6993): warning C4244: “=”: 从“double”转换到“float”，可能丢失数据 正在创建库 build\temp.win-amd64-cpython-310\Release\nms\cpu_nms.cp310-win_amd64.lib 和对象 build\temp.win-amd64-cpython-310\Release\nms\cpu_nms.cp310-win_amd64.exp正在生成代码 已完成代码的生成
python singleImage.py -f img 或 python singleImage_simple.py
The default inference requires a compatible GPU to run. If you would like to run on a CPU, please comment the .cuda() and load the pretrained weights into cpu.
推理时遇到错误：
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
解决：
需要在 singleImage.py 或 singleImage_simple.py 文件顶部写入环境变量：
import os os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
我们先来写检测函数 face_dec：
# 初始化 Mediapipe Face Mesh
mp_face_mesh = mp.solutions.face_mesh
face_mesh = mp_face_mesh.FaceMesh(
static_image_mode=True, # 如果是视频改成 False
max_num_faces=1,
refine_landmarks=True,
min_detection_confidence=0.5,
)
# 检测人脸 mediapipe 方案
def dec_face(image, idnx=0):
#image = cv2.imread(r'test\a13.jpg') # 加载图片
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
h, w, _ = image.shape
# 检测人脸
results = face_mesh.process(image)
# 取 landmarks
if results.multi_face_landmarks:
landmarks = results.multi_face_landmarks[0].landmark
landmark_indices = [1, 133, 362, 78, 308, 152] # 鼻尖、眼睛、嘴角、下巴
# 提取几个关键点，适合估计头部姿态
image_points = np.array([
(landmarks[1].x * w, landmarks[1].y * h), # 鼻尖
(landmarks[133].x * w, landmarks[133].y * h), # 左眼内角
(landmarks[362].x * w, landmarks[362].y * h), # 右眼内角
(landmarks[78].x * w, landmarks[78].y * h), # 嘴左
(landmarks[308].x * w, landmarks[308].y * h), # 嘴右
(landmarks[152].x * w, landmarks[152].y * h) # 下巴
], dtype='double')
# 绘制关键点
if 1 == 1:
for (x, y) in image_points:
center_coordinates = (int(x), int(y)) # 像素坐标整数化
color = (0, 255, 0) # 绿色 (BGR格式)
thickness = 2 # 线宽
radius = 3 # 半径
cv2.circle(image, center_coordinates, radius, color, thickness)
for idx, (x, y) in enumerate(image_points):#打序号
center_coordinates = (int(x), int(y))
cv2.circle(image, center_coordinates, 3, (0, 255, 0), 2)
cv2.putText(image, str(idx), (int(x)+5, int(y)-5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 1)
cv2.imshow('Face with Key Points', image)#显示图像
cv2.waitKey(0)
cv2.destroyAllWindows()
# 世界坐标 face_mesh 3D标准模型
model_points = np.array([
(landmarks[idx].x * w, landmarks[idx].y * h, landmarks[idx].z * w)
for idx in landmark_indices
], dtype='double')
# 相机内参矩阵 - 自适应 focal_length
x_list = [lm.x for lm in landmarks]
y_list = [lm.y for lm in landmarks]
xmin = min(x_list) * w
xmax = max(x_list) * w
ymin = min(y_list) * h
ymax = max(y_list) * h
face_width = xmax - xmin
face_height = ymax - ymin
focal_length = 1.2 * max(face_width, face_height)
center = (w / 2, h / 2)
camera_matrix = np.array([
[focal_length, 0, center[0]],
[0, focal_length, center[1]],
[0, 0, 1]
], dtype='double')
dist_coeffs = np.zeros((4, 1)) # 假设无畸变
success, rotation_vector, translation_vector = cv2.solvePnP(
model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.SOLVEPNP_ITERATIVE)
# 将旋转向量转换成旋转矩阵
rmat, _ = cv2.Rodrigues(rotation_vector)
# 从旋转矩阵提取欧拉角 (pitch, yaw, roll)
sy = np.sqrt(rmat[0, 0] ** 2 + rmat[1, 0] ** 2)
singular = sy < 1e-6
if not singular:
x = np.arctan2(rmat[2, 1], rmat[2, 2])
y = np.arctan2(-rmat[2, 0], sy)
z = np.arctan2(rmat[1, 0], rmat[0, 0])
else:
x = np.arctan2(-rmat[1, 2], rmat[1, 1])
y = np.arctan2(-rmat[2, 0], sy)
z = 0
# 弧度转角度 - Pitch (上下抬头低头)\Yaw (左右转头)\Roll (头部倾斜)
pitch = float(np.degrees(x))
yaw = float(np.degrees(y))
roll = float(np.degrees(z))
print(f"{idnx}.Pitch (上下抬头低头): {pitch:.2f}, Yaw (左右转头): {yaw:.2f}, Roll (头部倾斜): {roll:.2f}")
# 判断人脸选择角度 大于45°非法
if abs(pitch) > 45 or abs(yaw) > 45 or abs(pitch) > 45:
print(f"{idnx}.Pitch (上下抬头低头): {pitch:.2f}, Yaw (左右转头): {yaw:.2f}, Roll (头部倾斜): {roll:.2f}")
return 1
return 0
else:
print(f"{idnx}.没有检测到人脸")
return -1然后来检测图片：
dec_face(cv2.imread(r'.\img\0109.png'))
随便拿几张图片来验证下：
Pitch (上下抬头低头): 9.87, Yaw (左右转头): 3.43, Roll (头部倾斜): 0.26 ↑
Pitch (上下抬头低头): -6.23, Yaw (左右转头): 0.87, Roll (头部倾斜): 0.22↑
Pitch (上下抬头低头): 1.10, Yaw (左右转头): 12.63, Roll (头部倾斜): 0.27↑
Pitch (上下抬头低头): -2.81, Yaw (左右转头): 8.94, Roll (头部倾斜): -0.17↑
Pitch (上下抬头低头): -32.13, Yaw (左右转头): -16.38, Roll (头部倾斜): -173.63↑
如果检测的是视频，需要把视频的所有帧进行遍历，逐帧判断每一个画面是否有人脸旋转的角度过大即可。