feat: publish workphone SDK deployment and API docs
This commit is contained in:
186
sdk/agent/skills/README.md
Normal file
186
sdk/agent/skills/README.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# Agent Skills 技能系统
|
||||
|
||||
## 📋 技能列表
|
||||
|
||||
### 1. BaseSkill - 基础技能
|
||||
所有技能的基类,提供通用UI操作方法。
|
||||
|
||||
**功能:**
|
||||
- ✅ APP启动/关闭
|
||||
- ✅ 点击/输入/滑动
|
||||
- ✅ 元素查找
|
||||
- ✅ 截图
|
||||
- ✅ **搜索功能**(新增)
|
||||
- ✅ **复合命令执行**(新增)
|
||||
- ✅ **智能等待**(新增)
|
||||
|
||||
### 2. VoiceControlSkill - 语音控制技能
|
||||
解析和执行语音命令,支持复合命令。
|
||||
|
||||
**功能:**
|
||||
- ✅ 语音命令解析
|
||||
- ✅ 复合命令拆分("打开豆包,搜索今天去哪")
|
||||
- ✅ 搜索关键词提取
|
||||
- ✅ 自动执行操作序列
|
||||
|
||||
**示例:**
|
||||
```python
|
||||
skill = VoiceControlSkill(device)
|
||||
result = skill.execute_voice_command("打开豆包,搜索今天去哪")
|
||||
```
|
||||
|
||||
### 3. AppManagerSkill - 应用管理技能
|
||||
管理应用的打开、关闭、切换。
|
||||
|
||||
**功能:**
|
||||
- ✅ 打开应用(支持中文名称)
|
||||
- ✅ 关闭应用
|
||||
- ✅ 切换应用
|
||||
- ✅ 获取运行中的应用
|
||||
- ✅ 获取已安装应用列表
|
||||
|
||||
**示例:**
|
||||
```python
|
||||
skill = AppManagerSkill(device)
|
||||
result = skill.open_app("微信")
|
||||
result = skill.switch_app("豆包")
|
||||
```
|
||||
|
||||
### 4. SearchSkill - 通用搜索技能
|
||||
在任何APP内执行搜索操作。
|
||||
|
||||
**功能:**
|
||||
- ✅ 智能查找搜索框
|
||||
- ✅ 输入搜索关键词
|
||||
- ✅ 执行搜索
|
||||
- ✅ 支持语音输入(中文)
|
||||
|
||||
**示例:**
|
||||
```python
|
||||
skill = SearchSkill(device)
|
||||
result = skill.search_in_app("今天去哪", app_package="com.bytedance.doubao")
|
||||
```
|
||||
|
||||
### 5. WechatSkill - 微信技能(已优化)
|
||||
微信操作技能,支持复合任务。
|
||||
|
||||
**新增功能:**
|
||||
- ✅ `send_message_with_search()` - 通过搜索发送消息
|
||||
- ✅ `execute_compound_task()` - 执行复合任务("给张三发消息:你好")
|
||||
- ✅ 使用`wait_for_app_ready()`等待应用加载
|
||||
|
||||
**示例:**
|
||||
```python
|
||||
skill = WechatSkill(device)
|
||||
# 方式1:普通发送
|
||||
result = skill.send_message("张三", "你好")
|
||||
|
||||
# 方式2:通过搜索发送(更可靠)
|
||||
result = skill.send_message_with_search("张三", "你好")
|
||||
|
||||
# 方式3:复合任务
|
||||
result = skill.execute_compound_task("给张三发消息:下午开会")
|
||||
```
|
||||
|
||||
### 6. DouyinSkill - 抖音技能(已优化)
|
||||
抖音操作技能。
|
||||
|
||||
**优化:**
|
||||
- ✅ 使用`wait_for_app_ready()`等待应用加载
|
||||
- ✅ 改进错误处理
|
||||
|
||||
---
|
||||
|
||||
## 🚀 使用方式
|
||||
|
||||
### 方式1:直接使用Skill
|
||||
|
||||
```python
|
||||
from agent.skills import WechatSkill, VoiceControlSkill
|
||||
|
||||
# 初始化
|
||||
wechat = WechatSkill(device)
|
||||
voice = VoiceControlSkill(device)
|
||||
|
||||
# 执行操作
|
||||
result = wechat.send_message("张三", "你好")
|
||||
result = voice.execute_voice_command("打开微信")
|
||||
```
|
||||
|
||||
### 方式2:使用SkillExecutor(推荐)
|
||||
|
||||
```python
|
||||
from agent.skill_executor import SkillExecutor
|
||||
|
||||
executor = SkillExecutor(device)
|
||||
|
||||
# 自动选择合适的Skill执行
|
||||
result = executor.execute_command("打开豆包,搜索今天去哪")
|
||||
result = executor.execute_wechat_task("给张三发消息:下午开会")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 支持的语音命令
|
||||
|
||||
### 应用操作
|
||||
- "打开微信"
|
||||
- "打开豆包"
|
||||
- "打开抖音"
|
||||
- "切换到微信"
|
||||
|
||||
### 复合命令
|
||||
- "打开豆包,搜索今天去哪"
|
||||
- "打开微信,给张三发消息:你好"
|
||||
- "打开抖音,向上滑动"
|
||||
|
||||
### 导航操作
|
||||
- "返回"
|
||||
- "回到桌面"
|
||||
- "最近任务"
|
||||
|
||||
### 滑动操作
|
||||
- "向上滑"
|
||||
- "向下滑"
|
||||
- "左滑"
|
||||
- "右滑"
|
||||
|
||||
### 系统操作
|
||||
- "截图"
|
||||
- "锁屏"
|
||||
|
||||
---
|
||||
|
||||
## 🔧 技能注册
|
||||
|
||||
所有技能在`skills/__init__.py`中注册:
|
||||
|
||||
```python
|
||||
SKILL_REGISTRY = {
|
||||
"wechat": WechatSkill,
|
||||
"douyin": DouyinSkill,
|
||||
"xhs": XhsSkill,
|
||||
"voice_control": VoiceControlSkill,
|
||||
"app_manager": AppManagerSkill,
|
||||
"search": SearchSkill,
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 最佳实践
|
||||
|
||||
1. **使用SkillExecutor**:自动选择合适的Skill
|
||||
2. **复合命令**:使用语音控制技能处理复杂任务
|
||||
3. **错误处理**:检查返回的`success`字段
|
||||
4. **等待应用**:使用`wait_for_app_ready()`确保应用加载完成
|
||||
|
||||
---
|
||||
|
||||
## 📚 相关文档
|
||||
|
||||
- `base.py` - 基础技能类
|
||||
- `skill_executor.py` - 技能执行器
|
||||
- `voice_control.py` - 语音控制实现
|
||||
- `app_manager.py` - 应用管理实现
|
||||
- `search.py` - 搜索功能实现
|
||||
Reference in New Issue
Block a user