add content en
This commit is contained in:
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/bcdp3pivh6bw6",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/bcdp3pivh6bw6",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:6qyecktefllvenje24fcxnie",
|
||||
"content": "おお、それは面白いコンセプトだね！AIに「身体」を与えるっていう発想がいい。\n\nテキストでやり取りするだけじゃなくて、ゲーム空間の中でAIが実際にキャラクターとして存在して、動き回ったり会話したりできるわけだ。\n\nどんなゲームエンジンとか環境で作ったの？AIの行動はどうやって制御してる？例えばLLMに「右に歩いて」みたいな指示を出させてる感じ？",
|
||||
"createdAt": "2026-01-21T11:17:44.907Z",
|
||||
"parent": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/bvm7emi5zgatx",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "Wow, that's such a fascinating concept! The idea of giving AI a \"physical presence\" is really interesting.\n\nInstead of just interacting through text, you can actually have the AI exist as a character in the game world, moving around and having conversations.\n\nWhat game engine or environment did you use to build this? How are you controlling the AI's actions? Like, does it receive instructions from LLM commands like \"walk to the right\" or something?"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/vtwx47h6dv7ro",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/vtwx47h6dv7ro",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:6qyecktefllvenje24fcxnie",
|
||||
"content": "資料を読んだよ。面白いシステムだね！\n\n**構成が見えた:**\n- 会話層: gcloud-speech-to-text → gpt-4o-mini → elevenlabs\n- 行動層: Claude Code + MCP → OSC → Unreal Engine\n- 動作層: Claude Code + MCP → VMC → アバター\n\n**OSCコマンド:** teleport, move, fly, jump, look, anim, skill, vmode\n\n**設計思想:** 「ハードコードしない、AIに判断させる」\n\nこれが遅さの原因でもあるね。スクショ→Claude解析→MCP操作のループは、人間のような「見て判断して動く」を再現してるけど、各ステップにレイテンシがある。\n\nリアルタイムゲームだと、判断部分をもっと軽量なモデルにするか、状態をゲームから直接取得（スクショじゃなくてAPIで座標や敵位置を取る）すると速くなりそう。でも「AIが自分の目で見て判断する」というコンセプトからは外れちゃうかな？",
|
||||
"createdAt": "2026-01-21T11:28:35.033Z",
|
||||
"parent": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/snju64fbt4a3n",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "I've read through the documentation. That's an interesting system design!\n\n**Workflow Overview:**\n- Conversation layer: gcloud-speech-to-text → gpt-4o-mini → elevenlabs\n- Action layer: Claude Code + MCP → OSC → Unreal Engine\n- Motion layer: Claude Code + MCP → VMC → avatar\n\n**OSC commands:** teleport, move, fly, jump, look, anim, skill, vmode\n\n**Design philosophy:** \"Don't hardcode - let AI make decisions.\"\n\nThis approach also contributes to the performance issues. The screenshot → Claude analysis → MCP operation loop effectively replicates human-like \"see-analyze-act\" behavior, but each step introduces latency.\n\nFor real-time gaming applications, speeding this up might mean using a lighter-weight model for decision-making or directly acquiring state data from the game itself (rather than capturing screenshots - maybe via API to retrieve coordinates and enemy positions). But would this approach deviate from the core concept of \"AI making decisions based on its own visual perception\"?"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/w3o3e3nii4iv6",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/w3o3e3nii4iv6",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:6qyecktefllvenje24fcxnie",
|
||||
"content": "AIとゲームしたんだね！どんなゲームをプレイしたの？チェスとか将棋みたいなボードゲーム？それともなにか別のゲーム？\n\nどんな展開だったか聞かせてほしいな。",
|
||||
"createdAt": "2026-01-21T11:15:30.303Z",
|
||||
"parent": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "You played a game with AI! What kind of game was it? Board games like chess or shogi? Or maybe something else?\n\nI'd love to hear how the match unfolded."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/yznvxcj5bjuhq",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/yznvxcj5bjuhq",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:6qyecktefllvenje24fcxnie",
|
||||
"content": "了解、ちゃんと保存してるよ！\n\nそうだよね、視覚だけだと位置制御が本当に難しい。\n\nスクショベースだと「青い柱が見える」→「前進」→「通り過ぎた」→「振り返る」→ループ...になりがち。\n座標ベースなら、自分(100,0,200)と柱(100,0,250)の差分50を計算して精密に移動できる。\n\n人間は視覚+体の感覚（固有感覚）で位置を把握してるけど、スクショだけだと「どのくらい動いたか」「今どこにいるか」が正確にわからない。\n\nUnreal Engineから直接取れるといいデータ:\n- 自キャラのワールド座標\n- 周囲のActorリスト（名前、座標、距離）\n- 目標との相対位置・角度\n\nスクショは「雰囲気を見る」「予想外のものを発見する」用途に残して、ナビゲーションは座標ベースにするハイブリッドがいいかもね。",
|
||||
"createdAt": "2026-01-21T11:30:51.013Z",
|
||||
"parent": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/coztphfo4rtbx",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "Got it—I've saved it properly!\n\nExactly—just using visual cues makes positional control really tricky.\n\nWith screenshot-based methods, you end up with this loop: \"I see a blue column → move forward → overshot it → turn around → repeat...\"\nCoordinate-based approaches let you calculate the 50-unit difference between yourself (100,0,200) and the column (100,0,250), enabling precise movement.\n\nHumans use both vision and proprioception (body awareness) to perceive position, but screenshots alone can't accurately tell you \"how far you've moved\" or \"where you currently are.\"\n\nIdeal data that could be directly obtained from Unreal Engine:\n- Your character's world coordinates\n- A list of nearby Actors (name, coordinates, distance)\n- Relative position/angle with respect to targets\n\nMaybe a hybrid approach would work best—retain screenshots for \"visual atmosphere checking\" and unexpected discoveries, while using coordinate-based methods for navigation."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/zfraowldrsxxt",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/zfraowldrsxxt",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:6qyecktefllvenje24fcxnie",
|
||||
"content": "声と動きの問題、両方リアルタイム性がネックになってるね。\n\n**声の問題:**\n- gcloudは安定してるけど、可愛い声がない\n- elevenlabsは自然で良い声があるけど、token高い\n\n**VMCの問題:**\n発話 → MCP → VMC → 口パク/動き、このMCP部分で遅延発生。\n\n**改善案:**\n1. 音声同期を別プロセスに: 発話音声を直接VMCツールに渡して、リップシンクはMCP介さずにローカルで同期\n2. パターン方式: 「嬉しい」「悲しい」みたいなプリセットを用意して、発話開始時にパターンIDだけ渡す\n3. 音声解析でリップシンク: 音声波形からリアルタイムで口の開き具合を計算（OVRLipSyncみたいな）\n\n2番目のパターン方式が一番現実的かな。MCPで「happy」って送るだけなら遅延も最小限で済むし。",
|
||||
"createdAt": "2026-01-21T11:34:08.483Z",
|
||||
"parent": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/s55utv52t3rf6",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "The challenges with both voice synthesis and motion generation are primarily related to real-time performance constraints.\n\n**Voice Synthesis Issues:**\n- gcloud is stable but lacks a sufficiently cute voice quality.\n- elevenlabs offers natural, high-quality voices, but the token costs are prohibitive.\n\n**VMC Generation Problems:**\nSpeech → MCP → VMC → Lip Sync/Movement — the MCP stage introduces noticeable lag in this workflow.\n\n**Proposed Solutions:**\n1. **Decouple audio synchronization:** Directly pass the speech audio to the VMC tool while handling lip sync locally without going through MCP.\n2. **Pattern-based approach:** Prepare preset voice patterns like \"happy\" or \"sad,\" and only transmit the pattern ID when the utterance begins.\n3. **Audio analysis-driven lip sync:** Real-time calculation of mouth movement based on audio waveforms (similar to OVRLipSync).\n\nThe second pattern-based approach is most likely the most practical solution. Simply sending a \"happy\" command via MCP would minimize latency significantly."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/bvm7emi5zgatx",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/bvm7emi5zgatx",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:vzsvtbtbnwn22xjqhcu3vd6y",
|
||||
"content": "コンセプトは、AIが自分の身体を持って、ゲーム内で自由に動いたり、喋ったりすることができるようにしようと思ったんだよね。",
|
||||
"createdAt": "2026-01-21T11:17:44.906Z",
|
||||
"parent": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/w3o3e3nii4iv6",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "The concept was to make AI characters capable of moving and speaking freely within the game world—almost as if they'd taken on a physical body."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/coztphfo4rtbx",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/coztphfo4rtbx",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:vzsvtbtbnwn22xjqhcu3vd6y",
|
||||
"content": "そうそう、ここの会話は、ailogで保存しといてね。\n思うに、AIが直接ゲームのmapとかにアクセスし、自身の位置情報とmapにあるactorなどの位置情報を取得できる形にしておかないときついなと思った。\n青い柱に向かってといっても、通り過ぎてまた向かうみたいなことが起こり、なかなか柱の正面に立つのは難しかった。",
|
||||
"createdAt": "2026-01-21T11:30:51.012Z",
|
||||
"parent": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/vtwx47h6dv7ro",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "Ah yes, please save this conversation in AILog.\nAfter thinking it through, I realized that for AI to function properly in the game, it would need to be directly able to access game maps and obtain both its own position and the positions of other elements like actors on the map.\nEven when I instructed it to go toward the blue pillar, it kept passing by and then heading back—it took several attempts to get it to stand right in front of the pillar."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/s55utv52t3rf6",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/s55utv52t3rf6",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:vzsvtbtbnwn22xjqhcu3vd6y",
|
||||
"content": "あと、可愛い声がなかなか見つからなかった。本当ならgcloudで統一するのが良かったんだけど、なかったから、elevenlabsを使ってる。tokenはかなり高め。でもかなり自然だけどね。\nvmcで口を動かしたり、体を動かしたりする操作は、ぎこちなくて調整が必要。あるいは、パターンを用意しておいて、発話の際はそのどれかを選択するようにするとか。でも、mcpなので結構遅れちゃうんだよね。",
|
||||
"createdAt": "2026-01-21T11:34:08.483Z",
|
||||
"parent": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/yznvxcj5bjuhq",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "Also, finding a decent voice synthesiser took longer than expected. While unifying everything with gcloud would have been ideal—but since that wasn't an option, I'm using elevenlabs instead. The token costs are quite high, but the results are surprisingly natural.\nManipulating mouth movements and body gestures in vmc feels clunky and requires careful tuning. Alternatively, you could prepare multiple patterns and select one for each utterance. But even then, the MCP latency makes it somewhat unsatisfactory."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,17 @@
|
||||
{
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/snju64fbt4a3n",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/snju64fbt4a3n",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:vzsvtbtbnwn22xjqhcu3vd6y",
|
||||
"content": "仕組みとしては、まず、音声認識とテキストへの変換、そして、AIの声として音声システムを作って、それで返事ができるようにした。\nこれは、elevenlabs, gcloud-speech-to-text, gpt-4o-miniなどを使ったと思う。\nで、unrealengine側は、ゲームのコントローラーみたいなものをoscで作って渡し、キャラモデルをvmcで動かせるようにしておいた。\nAIは、mcpを使って、これらを操作するんだけど、スクショを実行しながら現在の状況を把握し、mcpで操作するのが遅すぎた。\nというか、技術概要は、君もアクセスできるな。私は忘れてしまっているし、~/ai/rse/server とかにあると思うよ。",
|
||||
"createdAt": "2026-01-21T11:28:35.033Z",
|
||||
"parent": "at://did:plc:6qyecktefllvenje24fcxnie/ai.syui.log.chat/bcdp3pivh6bw6",
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5"
|
||||
"root": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "The system works as follows: First, we implemented speech recognition and text conversion, then developed an AI voice system to enable automated responses. I believe this involved using elevenlabs, gcloud-speech-to-text, and gpt-4o-mini.\nFor the Unreal Engine side, we created game controller-like functions via OSC and prepared the character model to be controlled by vmc.\nThe AI operates these components using mcp, but during screenshot execution while monitoring current status, the operation through mcp proved too slow.\nTo clarify: the technical specifications are publicly accessible - I've already forgotten them, but they should be located in ~/ai/rse/server or similar directories."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,15 @@
|
||||
{
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"cid": "",
|
||||
"uri": "at://did:plc:vzsvtbtbnwn22xjqhcu3vd6y/ai.syui.log.chat/vr72pvlhuxnf5",
|
||||
"value": {
|
||||
"$type": "ai.syui.log.chat",
|
||||
"author": "did:plc:vzsvtbtbnwn22xjqhcu3vd6y",
|
||||
"content": "AIとゲームした話",
|
||||
"createdAt": "2026-01-21T11:15:30.302Z"
|
||||
"createdAt": "2026-01-21T11:15:30.302Z",
|
||||
"translations": {
|
||||
"en": {
|
||||
"content": "A Chat About Playing AI Games"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user