Performance Analysis of Vision-Language-Action Models: Evaluating Turkish Command Understanding in OpenVLA Görüntü-Dil-Aksiyon Modellerinin Çapraz Dil Performans Analizi: OpenVLA'da Türkçe Komut Anlayisinin Degerlendirilmesi Cross-Lingual
34th Signal Processing and Communications Applications Conference, SIU 2026, İstanbul, Türkiye, 7 - 10 Temmuz 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu71813.2026.11636432
- Basıldığı Şehir: İstanbul
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: cross-language analysis, multilingual evaluation, OpenVLA, robotic manipulation, Vision-Language-Action models
- İstanbul Ticaret Üniversitesi Adresli: Evet
Özet
Vision-Language-Action (VLA) models unify visual perception, language comprehension, and action generation under a single architecture, yet they are trained almost exclusively on English data. We evaluated the OpenVLA-7B model by issuing the same 50 manipulation commands in both Turkish and English and comparing the resulting action vectors. The paired measurements revealed substantial cross-lingual divergence: the mean L2 error reached 0.501 and cosine similarity dropped to 0.372, both statistically significant. No linear relationship between command complexity and performance degradation was observed. Although Turkish commands incurred a 30.8% token overhead, no meaningful latency difference was recorded. Our findings highlight a deep-seated deficiency in multilingual action generation within current VLA architectures.