ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions

Main Contributions

• We introduce , a unified framework for expressive and instruction-based image editing encompassing non-rigid motions.
• We present and , a large-scale dataset and a comprehensive benchmark, with high-quality image pairs for training and evaluation, addressing various dynamic editing scenarios, including camera motion, object transformation, human articulation, and human-object interaction.
• With the proposed dataset, we finetune , a DiT-based model specifically developed for instruction-guided, motion-centric image editing, setting a baseline for performance on non-rigid motion editing tasks.

ByteMorph Dataset Construction

Our methodology leverages video generation models to produce natural and coherent transitions between source and target images, ensuring edits are both realistic and motion-consistent.

Overview of Synthetic Data Construction. Given a source frame extracted from the real video, our pipeline proceeds in three steps. a) A Vision-Language Model (VLM) creates a Motion Caption from the instruction template database to animate the given frame. b) This caption guides a video generation model Seaweed to create a natural transformation. c) We sampled frames uniformly from the generated dynamic videos with a fixed interval and treated each pair of neighbouring frames as an image editing pair. We re-captioned the editing instruction by the same VLM, as well as the general description of each sampled frame (not shown in the figure).

Leaderboard

We run each method four times and report the average value. In addition to CLIP similarity metrics, we use Claude-3.7-Sonnet to evaluate the overall editing quality (VLM-Score). We also ask human participants to evaluate the instruction-following quality (Human-Eval-FL) and identity-preserving quality (Human-Eval-ID).

Benchmarking Open-Sourced Models

Category	Method	CLIP-SIMtxt↑	CLIP-Dtxt↑	CLIP-SIMimg↑	CLIP-Dimg↑	VLM-Eval↑
Camera Zoom	InstructPix2Pix	0.270	0.021	0.737	0.266	42.37
	MagicBrush	0.311	0.002	0.907	0.202	49.37
	UltraEdit (SD3)	0.299	0.000	0.864	0.249	54.74
	AnySD	0.309	0.001	0.911	0.182	40.92
	InstrcutMove	0.283	0.027	0.821	0.294	70.66
	OminiControl	0.251	0.022	0.722	0.300	45.79
	†InstrcutMove	0.301	0.045	0.846	0.425	82.29
	†OminiControl	0.310	0.039	0.801	0.414	74.15
	†ByteMorpher (Ours)	0.301	0.048	0.847	0.463	84.08
	GT	0.317	0.075	0.890	1.000	87.11
Camera Move	InstructPix2Pix	0.318	0.010	0.709	0.200	32.20
	MagicBrush	0.317	0.009	0.913	0.195	52.63
	UltraEdit (SD3)	0.306	0.012	0.885	0.240	59.01
	AnySD	0.318	0.010	0.909	0.200	49.37
	InstrcutMove	0.305	0.016	0.862	0.291	74.86
	OminiControl	0.243	0.022	0.687	0.243	16.71
	†InstrcutMove	0.304	0.027	0.883	0.412	82.53
	†OminiControl	0.298	0.025	0.891	0.304	79.26
	†ByteMorpher (Ours)	0.319	0.041	0.894	0.426	84.18
	GT	0.320	0.039	0.915	1.000	86.37
Object Motion	InstructPix2Pix	0.299	0.026	0.789	0.257	36.47
	MagicBrush	0.328	0.007	0.901	0.163	47.49
	UltraEdit (SD3)	0.324	0.012	0.887	0.237	62.13
	AnySD	0.319	0.008	0.879	0.189	48.31
	InstrcutMove	0.325	0.015	0.870	0.318	72.44
	OminiControl	0.279	0.023	0.753	0.270	34.11
	†InstrcutMove	0.328	0.043	0.891	0.481	87.97
	†OminiControl	0.330	0.036	0.892	0.470	86.48
	†ByteMorpher (Ours)	0.332	0.044	0.896	0.472	89.07
	GT	0.335	0.056	0.919	1.000	89.53
Human Motion	InstructPix2Pix	0.248	0.012	0.694	0.211	23.60
	MagicBrush	0.317	0.001	0.911	0.146	46.27
	UltraEdit (SD3)	0.313	0.011	0.900	0.195	50.64
	AnySD	0.312	0.003	0.894	0.156	38.12
	InstrcutMove	0.308	0.013	0.861	0.278	69.43
	OminiControl	0.230	0.018	0.660	0.229	25.18
	†InstrcutMove	0.314	0.023	0.901	0.442	84.70
	†OminiControl	0.311	0.016	0.880	0.399	80.78
	†ByteMorpher (Ours)	0.316	0.022	0.899	0.440	85.66
	GT	0.321	0.031	0.922	1.000	86.10
Interaction	InstructPix2Pix	0.271	0.020	0.732	0.263	31.29
	MagicBrush	0.317	0.004	0.914	0.167	39.98
	UltraEdit (SD3)	0.314	0.018	0.892	0.226	52.24
	AnySD	0.315	0.005	0.909	0.173	37.23
	InstrcutMove	0.309	0.019	0.855	0.318	67.07
	OminiControl	0.258	0.021	0.689	0.265	32.99
	†InstrcutMove	0.314	0.043	0.885	0.477	85.83
	†OminiControl	0.295	0.041	0.768	0.433	78.90
	†ByteMorpher (Ours)	0.320	0.045	0.884	0.483	86.61
	GT	0.324	0.046	0.905	1.000	88.84

Quantitative evaluation of open-sourced methods on ByteMorph-Bench. † indicates the method is trained on ByteMorph-6M. Best results are in bold, second best are underlined.

Bibtex

@article{chang2025bytemorph,
  title={ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions},
  author={Chang, Di and Cao, Mingdeng and Shi, Yichun and Liu, Bo and Cai, Shengqu and Zhou, Shijie and Huang, Weilin and Wetzstein, Gordon and Soleymani, Mohammad and Wang, Peng},
  journal={arXiv preprint arXiv:2506.03107},
  year={2025}
}
      

License

Our dataset ByteMorph-6M and evaluation benchmark ByteMorph-Bench are released under CC0-1.0 Creative Commons Zero v1.0 Universal License. The baseline model ByteMorpher, including code and weights, is released under FLUX.1-dev Non-Commercial License.

Disclaimer

Your access to and use of this dataset are at your own risk. We do not guarantee the accuracy of this dataset. The dataset is provided “as is” and we make no warranty or representation to you with respect to it and we expressly disclaim, and hereby expressly waive, all warranties, express, implied, statutory or otherwise. This includes, without limitation, warranties of quality, performance, merchantability or fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. In no event will we be liable to you on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this public license or use of the licensed material.The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.