It feels like it would make more sense to allow the model to do Levenshtein-like... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		bonoboTP 60 days ago \| parent \| context \| favorite \| on: BERT is just a single text diffusion step It feels like it would make more sense to allow the model to do Levenshtein-like edits instead of just masking and filling in the masked tokens. It seems that intuitively it's really hard in this diffusion setup to just swap one word with a longer but better synonym towards the end, because there's no way to shift everything to the right afterwards.

lucidrains 60 days ago [–]

there has been some movement on that front, in the form of adding expand / delete tokens! https://hkunlp.github.io/blog/2025/dreamon/

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact