The Hilbert scheme of points on $X$ is a moduli space of finite schemes embedded into a fixed scheme $X$. It was first shown by Jelisiejew that the Hilbert scheme of points on smooth variety is non-reduced for large number of points. Since then, several other examples appeared, also for the nested Hilbert scheme, but there was no example in the Gorenstein locus. In this talk I will present an uniform perspective at the non-reduceness results. Moreover, I will show examples of non-reduced points on the Gorenstein locus of the Hilbert scheme.