You shouldn't give up because of that! >w<
I'm 4'11'', and I weight 160lbs.
Low metabolism runs in my family, on top of me having hip surgery back in 2009, which is the reason I gained weight so badly. It's hard to exercise to keep the weight off.
BUT, I just bought my first lolita today!
There are many ways to work around weight to get a look that is cute in it own. :3
For example, to help keep a nice shape, I plan to wear a bodice with my lolita.
If you're tight on money like me, try looking through places that have second hand used lolita.
Is has lolita galore that people are selling for less then the price they bought them, usually.
There's also here: http://egl-plus.livejournal.com/
This is a plus sized community where you can also find sales posts just for plus sized items. :3
I can also recommend some Facebook groups and such where you can find more sales.
Just let me know. ^ ^
I know exactly how you feel, but never give up! It's possible if you look in the right places.