I would, frankly question the need for any version of IE at this point... The extra overhead is significantly higher for supporting anything that isn't a modern browser, Edge, Chrome and Firefax update regularly (what is your userbase). I'm working on a Government project of all things and the cutoff we went with is browsers supporting async functions (es2017). A lot of polyfills for everything from generators/async to fetch are just eliminted by that. ymmv. Supporting IE8 will blow up your bundle size with many polyfills and transforms if you want to use modern JS.
I have a modern app set for es2017+ browser with webpack, babel, react, redux, material-ui and more. The main bundle is 156k gzipped[0]. If you want more lightweight, you could use a different UI kit, and use inferno. Alternatives include Vue, but the two apps I've been involved with using it I haven't cared for either. All said, 160K for JS + CSS + Controls isn't so bad. It's not as lightweight as you can get, but it's a pretty rich feature set. If you look at the core-js section, if you include IE8, it will be *MASSIVE*
If you work at it, and don't need routing, or fancy controls, etc... you can pretty readily create a bundle that's under 30k.
[0] https://imgur.com/a/CSvjNrb