Changing props of children without any visual information (like a renderProps pattern or a custom hook) can be bad if you work in team.
You should really consider removing magic from your lib.
I've been using this approach in a couple of projects now, and I must say it's a pleasure to work with and saves a lot of repetitive coding, and improves UX easily.
I've now rewritten it to use less code (thanks to the hooks API) and I added some examples and wrappers for popular UI libraries (semantic-ui-react, react-bootstrap, antd, plus Material UI example)