If this happens often, you can disable write caching for that drive. It’ll feel slightly slower (since it’s actually operating at the speed of the hardware instead of caching operations in RAM and gradually writing them to disk in the background), but you’ll be able to remove the drive almost instantly.