While installing updates in Windows is a simple process, the mechanism behind the scenes that manages it all is quite complicated. There is also a lot of misinformation on the Internet about how to deal with issues related to Windows Update.
HOW TO EASILY REMOVE WINDOWS UPDATES
One of the biggest misconceptions is that all Windows updates can be installed at once by simply deleting the SoftwareDistribution folder stored in C:/Windows. This is completely false. Yes, you might save some space (from 500 MB to several GBs), but deleting the folder does not remove any installed updates.
Additionally, it is a bad idea to delete the SoftwareDistribution folder unless absolutely necessary. There is a subfolder called DataStore that has a large database of all available Windows Updates. This database will likely be a few hundred MB in size. Deleting this database will only delete the Windows Update history for the computer.
To prove this to myself, I did a test. I opened the Control Panel, clicked on Windows Update, and then clicked on View update history in the left menu.