0. Add a submodule

git submodule add [-b <branch>]<repository> [<path>]
git submodule init
git submodule update

1. Remove config entries:

git config -f .git/config --remove-section submodule.$submodulepath
git config -f .gitmodules --remove-section submodule.$submodulepath

2. Remove directory from index:

git rm --cached $submodulepath

3. Commit

4. Delete unused files:

rm -rf $submodulepath
rm -rf .git/modules/$submodulepath

Background

When you do git submodule add, it only adds it to .gitmodules, but once you did git submodule init, it added to .git/config.

So if you wish to remove the modules, but be able to restore it quickly, then do just this:

git rm --cached $submodulepath
git config -f .git/config --remove-section submodule.$submodulepath

It is a good idea to do git rebase HEAD first and git commit at the end, if you put this in a script.