The launcher itself doesn't slow down the download and only the patch 1.1.3.0 doesn't help you either. These are multiple downloads.
Manual download is not recommended.
It can make a lot of sense if we could see your Launcher.Log.
You can find it under ...\Skyrim\EnderalLauncher\system\log. (just attach it to the post here)
What operating system do you have? Maybe any background application is slowing down your system?