To move or remove icons from the default Home screen, hold your finder over an icon for two seconds; all icons will then start "vibrating" and this indicates that you're in Edit mode. You can now drag & drop icons to different positions, thus changing their order. If you move an icon outside of the screen, on the right-hand or left-hand side, you will place it on a secondary page. To delete an icon click on the X symbol of the icon (note that this won't delete the actual application).
To save your changes click on the physical Home button. If you moved an icon to a secondary page, you'll see two or more "dots" at the bottom of the icons list, which will allow you to switch the current page.