It is now possible to connect to dedicated servers in-game.
To connect to your server from the in game menu (Note: At this time it does not support name, it is IP only.)
1. Open Valheim
2. Click Start Game
3. Pick Character and press Start
4. Press Join Game tab
5. Press Join IP button
6. Paste in your IP and Port. (You can get the IP from the control panel under the Management > Allocation Settings menu in the Server Control Panel)
To save it as a favourite and connect the old way to your dedicated server.
1. Close Valheim game
2. Open the main Steam Window
3. Click View
4. Click Servers
5. Click Favourites
6. Click Add Server
7. Paste in the game server address + port
8. NOTE: the port must be 1 above your actual game server port. So if your game server shows the port as 25560 in the control panel, you need to amend the address in the Steam favourites window to be 25561.
9. Press Connect and Enter password
10. Once the game opens pick your character and press Start