PGP and GnuPG for newbies

Recently, I played around with PGP keys for mail, files, and in general data encryption/verification between a sender and a recipient.
The PGP standard (stand for Pretty Good Privacy) have an open source suite implementation: GPG (GnuPG). This is quite simple to use if you want to ensure that only the person you want can read your message, and on the other side be sure that you can trust message incoming.
This post is a short one to summarize what is a PGP keys pair, what's the point with it, and how to use it thanks to GnuPG.