What is this?

It's a hardened texthost featuring highly secure AES-256 encryption called Getgle Docs.

I forgot the password to my paste, can it be recovered?

No, it cannot be recovered. Secure pastes are encrypted with military-grade encryption and only people with access to the password can view them.
Passwords are not stored on docs.getgle.org so I would have no way of helping you.

Why are some encrypted posts displayed as gibberish?

It's because you put the wrong password in.