SSH: "Permissions 0644 for 'my_key.pub' are too open."

Why is 0644 i.e. -rw-r--r-- too open for a SSH key? Also I could not find any false permissions on the .ssh directory (0700) or the home directory (0731). Btw I'm getting this error when testing the