Swiss-born artist Félix Vallotton created disquieting paintings that conjure a sense of mystery – and menace, writes Kelly Grovier.