Conda set LD_LIBRARY_PATH for env only

I have an installation of miniconda3 where I have created a virtual environment called py35. I have some libraries that I only want to use from within this environment. hence they are under /.../