Yes. Like humans, your fish respond to environmental and physical stimulants by getting stressed. Stress will adversely affect their immune system, making them more susceptible to infection and disease. If your fish contracts or dies of some illness or other, it may have been triggered by a high level of stress.
The most common source of stress in fish is poor water quality. Other unacceptable, thus stress-causing, habitat conditions for your aquarium fish are:
You may have given your fish the best possible aquarium setup, including pristine water, but if you have not been handling them properly, you could be causing them a lot of stress. Examples of stressful situations are: