I have been guilty of this, and sometimes still struggle with it.
Don't follow fickle feelings; follow God.
Webster's Dictionary defines "fickle" as "Not fixed or firm; liable to change; unstable." Your emotions and how you feel are unstable and liable to change. So, if you follow your feelings, then your life, your choices, and your actions will sway like a tree in strong wind.
Because your feelings can be unstable, it is important not to follow them exclusively. For example, there will probably be some days when you don't feel like being nice to your spouse. If you follow your feelings and act impulsively, then your marriage will have problems. Because feelings are unstable and unpredictable, someone who lives based on their feelings will also be unstable and unpredictable.
Indeed, James 1:8 says that someone who follows his feelings "is a double-minded man, unstable in all he does."
Instead of following our feelings, we need to follow God, for He is the only source of true stability. We need to take the same attitude as an old hymn, which says, "On Christ the solid Rock I stand, All other ground is sinking sand."
In Matthew 7:24-27, Jesus says, "Everyone who hears these words of mine and puts them into practice is like a wise man who built his house on the rock. The rain came down, the streams rose, and the winds blew and beat against that house; yet it did not fall, because it had its foundation on the rock. But everyone who hears these words of mine and does not put them into practice is like a foolish man who built his house on sand. The rain came down, the streams rose, and the winds blew and beat against that house, and it fell with a great crash."
That "rock" is God's Word. If we want to live solid, stable lives, they must be built on the rock—God, himself. Therefore, don't follow fickle feelings; follow God.