So here's the lowdown of what happened between season 3 and 4:

There were problems with Garrett Wang during season three. He would often come to the set late, drunk/high, not knowing his lines, and acted like a bit of a "diva." He also did not get on with Rick Berman. He was suspended from the show without pay for two episodes, "Fair Trade" and "Blood Fever," due to his behavior. Already feeling like they had too many characters on the show, producers allegedly decided to write Kim off the show.

Initially, the third season finale was supposed to be the start of "The Year of Hell," and the plan was Kim would be killed off during these events. However, with the success of First Contact a few months earlier, UPN (wanting to bring the ratings up) suggested bringing the Borg back on the show. Additionally, they wanted the writers to 'shake things up' for season 4 to boost declining ratings. At this time, writer Brannon Braga came up with the idea of adding a quote: "Borg babe" to the show. So "Scorpion Part 1" was written. Harry's injuries in the episode were added to write Wang out of the series in Part 2 with having Kim evolve due to his contact with Species 8472 (like they ultimately did with Kes).

While filming Scorpion Part 1, the cast became aware that one of them was going to be released from their contract to make room for a new character. Everyone knew it was going to be Wang initially. Even Wang has admitted he knew he was in danger of being fired at the end of season 3. But Wang was then chosen as one of People Magazine's Most Beautiful People, and UPN nixed the producers' decision to fire him because they felt it would be a wrong business move to get rid of the free publicity. So, another cast member had to be canned. Allegedly, Ethan Phillips was confident it was going to be him. His character had very little to do, and he knew that his character was also not the most popular. He was originally supposed to be the comic relief, but the Doctor ultimately ended up being the funny character. Phillips has said he started looking for new work during the hiatus.

It's believed the decision to fire Lien was down to Jeri Taylor, who was showrunner at that point. There is gossip that Taylor had a personal issue with Lien (apparently Lien briefly dated her son Alexander Enberg aka Vorik, and it ended badly or something), but that's never been proven. Other writers like Kenneth Biller have stated Kes was not the character they would have written off the show, and Rick Berman has noted that the decision to fired Lien had more to the network making the call (you shouldn't have two blonde women on the show). Interestingly, it was Berman who wrote "Fury," Kes' horrific swan song to Voyager.

There are conflicting accounts of who Lien was notified (some cast say she didn't find out until they got the script for "The Gift"), but it seems she was told during the summer so she could look for other jobs. Also, Kes was supposed to be in the first five episodes of season four, with the 5th episode being the one where she leaves, but Lien had already booked another job and had to go, so her story was tacked on to the second episode, which was originally supposed to be just about Seven. If you actually watch "Scorpion Part 2" and "The Gift" it kind of looks like Lien has been crying and she's not "fully invested" in her scenes any more.