Elijah was Hannah's boyfriend during their last two years of college. He is also the one who introduced Marnie to Charlie. After he and Hannah broke up for unspecified reasons, he came out of the closet (or realized he was gay).
Elijah reveals to Hannah that he is gay in the episode "All Adventurous Women Do", when he also notably tells her that her Dad is gay. It's also revealed that he made out with Marnie once while they were in college. He becomes Hannah's new roommate at the end of the first season finale, "She Did". In the premiere of Season 2, "It's About Time", he begins hooking up with Marnie, but backs out when he thinks she rolls her eyes at him. In the episode Bad Friend, while they're both under cocaine, Elijah tells Hannah he slept with Marnie and she gets upset and kick him out of the apartment.
Elijah's character makes several small cameo appearances in Season 3, but does not truly re-emerge as a protagonist until Season 4 when he goes to visit Hannah at graduate school in Iowa. Elijah starts the season off as a release for Hannah and a taste of home for her, but when she begins to doubt herself in the episode Female Author, he consoles her concerning her future in graduate school and as a writer.
Later in the season, Elijah moves back in to Hannah's old apartment in New York after her messy break up with Adam. We also find out in Season 4 that Elijah's previous accusation of Hannah's father being gay was correct when Hannah's father reveals to his family that he is, indeed, homosexual. Elijah does not forget to remind Hannah that his assumptions were correct.