Differences between -던 and -았/었던 (retrospective modifiers)

The retrospective modifier V/Adj-던 (e.g., 하던) and its past version V/Adj-았/었던 (했던) are both used to describe processes, states, or situations that happened/took place in the past. In a lot of cases, V/Adj-던 and V/Adj-았/었던 are very similar. The subtle differences manifest if the speaker chooses intentionally between one or the other.

V/Adj-던 describes a situation that was not finished in the past and interrupted before completion; or an action that was repeated continuously in the past or was done just once but may happen again.

However, when the modifier -던 has the past marker in front (V/Adj-았/었던), it means the past action/situation was already completed at the time of speaking.

In short, V/Adj-던 and V/Adj-았/었던 are both for past events but V/Adj-던 is more used for actions that were continued/sustained/repeated habitually or a past action that was in progress but was interrupted (not finished) at the time of speaking, whereas V/Adj-았/었던 is used for actions that have finished in the past and are totally disconnected from the present.

Notes for Korean learners:

The most useful as a learner might be to recognize the difference between -던 and -았/었던 when reading books or listening to conversations. If you try to use these patterns in conversations or writing, stick with the -았/었던 form which tends to be more common, except if you are trying to talk about an action that was interrupted and therefore not completed yet at the time of speaking, in which case you would use -던. E.g., 커피를 마시던 사람이 방금 울었어요. The person who has been drinking coffee just cried (and maybe he will continue drinking coffee afterward) vs. 커피를 마셨던 사람이 울었어요. The person who had been drinking coffee cried (he is done drinking coffee).
for adjectives, the difference between Adj-던 and Adj-았/었던 is less pronounced than for verbs. When in doubt, use Adj-았/었던 over Adj-던 as the latter is very infrequent.
How to Study Korean has additional examples and a nice diagram comparing -ㄴ/은, -는, -던, and -았/었던.

Examples

세계여행 하던 사람. The person who has been doing world tours (he may have done several world tours, and he may go on another world tour tomorrow).

세계여행 하던 사람은 오늘 왔어요.

세계여행 했던 사람. The person who was on a world tour. The tour is completely over and he’s probably not going to go back on a world tour in the near future.

한국에 가던 사람. The person has a habit of going to Korea and just came back from one of the trips.

한국에 갔던 사람. The person that went to Korea, the trip is over and is probably not going to Korea before a while.

Source: National Institute of the Korean Language; Korean: A Comprehensive Grammar (Routledge).