The simple answer is: ...always before you flush your response to the client.
In studying for the SCWCD exam I was left with the impression that if you attempted to play with the response after it was committed, a java.lang.IllegalStateException would be thrown.
No! Do not let this trip you up on the exam.
For instance, below is some code I came across that had me saying "Whoa! this is going to throw an IllegalStateException. Danger Will Robinson!"
ServletOutputStream out = response.getOutputStream();
// set headers
So as it turns out nothing was happening. I thought that the ISE was being skirted because headers are buffered and the default buffer size was just large enough...but it was a poster on JavaRanch that clued me in.
The Servlet specification answers the question quite clearly: headers set after the response is committed (javax.servlet.ServletResponse#isCommitted()) are simply ignored. (see SRV 5.2)