There is no specific limit to the amount of output, however in practice we’ve found that there will come a point at which it will no longer display properly (this seems to be a limitation of the underlying text display object). Here’s a little script you can run to try this:
set the numberFormat to "00000000"
repeat a million times
put repeatIndex() && "abcdefghij"
When I ran this on my machine just now, the last line I saw in the display was line 769229. However, after the script finished I clicked in the output field and did a Select All (Cmd-a) and Copy (Cmd-c). I then opened a new TextEdit window and did a Paste and all 20 million characters of the output was pasted there. So it seems to be possible to get quite a bit of output even if it doesn’t all show up in the output field.
Your results may be different. In particular, I suspect that the output from an ordinary test run (which will likely be showing output in varying colors) may be more limited, but I don’t know for sure.