read synchronization from process /socket

I have a sync problem while reading from a process.

After writing to a process if I wait long enough the I can read successfully, e.g. after 10 sec. But I want to sync the process, because data may be available in 1 sec, 2 sec, etc.

In other words, Is there a way to read data after they arrived?

Yes, if you specify “until end” on a read command from a process or socket, you should get whatever data is available. If no input is immediately available from that socket or process, the read command will wait until either some input is available, or the duration of the readTimeout global property has elapsed.

So, in other words, in most cases it should “just work” without the need to include any artificial waits in your script. Here’s a trivial example:

open process "#1"
write "ls" & return to process "#1"
read from process "#1" until end
put it -- this won't be empty
close process "#1"

If a lot of data is returned, you may only get part of it, but you shouldn’t get nothing (empty) unless no data becomes available within the readTimeout seconds.

May be I did not properly explained the issue I face.

I open telnet to a remote terminal, post a command and while reading until end, data is still coming, I do not get whole output. So just using single “read until end” statement does not work here.

I had to implement the function below.

I have to wait 1 sec at the end of every function call to make sure I do not have anymore incoming data.

The problem with the function is that in case of a network delay 1 sec is not enough, I could not find a mechanism to establish kind of a session and find out if session ended or not. In other words, “until end” portion does not work with telnet.

I have not tried to do this over the socket yet but how doubts the it could work either.


function execTermCmd_Int term_id, term_cmd
if term_cmd is empty then set term_cmd to return
write term_cmd & return to process term_id
set tmp_txt to “”

repeat forever
	wait 0.2
	read from process term_id until end in 1 second into tmp_buffer
	if tmp_buffer is empty then
		put words first to last of tmp_txt into tmp_txt
		return (data:tmp_txt, code:0)
	end if
	set tmp_txt to tmp_txt & tmp_buffer
	if repeatIndex() > 10000 then return (data:"", code:1)
end repeat

return (data:"", code:-9)

end execTermCmd_Int

The problem is that the read command doesn’t know whether there is more data still to come or not. You write something to an external process, and the external process writes something back – but how do you know when it is done sending data back to you? It might send back a few characters. It might send back megabytes of data.

If the external process sends a lot of data, it can be read a little at a time (as you are doing in your script) but unless there is a definitive way to know when it has finished sending data, you’ll have to make assumptions such as guessing that if a second goes by with no more data then it must be finished. It appears your script is doing that as well.

The only way to improve the situation is by having some advance knowledge of what to expect, so your script can know that it has received all of the data and stop trying to read more. If there is some “end of data” marker that is sent, for example, or something at the beginning of the output that indicates the amount of data that will be returned, then you can use that information to decide how much data to read.

If you don’t have either of those situations, you may be able to force your own “end of data” marker. For example, here’s a modified version of my example script that causes 4 dollar signs to be sent back after the “ls” command finishes:

open process "#1"
write "ls; echo '$$$$' " & return to process "#1"
read from process "#1" until "$$$$"
put it
close process "#1"

This ensures that the entire output of the ls command will always be read, provided that 4 dollar signs will never occur as part of the normal output from the command. You may be able to adapt this approach to solve your situation.