Table of Contents

• 1. token
  • 1.1. Syntax
  • 1.2. Stack Effects
  • 1.3. Description
  • 1.4. PostScript Level
  • 1.5. Examples
  • 1.6. Common Use Cases
    • 1.6.1. Configuration File Parsing
    • 1.6.2. String to Number Conversion
    • 1.6.3. Multi-token Parsing
  • 1.7. Common Pitfalls
  • 1.8. Error Conditions
  • 1.9. Token Consumption Rules
  • 1.10. Implementation Notes
  • 1.11. Advanced Example
  • 1.12. See Also

1. token

Parses and returns the next PostScript token from a file or string.

1.1. Syntax

file token → any true (if token found)
           → false (if end-of-file)
string token → post any true (if token found)
             → false (if only whitespace)

1.2. Stack Effects

Table 1. File Input (success)

Level	Object
1	true
0	any (scanned token object)

Table 2. File Input (end-of-file)

Level	Object
0	false

Table 3. String Input (success)

Level	Object
2	true
1	any (scanned token object)
0	post (remaining string)

Table 4. String Input (no token)

Level	Object
0	false

1.3. Description

token reads characters from file or string, interpreting them according to PostScript syntax rules until a complete object is scanned.

File case: * Returns the scanned object and true if successful * Closes file and returns false if end-of-file encountered before any non-whitespace characters

String case: * Returns post (remaining substring), scanned object, and true if successful * Returns false if only whitespace characters present

The scanned object may be: * Simple: integer, real, or name * Composite: string (…​) or procedure { …​ }

1.4. PostScript Level

Level 1 and later

1.5. Examples

Parsing from string

(15(St1) { 1 2 add }) token
% Result: ((St1) { 1 2 add }) 15 true

Sequential parsing

((St1) { 1 2 add }) token
% Result: ( { 1 2 add }) (St1) true

( { 1 2 add }) token
% Result: ( ) { 1 2 add } true

Empty string

( ) token
% Result: false

Parsing numbers

(123 456) token
% Result: ( 456) 123 true

1.6. Common Use Cases

1.6.1. Configuration File Parsing

configfile {
  token not { exit } if
  % Process each token
} loop

1.6.2. String to Number Conversion

(3.14159) token pop  % Discard remainder
% Stack: 3.14159

1.6.3. Multi-token Parsing

/line (name: value) def
line token {
  /key exch def      % Save token as key
  token {
    /val exch def    % Save token as value
  } if
  pop                % Discard remainder
} if

1.7. Common Pitfalls

Consumes Whitespace - token consumes trailing whitespace after names/numbers.

(123 456) token  % Consumes space after 123
% Result: ( 456) not (456)

Evaluates Syntax - Returns actual objects, not string representations.

(123) token  % Returns integer 123, not string
({ 1 2 }) token  % Returns procedure, not string

Special Character Handling - Different characters consumed differently after tokens.

Binary Token Support - token also handles binary tokens and binary object sequences.

1.8. Error Conditions

Error	Condition
[invalidaccess]	String/file has no-access attribute
[ioerror]	File I/O error
[limitcheck]	Token too large for implementation
[stackoverflow]	Not enough room for results
[stackunderflow]	No operand on stack
[syntaxerror]	Invalid PostScript syntax
[typecheck]	Operand not file or string
[undefinedresult]	Number out of range
[VMerror]	Insufficient VM for composite object

1.9. Token Consumption Rules

token consumes characters differently based on token type:

Token Type	Characters Consumed
Name/number + whitespace	Token + first whitespace char
String (…​)	Including closing )
Procedure { …​ }	Including closing }
Array [ …​ ]	Including closing ]
Name preceded by /	Not including the /
Binary token	Exact bytes, no extra

1.10. Implementation Notes

• Same parsing logic as the PostScript interpreter

• Returns literal objects (use cvx) if execution needed

• File position updated past consumed characters

• For strings, post points into original string (shared value)

1.11. Advanced Example

Complete string tokenization

/tokenize {  % string => array-of-tokens
  [ exch
  {
    token not { exit } if
  } loop
  ]
} def

(123 (abc) /name { 1 2 add }) tokenize
% Result: [123 (abc) /name { 1 2 add }]

1.12. See Also

• search - Find substring

• anchorsearch - Test for prefix

• cvs - Convert to string

• cvn - Convert to name

• cvx - Make executable