|
Are macros very complicated to use?
They can be but don't have to be. The following shows a valid usage in 4 lines
package require Tmac
tmac::tmeval {
MAC-BLOCK opts {-bg red -fg white -width 20 -font helvetica}
set l1 [label .l1 <:opts:> -text {Enter Name Below}]
set l2 [label .l2 <:opts:> -text {Enter Address Below}]
}
So to use macros you need to
1. Require the Tmac package (or source tmac.tcl)
2. Define a macro (numerous means available and doesn't have to precede the usage in file)
3. Call/Invoke the macro
4. Get Tmac to process the code.
To process the code you can use tmeval, as done here, or tmsource or tmproc, etc.
I see tcllib already has the expand package. How is Tmac different?
Expand is aimed at producing documents like web pages from a template. Tmac is aimed at producing Tcl source code. Here are some of the
ways this works out differently in design.
- Tmac macros are named and the parameters are named and known to the processor.
- Naming allows the macros to be managed as to scope, redefinition, types, behavior options and error detection. For example, Tmac can be told to specially process
parameters that will be passed to expr.
- Tmac can automatically substitute in values for named parameters in a code block. This is a big convenience for programers. Simply take a working code block and replace
literal parameters (that is, strings or variables references) with parameter references. No need to write a proc somewhere using subst or format, etc.
- Tmac incorporates a multi-pass (preprocessor + evaluator) design so that macro definitions can be placed anywhere in source code that suits the programmer. To convert
repetitive code to a macro you can just leave it in place and format it as a macro definition. The macro definitions are automatically removed from the code stream.
- In Expand, all parameters to expansion commands are parsed by Tcl itself. If you used expand for processing source code this would be quite problematic. Whenever you
wanted to reference commands and variables that need to be expanded later those references would have to be escaped with backslashes or protected with braces. This makes constructing macros with
parameters more difficult and error prone. Tmac gets over this problem by offering its own parameter parsing options that always defer evaluation. Native Tcl evaluation is still available if needed.
- Tmac lets you easily tailor how and when the macro processing and evaluation will occur. This can be done on a string basis (tmeval), a proc definition basis (tmproc),
and a file basis (tmsource and tmfileio). Additionally, macros can be defined by a procedure call so that the preprocessor detection phase need not be used. And there are lower level calls with which to make your own
methods.
- Tmac definitions and calls can be nested in useful ways. These behaviors are concisely documented in the manual. Expand does not address nesting behavior.
Bottom Line: Expand is very nice for template-based document generation and Tmac is much more useful for source code generation (and composing macros) because that's what
it's designed for.
Aren't macros going to slow things down?
Yes, a little bit at startup but you might not notice it. The macro processor reads source code quite quickly. For example, when loading the largest file in tcllib, mime.tcl, here is the result on a 600 MHz
Windows box:
% pwd
C:/Program Files/tcl/lib/tcllib1.3/mime
% wc mime.tcl
3586 11515 98020 mime.tcl
% time {source mime.tcl} 100
24182 microseconds per iteration
(% package require tmac
0.2
% time {tmac::tmsource mime.tcl} 100
44597 microseconds per iteration
In this case, source with macro search added about 20,000 micro seconds or 2/100 of a second. Two tenths of a second was required to scan 3,586 lines for macros and in this case find none.
For a macro substitution yardstick, the same mime.tcl file was altered to include a 20 line
block macro definition and 20 invocations all in the same file. The macro had 5 parameters, substituted into the code block a total of 18 times per call. The 'expr' parameter was substituted 2 times per call.
% time {tmsource mime-perftest-mac.tcl} 100
83034 microseconds per iteration
% time {tmsource mime-perftest-mac.tcl} 100
82998 microseconds per iteration
% wc mime* perft*
3634 11776 99739 mime-perftest-mac.tcl
3586 11515 98020 mime-perftest-plain.tcl
4051 13283 115919 perftest.out
11271 36574 313678 total
Processing the 100k file with 1 macro definition and 20 macro invocations took about .08 seconds including disk io and the source eval itself.
As another yardstick, I currently do startup macro processing on 25 files in a largish GUI where macros are lightly used. Most of the files do not contain macros. There are 20 block macro definitions and 48
invocations in the 25 files (total 29,607 lines, 919,049 bytes). Startup time for the overall application is 4-5 seconds and did not increase by more than 1/2 second after adding macros.
There are also reasons macros can speedup your code and the main reason is avoiding procedure call overhead particular if the procedure would need multiple arguments, upvars, and global commands.
How do I know Tmac is working correctly?
Well, first off there is a test suit. There are about 70 tests covering all operations. There is also a test driver script that will run the tests repeatedly with odd combinations of macro separator characters.
Another efficient way to validate is to use the supplied macro viewer application, tmac-viewer.tcl. The viewer provides interactive experimentation: Paste in or type some macro enhanced code and see both the preprocessor output
and the Tcl eval output immediately, edit to taste and redo.
A lot of macro usage in the beginning is validating ones own understanding of how/when substitution is occurring. If you think you found a processor defect please post me an e-mail and I will investigate. Tmac is used successfully in a large GUI with nontrivial macros and macro nesting. It works.
None of the examples is very compelling. What's Tmac really good for?
Right. Examples aren't compelling because they are simple and in simple cases it's easy to see alternative codings that don't require macros. I wanted macros for various conditions where the code had a lot of
branches (if/else) and yet there was similarity. Some or most of the code was repeated with only minor changes across the branches. This code is a headache to maintain because of the subtly editing required across all the
branches for any changes. One alternative is to just use the Tcl eval command on a string of code. Eval works but is slower (it can't be byte compiled). And with eval you can certainly embed parameters in the code but they are
not explicitly passed so this style of coding is error-prone and slower. Another alternative is to start writing clever procs that break off the bits of the problem into separate contexts. Often writing procs is the way to go but procs raise a couple more issues. 1) Chance for more errors because you've got to collect up the logic into parameters to the proc and then transfer your mindset to the new context, and 2) The overhead of param passing, global commands and upvars may be too high and slow things down. So for my programming style I wanted macros to fill this need. Here's a real (smaller) use from my Ultranotes™ application. The code deals with statistics about projects, tasks, and items which all relate in a hierarchy:
# Calc num of tasks associated and number of days associated
# Note how search string needs to end in a space to enable match on last
MAC-BLOCK objCount i gvar fld {[regexp -all " @i " "[array get @gvar *,@fld] "]}
if {$o3 eq "prj"} {
# TODO "strCount" offers modest speedup vs. regexp -all
set prjcnt 0
set taskcnt <:objCount $i ptTasks project:>
set itemcnt <:objCount $i ptData project:>
} elseif {$o3 eq "tsk"} {
set prjcnt 0
set taskcnt 0
set itemcnt <:objCount $i ptData task:>
} else {
# clients
# Must loop over projects
# TODO - later add cli->...->task connectsion (bill/code or rate)
set taskcnt 0
set itemcnt 0
set prjcnt 0
foreach pi [regexp -all -inline " $i " "[array get ptProjects *,client] "] {
incr prjcnt
incr taskcnt <:objCount $i ptTasks project:>
incr itemcnt <:objCount $i ptData project:>
}
}
Another macro case is when Tcl syntax is too verbose for the coding problem. At one point I started developing a Tcl version of sed. In writing code to parse sed syntax, I found the string commands got unwieldy.
set a$anum [string range $lt 1 [expr {$c1 + $bscnt-2} ]]
Was a typical case. Making macros that tailored the syntax and expressions to just those I needed at any point in the code really reduced the coding effort:
set a$anum <:srange $lt 1 c1+bscnt-2 :>
How can I see what's going on with macro processing?
Use tmac-viewer.tcl (included). It's a GUI tool that let's you enter a macro definition, macro call and push a button to see the result. It can also display all macro calls in a
file and even execute the whole file and capture results.
A tricky string substitution problem.
For complex text substitutions (that is beyond the power of a single regsub command) a common Tcl method is to use regexp to replace the changing text with a Tcl
procedure call. Then use subst to replace the text. Regsub+subst works for decoding URL's or as on the Tcler's wiki here. This hits problems if the text in question already has embedded commands. In that case you need some pretty clever code to pre-escape and then un-escape the parts you don't want to expand. Tmac can solve the escape issue by just bypassing it.
Problem (submitted from colleague):
given text containing patterns like this
@MEASURE;
i want to replace the value between the @ and ; with a mapped value.
how do i do that kind of pattern matching in tcl?
it says here that it's real easy for you to give me an example.
if the record.measure value is XYZ
then <TD>@MEASURE;</TD>
should be mapped to <TD>$record(measure)</TD>
and finally to <TD>XYZ</TD>
how to do that initial replacement?
use this file:/u01/iplanet/cgi-bin/pxmnet16KQET/locale/C/tworkbookfrm.htm
on galahad.
and extract/map the @XXX; patterns...
do it and i'll give you a gold star.. plus you'll make me a big hero...
}
Solution here:
source tmac.tcl ;# (or package require)
set chIN [open tworkbookfrm.htm r]
set chOUT [open tworkbookfrm-x.htm w]
set s [read $chIN]
close $chIN
# Step 1 does some subs and embeds a macro call => sout
regsub -all {@(\w+);} $s {$record(<:lower \1:>)} sout
# Step 2 define and run macro (no subst is needed so no risk to for other [ ] chars)
tmac::tmmac-filter lower -proc tml w {return [string tolower $w]}
set sout2 [tmac::tmexpand $sout]
# Step 3 save it
puts -nonewline $chOUT $sout2
close $chOUT
I want to create some custom Tcl commands. Got any examples?
Yes, see the *.tmac example files that come with the release. Controls.tmac has experimental enhanced and/or specialized version of foreach, switch, set, and others. There is also a set of string command abbreviations and wiki formatters
Ifdef with Tmac from comp.lang.tcl
Date: Thu, 27 Mar 2003 14:35:20 GMT
From: Roy Terry <royterry@earthlink.net>
Newsgroups: comp.lang.tcl
Vincent Wartelle wrote:
>
> Hi, your idea is quite interesting !
Glad you think so. There is a lot of potential
for macros used prudently.
>
> Questions :
> - how would you with Tmac, code the tcl equivalent of C
> #ifdef DEBUG
> do something
> #else
> do something else (faster, silent)
> #endif
Funny, I hadn't much considered this usage before,
but it is easy to do with a filter macro.
====== Here you go ====
# Example of #ifdef style functionality in tmac
package require tmac
# 1. Define a filter proc to implement the behavior.
# This code gets called at macro expansion time so it
# must be defined before target code is processed.
proc ifdef {var codetrue args} {
global $var
# Might wish to use string trim on the returns
if {[info exists $var]} {
return $codetrue
} else {
# works same w/wo supplied 'else'
return [lindex $args end]
}
}
# 2. Define a filter macro to call "ifdef"
# Note: simple parsing means tmac will be discard the {} delimiters
tmac::tmmac-filter ifdef -parse simple ifdef
# Code using the 'ifdef' macro could be in other files
# or wrapped in one of several tmac::tm* procs.
# For compactness in this example I've just wrapped the
# code in a tmeval.
tmac::tmeval {
# If this were is a tmac:tmsource'd file then there would
# be no tmeval wrapper
<:ifdef DEBUG {
puts stderr "I AM IN DEBUG MODE"
} else {
puts stderr production!
}
:>
}
>
> - can Tmac produce preprocessed files, so that the preprocessor
> doesn't have to be packaged with application files ?
> In this case, is there a way to change "tmsource::xxx" to
> "source yyy", where yyy is the preprocessed form of xxx.
Currently there is:
tmac::tmfileio macro-enhanced-file plain-tcl-file
So one approach would be
to simply loop over the files to create the expanded
set. Does that address the situation you envisioned?
Regards,
Roy
>
> Questions to the community :
> are there other people here using preprocessors with tcl ?
>
> V. Wartelle.
------------------------------------------------------------
David Gravereaux wrote:
>
> Roy Terry <royterry@earthlink.net> wrote:
>
> >> - can Tmac produce preprocessed files, so that the preprocessor
> >> doesn't have to be packaged with application files ?
> >> In this case, is there a way to change "tmsource::xxx" to
> >> "source yyy", where yyy is the preprocessed form of xxx.
> >
> >Currently there is:
> > tmac::tmfilio macro-enhanced-file plain-tcl-file
> >
> >So one approach would be
> >to simply loop over the files to create the expanded
> >set. Does that address the situation you envisioned?
>
> Seems almost like a mini compile step. That's very interesting. I work mostly
> on the compiled side of Tcl and miss out in all the scripting fun. Tcl does
> give one $tcl_platform(debug) if the core has symbols and I think there's
> another for if your running in an instrumented interp with prodebug, too.
>
> Just wondering this... Could it be possible to preprocess a script for
> documentation extraction?
Sure. In that case you could write a filter macro
that copied data, perhaps with edits or formatting, to a
secondary channel. You could then wrap the bits you wanted
to document in this/these macros. The macro proc could either
delete the content (by returning "") or echo/process the
content back into the source stream.
Something like
###############
package require tmac
namespace import tmac::tm*
set docchan [open tmac-example-for-doc-process.out w]
proc docmake {nameofthing bodyofthing} {
global docchan
# Get as fancy as you like by adding more formal params
# to this proc and/or by "interpreting" the input strings
puts $docchan "** $nameofthing **\n$bodyofthing"
# Eat the incoming data; it disappears from source stream
return ""
}
tmmac-filter DOC -parse simple docmake
# .....
# Most likely in another file and
# extracted by tmsource or tmexpand [file2string xxx.tcl]
# but tmexpand-wrapped here for demo:
tmexpand {
<:DOC MagicProc52 {
Remark: This proc is classified top secret.
Input: url, date, gender, date-of-birth
Output: favorite beer
Lastmod: April fools 2002
}:>
}
close $docchan
##############
# the docchange file will contain:
** MagicProc52 **
Remark: This proc is classified top secret.
Input: url, date, gender, date-of-birth
Output: favorite beer
Lastmod: April fools 2002
Is this similar to your thinking?
> --
> David Gravereaux <davygrvy@pobox.com>
> [species: human; planet: earth,milkyway(western spiral arm),alpha sector]
|
