Get the sample code here: PyObjC-Highlight.zip
Just about every program that displays, manages, or edits structured or parsable data has some form of syntax highlighting. This sample application presents a naive syntax highlighter for a YAML like markup language, using PyObjC and Cocoa.
Our highlighter covers three basic elements:
- Comments, which start with a pound (#) symbol
- Key – value pairs, which are separated by a colon (:)
- List items, which start with a dash (-)
Comment lines are formatted with green text and a normal font. The Key of the key – value pair is colored blue and presented in a bold font. The dash of all list items is colored red, using a normal font. The remainder of the text should be represented in a black system default font.
In this example the text highlighting should be applied whenever the text changes, which although convenient for a simple example does cause issues for large documents that take a long time to parse and highlight, as the UI will freeze during the highlight process. There are a few solutions to this problem:
- Run the highlighter in a background thread, triggered whenever typing has stopped
- Use an incremental parser/highlighter that only updates the current and immediately surrounding lines
We’ll bypass these solutions in favor of a straightforward example. To implement a highlighter that updates whenever the text changes we’ll use the textDidChange delegate method of the NSText class.
def textDidChange_(self, notification): """ Delegate method called by the NSTextView whenever the contents of the text view have changed. This is called after the text has changed and been committed to the view. See the Cocoa reference documents: http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/Classes/NSText_Class/Reference/Reference.html http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/Classes/NSTextView_Class/Reference/Reference.html Specifically the sections on Delegate Methods for information on additional delegate methods relating to text control is NSTextView objects. """ # Retrieve the current contents of the document and start highlighting content = self.highlightedText.string() self.highlightText(content)
In this method we retrieve the contents of the NSTextView in our UI, and pass that content to our highlightText method:
def highlightText(self, content):
"""
Apply our customized highlighting to the provided content. It is assumed that
this content was extracted from the NSTextView.
"""
# Calling the setAttributesForRange with no values creates
# a default that "resets" the formatting on all of the content
self.setAttributesForRange(None, None, None, None)
# We'll highlight the content by breaking it down into lines, and
# processing each line one by one. By storing how many characters
# have been processed we can maintain an "offset" into the overall
# content that we use to specify the range of text that is currently
# being highlighted.
contentLines = content.split("n")
highlightOffset = 0
for line in contentLines:
if line.strip().startswith("#"):
# Comment - we want to highlight the whole comment line
self.setAttributesForRange(NSColor.greenColor(), None, highlightOffset, len(line))
elif line.find(":") > -1:
# Tag - we only want to highlight the tag, not the colon or the remainder of the line
startOfLine = line[0: line.find(":")]
yamlTag = startOfLine.strip("t ")
yamlTagStart = line.find(yamlTag)
self.setAttributesForRange(NSColor.blueColor(), "bold", highlightOffset + yamlTagStart, len(yamlTag))
elif line.strip().startswith("-"):
# List item - we only want to highlight the dash
listIndex = line.find("-")
self.setAttributesForRange(NSColor.redColor(), None, highlightOffset + listIndex, 1)
# Add the processed line to our offset, as well as the newline that terminated the line
highlightOffset += len(line) + 1
The technique we use for parsing our content is primitive (and error prone) but suffices for a simple example. After breaking the content into lines, each line is checked against a basic format for our three highlighting targets. If the line matches, we determine the range of the text from that line to which highlighting will be applied. We call our setAttributesForRange method with a color (an NSColor object), a font strength (“normal” or “bold”), and the location and length of the text in the original document to highlight.
Throughout the parsing we need to maintain a tally of how many characters have been processed, as the highlighting is being applied to the original content, and we need an index into that content. Normally in Python we would use two indices into a List to denote a slice or range; unfortunately Objective-C uses an index and length to create a range (specifically with the NSRange class). This conversion doesn’t add a lot of code to our example, but it can be a point in the code for bugs to pop up.
The actual highlighting takes place in the setAttributesForRange method:
def setAttributesForRange(self, color, font, rangeStart, rangeLength):
"""
Set the visual attributes for a range of characters in the NSTextView. If
values for the color and font are None, defaults will be used.
The rangeStart is an index into the contents of the NSTextView, and
rangeLength is used in combination with this index to create an NSRange
structure, which is passed to the NSTextView methods for setting
text attributes. If either of these values are None, defaults will
be provided.
The "font" parameter is used as an key for the "fontMap", which contains
the associated NSFont objects for each font style.
"""
fontMap = {
"normal" : NSFont.systemFontOfSize_(self.fontSize),
"bold" : NSFont.boldSystemFontOfSize_(self.fontSize)
}
# Setup sane defaults for the color, font and range if no values
# are provided
if color is None:
color = NSColor.blackColor()
if font is None:
font = "normal"
if font not in fontMap:
font = "normal"
displayFont = fontMap[font]
if rangeStart is None:
rangeStart = 0
if rangeLength is None:
rangeLength = len(self.highlightedText.string()) - rangeStart
# Set the attributes for the specified character range
range = NSRange(rangeStart, rangeLength)
self.highlightedText.setTextColor_range_(color, range)
self.highlightedText.setFont_range_(displayFont, range)
This method takes a color (NSColor object), a font type (“normal” or “bold”), and the location and length of the text to highlight in the NSTextView. If any of the parameters is None a sensible default value is used. When called with all None parameters the default is to highlight the entire range of the document with a normal weight black font (in fact the method is called like this at the start of the highlightText method to reset the highlights on the document). The actual methods used to apply formatting are the setTextColor_range_ and setFont_range_ methods of the NSTextView.
Get the sample code here: PyObjC-Thread.zip
This is a small sample project showing how to invoke Python methods as background tasks when writing applications using the PyObjC Python to Objective-C bridge. Normally any method invoked as the result of a User Interface action is run in the UI thread, which for most fast tasks is just fine, there will be no noticable degredation of user experience.
But when a long running task is run on the primary application thread, the UI will become non-responsive, and updates to the UI from within the task will be delayed until the task completes, which means that any indications of progress will be lost. This can be seen in the sample application with the startProgressNoThread_ and runProgressNoThread methods:
@objc.IBAction
def startProgressNoThread_(self, sender):
"""
This method is triggered by the "Progress without Thread" button in
the UI.
"""
NSLog("Starting the progress bar without background thread")
self.runProgressNoThread()
def runProgressNoThread(self):
"""
Update an NSProgressIndicator in a loop, with a small delay between
each update to the indicator.
Because this method is called on the application event loop, nothing
will update on the UI until this method is complete. The NSProgressIndicator
will not properly update, and the calls to disable and then enable the
buttons will not happen properly. All of the UI updates will be delayed
until this method has finished. This method also completely locks the UI,
allowing no interaction or input.
"""
self.buttonsEnabled(False)
self.progress_bar.setMinValue_(0.0)
self.progress_bar.setMaxValue_(100.0)
self.progress_bar.setIndeterminate_(False)
for i in range(0, 101):
self.progress_bar.setDoubleValue_(i * 1.0)
time.sleep(0.05)
self.buttonsEnabled(True)
These methods can be found in the PyObjC_ThreadAppDelegate.py file. The startProgressNoThread_ method is triggered by the Progress without Thread button in the application interface. If you run this application, and click on this button, you’ll notice that the interface locks up for the duration of the runProgressNoThread method, with no UI updates and no UI response. You can’t even exit the program normally. To fix this user experience we can take advantage of the NSThread Objective-C class, and run this task in a background thread. Adding a background thread introduces only two new lines of code, and modifies one existing line of code as compared to the non-threaded example.
To invoke a background thread we need to create an NSThread object and initialize it with a method to execute in the background, and then start the thread:
@objc.IBAction
def startProgressThreaded_(self, sender):
"""
This method is triggered by the "Progress with Thread" button in the
UI. Instead of directly calling a Python method, we use the NSThread
class to start a new background thread, passing the object method to
be invoked in this new thread.
"""
NSLog("Starting progress bar with background thread")
t = NSThread.alloc().initWithTarget_selector_object_(self,self.runProgressThreaded, None)
t.start()
This method is triggered by the Progress with Thread button in our interface. Rather than directly calling our task (as the startProgressNoThread_ method does), we create an NSThread, with the method runProgressThreaded as an argument. The t.start() call kicks off our background thread.
Every thread in an Objective-C program uses a pool of memory to manage and control how and when memory is allocated to objects, and how that memory gets cleaned up. The application thread has already created the memory pool used by the UI and the rest of our program, but because we’re starting out own background thread, we need to make sure to create a new NSAutoreleasePool to be used by objects within our thread.
def runProgressThreaded(self): """ Update an NSProgressIndicator in a loop, with a small delay between each update to the indicator. This method is called in a background thread. Each thread in an Objective-C program has it's own NSAutoreleasePool for memory management. We need to create this pool before updating any objects in our UI. Creating and then releasing this autorelease pool only adds two lines of code, as compared to the runProgressNoThread method. While this method is running the UI will update normally and respond to user input. """ self.buttonsEnabled(False) pool = NSAutoreleasePool.alloc().init() self.progress_bar.setMinValue_(0.0) self.progress_bar.setMaxValue_(100.0) self.progress_bar.setIndeterminate_(False) for i in range(0, 101): self.progress_bar.setDoubleValue_(i * 1.0) time.sleep(0.05) pool.release() self.buttonsEnabled(True)
The runProgressThreaded method is nearly identical to the runProgressNoThread method, with the addition of the memory pool, which we create using the NSAutoreleasePool class. If you run the application you’ll see that when you click the Progress with Thread button the UI remains responsive, the NSProgressIndicator updates throughout the task process, and you can exit the program normally while the task is running.
This example is obviously contrived, but running a task in the background like this is feasible for any long running Python method.
Among various other tasks for a project this week I’ve been writing some Python classes to query Apple’s AddressBook database. OS X 10.5 ships with PyObjC, a Python to Objective-C bridge, preinstalled, which makes this kind of task fairly straight forward. Or, at least, it should. A few minutes of digging discovered that what few examples of working with Python and AddressBook were easy to find were incomplete, broken, or counter to the task I was trying to accomplish. Add to that a lack of any good documentation on the PyObjC methods, and it fell to experimenting and digging through Objective-C documentation.
What follows is a method for retrieving a list of People from the local AddressBook as a list of Python Dictionaries.
from AddressBook import *
import pprint
def addressBookToList():
"""
Read the current user's AddressBook database, converting each person
in the address book into a Dictionary of values. Some values (addresses,
phone numbers, email, etc) can have multiple values, in which case a
list of all of those values is stored. The result of this method is
a List of Dictionaries, with each person represented by a single record
in the list.
"""
# get the shared addressbook and the list of
# people from the book.
ab = ABAddressBook.sharedAddressBook()
people = ab.people()
peopleList = []
# convert the ABPerson to a hash
for person in people:
thisPerson = {}
props = person.allProperties()
for prop in props:
# skip some properties
if prop == "com.apple.ABPersonMeProperty":
continue
elif prop == "com.apple.ABImageData":
continue
# How we convert the value depends on the ObjC
# class used to represent it
val = person.valueForProperty_(prop)
if type(val) == objc.pyobjc_unicode:
# Unicode String
thisPerson[prop.lower()] = val
elif issubclass(val.__class__, NSDate):
# NSDate
thisPerson[prop.lower()] = val.description()
elif type(val) == ABMultiValueCoreDataWrapper:
# List -- convert each item in the list
# into the proper format
thisPerson[prop.lower()] = []
for valIndex in range(0, val.count()):
indexedValue = val.valueAtIndex_(valIndex)
if type(indexedValue) == objc.pyobjc_unicode:
# Unicode string
thisPerson[prop.lower()].append(indexedValue)
elif issubclass(indexedValue.__class__, NSDate):
# Date
thisPerson[prop.lower()].append(indexedValue.description())
elif type(indexedValue) == NSCFDictionary:
# NSDictionary -- convert to a Python Dictionary
propDict = {}
for propKey in indexedValue.keys():
propValue = indexedValue[propKey]
propDict[propKey.lower()] = propValue
thisPerson[prop.lower()].append(propDict)
peopleList.append(thisPerson)
return peopleList
So, given an entry in the AddressBook that looked like:
This method will have the following Dictionary in the list of returned People:
{ u'address': [ { u'city': u'Anytown',
u'country': u'USA',
u'countrycode': u'us',
u'state': u'NY',
u'street': u'123 Fake Street',
u'zip': u'10111'}],
u'aiminstant': [u'john_doe_aim'],
u'creation': u'2008-04-09 13:33:17 -0500',
u'email': [u'john@doe.com'],
u'first': u'John',
u'last': u'Doe',
u'modification': u'2008-04-09 13:33:17 -0500',
u'phone': [u'555-555-1212'],
u'uid': u'BBFAB17F-591A-4D3C-BE75-4FE5B25B984D:ABPerson'}