Continuum + two other new apps & bundle write support

It’s not that long since the previous news update, but there are a couple changes that are especially important:

Application bundle writable file system support

One problem with the app bundles was that by nature, their contents have read-only privileges to prevent unwanted modifications. However, as the emulator’s virtual filesystem needs to be able to not only create new files, but also modify existing ones (such as highscore saving in ZeroGravity and Continuum), using the files directory from inside the app bundle was not suitable.

To make the virtual file system writable, the emulator now relocates it to a safely writable location at /Users/<username>/Library/Application Support/MACE/<application-name>/vfs. This has a couple of neat features:

  • On every launch, any original files which are either not present or zero-length are copied from the app bundle to the writable location. This includes the initial launch, during which the entire filesystem gets replicated there.
  • Holding down the “Shift” key performs a “soft-reset” of the writable filesystem, meaning that all originals files are replicated even if they exist already in the writable filesystem. This is handy in cases if the original system or application files get damaged and would cause the emulation to fail to start. NOTE: There is not yet any prompt when performing the “soft-reset”, but there is plan to add a confirmation dialog (including a option for “hard-reset” feature) later at some time.
  • Any files modified or saved by user are fully accessible through this folder in AppleDouble format, which can be exchanged between applications and/or other emulators (or with appropriate conversion, real Macs).

MacOS Catalina support tweaks

As the new macOS Catalina was released recently, it was discovered that there was a unnecessary privilege used by the SDL2: It was listening to background key presses solely to track state of the caps lock key. Luckily, there was already a patch for this issue in SDL 2.0.10, so updating the SDL2 library (and adjusting some caps lock support code) fixed this problem.

(Please note that there will still be warning for unsigned applications, as to remove this Apple requires not only signing, but since 10.15 also notarization of any downloaded MacOS applications, both which require the paid Apple Developer Program membership.)

New test application: Continuum 1.04

Thanks to the personal permission by Brian Wilson, in combination with the writable bundle support we added this weekend, we have now Continuum 1.04 available as a fully functional MACE app bundle (including planet editor):

You can try the game out by visiting the downloads section. If you would like to know more about Continuum and its history, you can also visit the Continuum web page Brian Wilson has created here.

Other new test applications: Mac Concentration and GunShy 1.3

Thanks to the small fix done to the bug in SANE FP68K trap’s FMULD selector, Mac Concentration now works perfectly (it was previously stuck in game board generation loop due to the messed up multiplication results).

With the writable file system support, GunShy 1.3 is also finally feature-complete as a application bundle, as players need to be able to save & load games.

Like Continuum, these two new app bundles are available freely for trying out in the downloads section.

Full list of changes since last post

2019-10-27 07:49:25 +0200 • Relocate VFS to writable path on Mac OS X startup 
2019-10-27 04:52:17 +0200 • Update Alice, Continuum and GunShy JSON configs   
2019-10-27 04:45:56 +0200 • Adjust CapsLock check to work with new SDL version
2019-10-27 04:44:03 +0200 • Update to SDL2.0.10 to remove scary Catalina popup
2019-10-27 04:21:42 +0200 • Tweak InvalRgn/InvalRect to work better in Conti..
2019-10-26 05:17:58 +0300 • Update cmake JSON config for MacConcentration     
2019-10-26 04:25:41 +0300 • Adjust VBL routine call order for sound refresh   
2019-10-26 03:50:40 +0300 • Fix FMULD bug (param1 not assigned correctly) 

Standard File Package

It’s been quite a while since last update, which is due to combination of us being busy real-life jobs, and focusing work on this project on a monolithic feature, which in this case is nothing less than getting the Standard File Package to work!

Features

This is list of the features which the M.A.C.E. Standard File Package emulation is currently capable of:

  • All 8 selectors for Pack3 (SFPutFile, SFGetFile, SFPPutFile, SFPGetFile, StandardPutFile, StandardGetFile, CustomPutFile, & CustomGetFile). The ones tested are highlighted, but others should work too.
  • Support for custom file dialogs (with assorted hacks to convert MFS-style dialogs, such as the “Save” dialog in MS-BASIC 2.00, to the new HFS-style format)
  • Supports both SFReply and StandardFileReply outputs
  • File listing using List Manager, which displays currently System 6-style SICNs, but with System 7.1+ style spacing (later expandable to use the real icons, when we get Desktop Manager interface)
  • Path popup menu, visually and functionally like on real System 7.x
  • Desktop folder support
  • Focus on selected items (including file list with highlight frame), alterable by mouse or tab key
  • Most of common SF key shortcuts: Clipboard shortcuts, Cmd+S (Save), Cmd+O (Open), Cmd+N (New folder), Cmd+D (Go to desktop), Cmd+Up (Go to parent), Cmd+Shift+Up (Go to root/desktop), Cmd+Down (Open folder), (Cmd+Left/Right to browse volumes not yet implemented)
  • Filename field input filtering (prevents “:” from being used in filenames, which is path separator). Also, copy-pasted filenames will be filtered with proper alert messages.
  • Most of SF items/hooks implemented:
    • Items: sfItemOpenButton, sfItemCancelButton, sfItemVolumeUser, sfItemEjectButton, sfItemDesktopButton, sfItemFileListUser, sfItemPopUpMenuUser, sfItemDividerLinePict, sfItemFileNameTextEdit, sfItemPromptStaticText, sfItemNewFolderUser,
    • Hooks: sfHookFirstCall, sfHookCharOffset, sfHookNullEvent, sfHookRebuildList, sfHookFolderPopUp, sfHookOpenFolder, sfHookLastCall, sfHookGoToDesktop
    • Not yet implemented: sfHookOpenAlias, sfHookGoToAliasTarget, sfHookGoToNextDrive, sfHookGoToPrevDrive, sfHookChangeSelection, sfHookSetActiveOffset
  • File type filtering supported, both by typeList and FileFilterProc
A GIF animation showing the progress done in the past month, from scratch to the latest version

Known issues

Although most of the Standard File Package is functional, and usable, there are still some issues left:

  • Because DirCreate is not yet implemented in NativeFS layer, FindFolder cannot create the Desktop Folder/Trash folders to the file system, thus at least Desktop Folder needs to exist to allow access to “Desktop” level in the SF dialogs. This also prevents “New folder” from actually creating the folders.
  • Eject button does not yet do anything, as the NativeFS volumes are non-ejectable
  • There are minor differences to real System 7.x style SF dialogs, for example with certain elements being a few pixels off their intended locations, and popup menu’s width not always expanding
  • Some custom dialogs don’t yet work exactly as they should, for example THINK Pascal 4.0 “Add files” dialog erases left edge of the applications own custom list located below the SF file list, and for some reason it allows attempts to add folders as source files…
  • Location of the SF dialog is sometimes not correct (for example, Railroad Tycoon’s “Save game” dialog is partially under the menu bar)
  • We cannot yet create app bundles for the currently working freeware/shareware applications that use StandardFile, because when sandboxed the bundle file system is read-only, and thus attempting to save anything to it would straight out fail. To work around this, we need to implement certain changes to the way file system in bundled applications is handled, but we are already working on that for the next release

Other improvements made for Standard File

There were a couple features that were implemented as part of Standard File Package implementation, some of which were necessary to make it work

NativeFS AVL tree balancing and iteration support

One important feature was the need for catalog iteration, which is used by Standard File dialogs to enumerate the files in current folder to be displayed in the file list. This allows for example GetCatInfo to use ioFDirIndex in the parameter block to index specific files inside given directory.

Also, this gave a good excuse to finally implement the actual AVL balancing algorithm in the NativeFS file system, which until now had behaved mostly as a dummy binary tree. Below is a screenshot of how the file system structure was changed by the balancing algorithm:

“Before” and “After” debug visualizations of the NativeFS binary tree data structure (click for larger view)

On the right-hand side, is the unbalanced “dummy” binary tree, and on left-hand side is the same file system, when using AVL balancing algorithm to do the insertions. The deepest nodes were reduced from, in this case, from way over 100+ key comparisons into a rough maximum of 10+.

Popup Menu Control

One important feature of the HFS-style Standard File dialogs is the navigation aid provided by the path popup menu. For this, the Popup Menu Control CDEF 63 was implemented. On real Macs, this was originally part of Communications Toolbox, before it was merged to the System 7.x at some point. The Standard File dialogs use only a subset of its features, but foundation for most use cases was built at the same time, so taking it into more common use should be quite easy. As it uses the Menu Manager’s PopUpMenuSelect trap call, the actual menu selection was already there.

One fun and tricky part though was the need to add MDEF “wrapper” for SF popup path menu, to ensure that the opened popup menu would never be narrower than width of the popup control. To do this, the control imitates how the real Macs did this by adding temporarily a MDEF “wrapper”, which injects modified menuWidth value to the menu after it receives a mSizeMsg message from the Menu Manager, thus modifying the resulting popup menu size.

Picture Button Control

The CDEF 61, picture button, was implemented just to allow adding the “New folder” button to the “put” dialogs. It is quite simple, at least in monochrome version, just drawing a PICT resource as content of a regular push button

Dialog Manager improvements

The StdFilter, which is also used by Standard File dialogs, was improved to add features used by it:

  • Outlining default button with FrameRoundRect
  • Handle “cancel” item’s shortcut key as escape/cmd+dot
  • Track cursor to turn it into i-beam over enabled textedit fields
  • Handle update events for default items

Also, the default shortcut keys (Cmd+X/C/V for Cut/Copy/Paste) were also implemented in DialogSelect trap.

Other news: 68020/68040 support

In other news, Pukka has also been busy adding 68020/68040 support, which includes:

  • Bitfield instructions (bfins, bfset, bftst, bfchg, bfclr, bfextu, bfffo)
  • cas2.b/w/l instruction
  • divs.l/divu.l 32-bit division instructions
  • move16 instruction for 68040
  • Disassembler support for the new and improved instructions
  • Various other improvements

Full list of changes since last post

2019-10-21 02:25:22 +0300 • GetResource falls back to TopMap if CurMap invalid
2019-10-20 23:00:18 +0300 • Fix byteswap bug in LDEF -4000 (item font & size) 
2019-10-19 21:44:23 +0300 • Update ArmorAlley JSON config, it now almost works
2019-10-19 02:18:39 +0300 • Fix _Allocate, use ioFRefNum/FCB to get the vcbPtr
2019-10-19 02:06:57 +0300 • Make MemoryManager allocation alignment adjustable
2019-10-19 01:54:24 +0300 • Fix AddResource bug, ResErr not cleared on success
2019-10-19 01:44:31 +0300 • Implemented RmveResource                          
2019-10-18 17:44:45 +0300 • Set ioFDirIndex for GetCatInfo in SF popup refresh
2019-10-18 17:21:28 +0300 • Desktop folder support improved in SFUpdateReply  
2019-10-18 17:08:07 +0300 • Show files in Desktop Folders on desktop          
2019-10-18 16:25:39 +0300 • Update StdFile Eject button enable/disable state  
2019-10-18 03:39:11 +0300 • Close app resource file on exit from emulator     
2019-10-18 03:07:05 +0300 • Draw CDEF 63 popup triangle & hack to resize menu 
2019-10-17 00:07:14 +0300 • Fix StdFile dirID byteswap bug for popup menu item
2019-10-17 00:05:13 +0300 • Rough proto of autoTrack to CDEF 63 popup tracking
2019-10-16 23:28:35 +0300 • Add sfHookFolderPopUp trigger & handler to StdFile
2019-10-16 23:27:14 +0300 • Implement testCntl in CDEF 63 (& refactor headers)
2019-10-16 22:47:54 +0300 • Merge branch 'master' of  
2019-10-16 22:47:13 +0300 • Fix popup menu creation (fix title & set icon cmd)
2019-10-16 22:45:09 +0300 • Work on CDEF 63 (draw menu in the control)        
2019-10-16 22:44:10 +0300 • Implement mDrawItemMsg in MDEF 0                  
2019-10-16 22:40:13 +0300 • fix initializing 68040 processor type and linkin..
2019-10-16 04:50:31 +0300 • Implement mCalcItemMsg in MDEF 0                  
2019-10-16 03:59:46 +0300 • Implement calculation of popup bounds in CDEF 63  
2019-10-16 03:58:56 +0300 • Make Mac_GetMenu visible in public headers        
2019-10-16 03:58:31 +0300 • Fix bugs in StdFile path popup folder iteration   
2019-10-15 02:25:13 +0300 • Work on CDEF 63 init, dispose and calc messages   
2019-10-14 21:40:29 +0300 • disassembler fixes and support for bit field ins..
2019-10-14 02:50:31 +0300 • Work on CDEF 63: Init StdFile dir popup contents  
2019-10-13 07:21:56 +0300 • Fix filename in GetFileInfo for ioFDirIndex > 0   
2019-10-12 23:57:53 +0300 • Fix old-style StdFile filelist bounds byteswap bug
2019-10-12 23:56:36 +0300 • Fix ioFDirIndex>0 case for GetFileInfo in NativeFS
2019-10-12 04:29:25 +0300 • "New folder" dialog added, and many fixes & tweaks
2019-10-12 02:09:45 +0300 • Cmd+Down in SF & fix list arrow when no selection 
2019-10-11 22:40:18 +0300 • Handle Cmd+S, Cmd+O, cmd+D and cmd+N in StdFile   
2019-10-11 22:21:15 +0300 • Fix crash in CDEF 61 dispCntl message handler     
2019-10-11 05:12:59 +0300 • Enable "new folder" button in StdFile put dialogs 
2019-10-11 05:11:37 +0300 • Implemented CDEF 61 ('PICT' button control for SF)
2019-10-11 02:44:50 +0300 • Don't try to go to the parent of Desktop          
2019-10-11 02:39:49 +0300 • Open parent with cmd+up, desktop with cmd+shift+up
2019-10-11 01:49:34 +0300 • Implement cut/copy/paste handling in DialogSelect 
2019-10-11 00:08:41 +0300 • Fix inverted zonecheck result in handle validation
2019-10-11 00:06:45 +0300 • Finish IsCmdChar implementation (& fix signature) 
2019-10-11 00:05:16 +0300 • Fix KCHR load from ROM (RomMapInsert & TmpResLoad)
2019-10-10 22:27:30 +0300 • Cache KCHR 0 in script manager expand mem globals 
2019-10-10 10:32:50 +0300 • Handle escape key in StdFilter                    
2019-10-10 10:32:26 +0300 • Add dummy IsCmdChar to ScriptUtil dispatcher      
2019-10-10 02:43:08 +0300 • Handle iBeam cursor in StdFilter null/updateEvents
2019-10-10 02:25:40 +0300 • Frame default button in StdFilter (also disabled) 
2019-10-10 01:53:25 +0300 • Handle desktop folder and related buttons         
2019-10-09 22:58:06 +0300 • Use new defines for FindFolder folders in StdFile 
2019-10-09 22:57:33 +0300 • Call DirCreate in FindFolder when createFolder set
2019-10-09 22:56:44 +0300 • Add dummy DirCreate placeholder to FSDispatch     
2019-10-09 22:55:41 +0300 • Merge branch 'master' of  
2019-10-09 22:55:37 +0300 • Fix font strike size to inRec size scale ratio    
2019-10-09 22:20:59 +0300 • disassembler support for move16                   
2019-10-09 21:54:33 +0300 • remove unused sr_bits field from instruction data 
2019-10-09 21:33:53 +0300 • finish implementing move16 instruction            
2019-10-09 21:26:06 +0300 • update xcode 6 project                            
2019-10-09 17:23:48 +0300 • Fix FindFolder memory corrupt (ioNamePtr not set) 
2019-10-09 11:16:53 +0300 • Fix byteswap bug in fld# type comparison          
2019-10-09 11:15:00 +0300 • Fix fld# search to work on non-system folder lists
2019-10-09 03:46:40 +0300 • Add linecount script                              
2019-10-09 03:46:28 +0300 • Add empty desktop/trash folders to THINK Pascal   
2019-10-09 03:32:46 +0300 • AliasManager added, to implement FindFolder for SF
2019-10-09 02:42:02 +0300 • Implemented HOpenResFile                          
2019-10-08 00:43:44 +0300 • Implement sfHookCharOffset and sHookRebuildList   
2019-10-07 21:56:21 +0300 • Found hidden ListManager selector, used by StdFile
2019-10-07 21:55:09 +0300 • Add PageUp/Down and Home/End keys to SDL keymapper
2019-10-07 04:57:10 +0300 • Use inThumb also to draw control on SetCtlMin/Max 
2019-10-07 04:55:48 +0300 • Fix wrong zero-case part code bug in HiliteControl
2019-10-07 03:16:46 +0300 • Fix ignore dot & '%'-prefix items in InsertResMenu
2019-10-07 02:19:28 +0300 • Fix memory leak in DrawPicture                    
2019-10-07 02:18:10 +0300 • Fix conversion of uncompressed pixmap data to b&w 
2019-10-06 23:37:38 +0300 • fix divs.l, start implementing move16 instruction 
2019-10-06 00:00:12 +0300 • allocate move16 instruction                       
2019-10-05 23:07:36 +0300 • fixes for divs.l/divu.l instruction               
2019-10-05 22:00:21 +0300 • Merge branch 'master' of  
2019-10-05 21:59:44 +0300 • implement chk2.b/w/l instructions                 
2019-10-05 21:57:57 +0300 • Merge branch 'master' of  
2019-10-05 21:57:22 +0300 • Fix resource name remove bug                      
2019-10-05 21:27:28 +0300 • implement cas2.w and cas2.l instructions          
2019-10-05 20:34:46 +0300 • finish cas.b/w/l instruction implementation       
2019-10-05 04:50:14 +0300 • Fix bug in FindWindow (theWindow not initialized) 
2019-10-05 04:10:11 +0300 • Merge branch 'master' of  
2019-10-05 00:11:35 +0300 • make bsr instruction branchless                   
2019-10-04 23:43:40 +0300 • THINK Pascal def missing from root CMakeLists.txt 
2019-10-04 23:25:35 +0300 • register variants for bfins, bfset and bftst      
2019-10-04 23:05:14 +0300 • remove unused c16 param from bfxxx_read/write     
2019-10-04 22:42:22 +0300 • use read_pc_immediate for bit field extension     
2019-10-04 22:30:25 +0300 • fixes for bit field instructions                  
2019-10-04 01:06:20 +0300 • Add THINK Pascal 4.0 test app JSON config to CMake
2019-10-04 01:06:00 +0300 • Implemented MaxSizeRsrc                           
2019-10-03 22:25:37 +0300 • Reinforce fake handle handling in memory manager  
2019-10-03 22:21:15 +0300 • bfchg, bfclr, bfext, bfextu and bfffo instructio..
2019-10-03 22:09:06 +0300 • Revert "bfchg, bfclr, bfext, bfextu and bfffo in..
2019-10-03 22:08:15 +0300 • bfchg, bfclr, bfext, bfextu and bfffo instructio..
2019-10-03 04:04:45 +0300 • Escape MacRoman filenames & avoid UTF8 conversions
2019-10-03 00:27:19 +0300 • SF key filtering (max length, skip ':' & CR, etc.)
2019-10-03 00:26:24 +0300 • Support "autoKey" in keyboard and platform modules
2019-10-02 02:55:34 +0300 • Handle tab key in SFDialogFilter                  
2019-10-02 02:14:00 +0300 • Tweak list ListManagerClickTrackProc visible size 
2019-10-02 02:00:26 +0300 • Fix missing sfHookLastCall set on SF dialog close 
2019-10-02 01:58:50 +0300 • Escape characters in NFS (and fix full-path bug)  
2019-10-02 00:08:29 +0300 • Adjust Art Class JSON to use 68020 machine config 
2019-10-02 00:07:57 +0300 • Update SF reply records during event loop         
2019-10-02 00:07:24 +0300 • Handle invalid names and replace in "save" dialogs
2019-10-02 00:05:56 +0300 • Reduce keyboard debug logging                     
2019-10-01 22:34:26 +0300 • Fix missing origName in SFPutFile                 
2019-10-01 22:06:24 +0300 • Merge branch 'master' of  
2019-10-01 22:06:17 +0300 • Tweak Gestalt and fix M68kHelper for 68020 support
2019-10-01 21:35:17 +0300 • kemu header cleanup                               
2019-10-01 21:33:14 +0300 • update xcode6 project                             
2019-10-01 20:38:00 +0300 • 020 bit field instructions, optimizations, some ..
2019-10-01 17:01:23 +0300 • First successful "open" from Standard File        
2019-10-01 15:37:53 +0300 • Missing header update for no-read privilege flag  
2019-10-01 15:37:03 +0300 • Add missing part of previous LayerDispatch fix    
2019-10-01 15:36:20 +0300 • Handle cancel button & implement closing dialogs  
2019-10-01 15:20:57 +0300 • Handle opening of folder in file list             
2019-10-01 15:20:00 +0300 • Don't hilite disabled items in file list lDrawMsg 
2019-10-01 14:58:06 +0300 • Error dialog support for Standard File            
2019-10-01 14:57:19 +0300 • Add some more TODOs to StdFilter proc             
2019-10-01 14:56:52 +0300 • Fix bug in AutoPositionWindow lcParentWindow case 
2019-10-01 13:42:47 +0300 • Fix redraw bug in file list item selection        
2019-10-01 13:40:58 +0300 • Detect when a folder is opened instead of file    
2019-10-01 13:39:08 +0300 • Fix accidental file list framing in "Get" dialogs 
2019-10-01 03:24:36 +0300 • Update save/open button state (and title)         
2019-10-01 01:21:11 +0300 • Add Desktop Folder and Trash to TeachText config  
2019-10-01 01:20:45 +0300 • Add empty folder creation support to make.cmake   
2019-10-01 01:06:51 +0300 • Go to parent folder by clicking volume name       
2019-09-30 02:06:23 +0300 • Handle more events in SF(update, activate & mouse)
2019-09-30 02:02:42 +0300 • Fix byteswap bug in FindDialogItem                
2019-09-29 03:39:31 +0300 • SF activation handling: set reply & frame filelist
2019-09-29 03:35:48 +0300 • Fix more maxIndex bugs in LGetSelect and LSearch  
2019-09-28 03:54:21 +0300 • Add file type filtering & call custom file filters
2019-09-28 03:53:38 +0300 • Fix bug in Standard File dialog autopositioning   
2019-09-28 03:28:31 +0300 • Fix SF special folder detection bytes-wap bug     
2019-09-28 03:27:37 +0300 • Fix ioDrDirID for folders in GetCatInfo           
2019-09-28 03:08:52 +0300 • Handle list clicking in SF dialog event handler   
2019-09-28 03:08:12 +0300 • Fix maxIndex case in LClick selection clearing    
2019-09-28 03:07:09 +0300 • Set mouseLoc in ListRec in LClick                 
2019-09-28 03:05:32 +0300 • Fix ioDirFlg setup in GetCatInfo                  
2019-09-28 01:59:31 +0300 • Re-enable EraseRect calls in SF dialog item redraw
2019-09-28 01:57:24 +0300 • Fix ioNamePtr handling in GetCatInfo              
2019-09-28 01:56:01 +0300 • Draw volume icon & name in Standard File dialogs  
2019-09-27 15:40:16 +0300 • Finalize LDEF -4000 drawing for standard file list
2019-09-27 15:38:52 +0300 • Fix file type mapping bugs in standard file pack  
2019-09-27 15:37:44 +0300 • Add Scratch20 low memory global                   
2019-09-27 04:21:58 +0300 • First real filelist shown in Standard File Dialogs
2019-09-26 22:56:17 +0300 • Add balancing algorithm to the AVL tree (+debug)  
2019-09-26 22:54:32 +0300 • Add missing header from previous commit           
2019-09-26 22:53:19 +0300 • Finish volume/directory setup in SF initialization
2019-09-26 11:30:42 +0300 • Reset to VCBQHdr after last VCB in SF volume scan 
2019-09-26 04:06:05 +0300 • Work on volume state initialization in SF package 
2019-09-26 04:03:16 +0300 • Tweak FMGetDrive to allow passing nil driverRefNum
2019-09-26 04:02:01 +0300 • Support GetVolInfo calls with drvNum as vRefNum   
2019-09-26 04:01:13 +0300 • Add four flag bytes in front of DrvQEl in NativeFS
2019-09-25 04:33:14 +0300 • Set default SF volume&folder to application folder
2019-09-25 04:11:00 +0300 • Add indexing to GetCatInfo + fetch real ioDrNmFls 
2019-09-25 04:09:38 +0300 • Add rough iteration support to file system AVLTree
2019-09-22 13:09:58 +0300 • Refresh file list in StdFile dialogs              
2019-09-22 05:17:32 +0300 • First StdFile dialogs visible, not yet functional 
2019-09-22 05:15:58 +0300 • Support opcodes 0x12, 0x13 and 0x14 in pictures   
2019-09-22 04:41:36 +0300 • More progress on StdFilePack                      
2019-09-22 04:39:36 +0300 • Minor List Manager fixes and tweaks               
2019-09-22 04:37:15 +0300 • Placeholder for popup CDEF (63)                   
2019-09-19 23:06:31 +0300 • Finalize CheckWindow selector for LayerDispatch   
2019-09-19 03:25:17 +0300 • More work on Standard File Package                
2019-09-19 03:23:39 +0300 • Add two new selectors to LayerDispatch for StdFile
2019-09-18 03:58:15 +0300 • Some work on Standard File Package                
2019-09-18 03:54:32 +0300 • Use _RelString in IUMagIDString for now           
2019-09-13 01:48:34 +0300 • Mac OS X 10.6.8 compatibility                     
2019-09-11 21:30:30 +0300 • Enable GetNextEvent CPU throttle for Arkanoid     
2019-09-11 18:25:41 +0300 • Add Crystal Quest and Armor Alley test apps       
2019-09-09 01:31:04 +0300 • Bump up runtime version for new bundles           
2019-09-09 00:58:48 +0300 • Tweak IAGO cmake config                           
2019-09-07 14:04:32 +0300 • Add two more test app configs, IAGO and EveryMan 1
2019-09-07 14:04:02 +0300 • Add substitution for missing fonts in Font Manager

A bit of nVIR-A experience, and a new test application

Infection compatibility

We had this week a rather interesting thing happen by accident, which was rather unusual: While trying out a couple potential test applications with the emulator, we ran into an application which was infected by nVIR A type virus ( https://en.wikipedia.org/wiki/NVIR ). What makes this interesting, is that it apparently was able to infect the System file of that application’s bundle, as we now have near-complete resource write support:

Above is partial output of the System file’s resource map debug dump (and Rezilla screenshot of infected System file), and the ‘nVIR’ and ‘INIT’ resources were actually written into the System file by the virus in infected application – and the system resource map seemed to be (mostly) still functional! I only found this out because a bug in resource writing corrupted Geneva font, causing the infected application to crash. A kind of controversial achievement, having good enough compatibility for even viruses to work in the toolbox emulation…

One of the positive sides of the current per-application bundling of files as isolated file systems, is that the infection never was able to break out of the application bundle. We have also scrubbed through all the disk images and Mac files downloaded we use for development and testing, to make sure there will not be any risk of infection at later time (and run Disinfectant on the source Macs/emulators regularly). These viruses are ancient, but can awake up at any time it seems…

New test application: IAGO

After we got the infection sorted out and everything back to normal, we experimented with a couple of new test applications, and found out that IAGO, which is a public domain game written by David Reed in 1984.

We were able to identify one bug in ROXLI.L instruction, which caused the game timer to not advance correctly. After it was fixed, the game works as smoothly as it does on a (fast) real Macs. It is now also available for testing as application bundle in the downloads section.

Performance improvements and menu stuff

The past few weeks have mostly gone to improving performance of the runtime environment. Although all the apps have been running smoothly without any issues, there was not much focus on improving the actual CPU usage of the host system.

Unlike hardware-based emulators such as Mini vMac, where everything including applications, ROM, and system software run on 68K, in M.A.C.E. there is distinct separation of execution between 68K based code, and the Toolbox code written in native C. This creates some minor issues with managing performance, as especially in regards of early software – QuickDraw routines would be much slower on real 68000 hardware, and thus games depending on certain 68000 execution speed would experience certain slowdown of using those QuickDraw (and other Toolbox) routines.

However, in M.A.C.E., the native code always runs at full speed, and those expectations break down as experienced in Stunt Copter, Zero Gravity, Harrier Strike Mission II, Brickles among some other applications. To mitigate this, there is an artificial slowdown that the emulator can optionally enable delay throttle in GetNextEvent, which we have so far used to make most of the aforementioned applications usable.

68K emulation throttle

Pukka added an experimental throttle to the 68K emulator, which attempts to slow down applications which do for example busy-looping and polling of TickCount to perform timing. In the old non-multithreading MacOS, this kind of timing was pretty common especially in games that required precise control on the CPU resources. This should in future help CPU usage in programs heavy on 68K code in game loops, such as Dark Castle and Apache Strike. This however does not help the cases which rely a lot on Toolbox, such as the applications which benefit of the GetNextEvent delay throttle.

SystemTask and execution yielding in Toolbox routines

As some may remember, when Apple transitioned from Mac OS 9 to X, one way to yield execution was to add SystemTask calls to blocking loops (in cases where WaitNextEvent would not be desirable). As the Toolbox in M.A.C.E. is completely implemented in C, we have the power to choose yielding strategy ourself, and for that we are experimenting at the moment with very small delays of 1 millisecond on each call to SystemTask, and any other busy-looping toolbox routines, which include:

  • MenuSelect and PopUpMenuSelect
  • TrackControl and DragControl
  • DragTheRgn (affecting DragGrayRgn and DragWindow), GrowWindow and TrackBox
  • TEClick
  • LClick
  • Delay (it had earlier implementation used by the GetNextEvent hack, but was implemented in a rather stupid way which ended up as a busy-loop on host CPU…)
  • File Manager synchronous command queueing
  • Device Manager synchronous command execution, and driver close

Video rendering and dirty-rectangle detection

Another optimization feature was the addition of detection of video memory “dirty rectangles”, which define which areas of video memory have changed, so that only those would need to be drawn to the screen. This optimization is being experimented especially to improve the future fullscreen and pixeldouble modes (which are not yet enabled due to requiring still some work), and depending on the machine and application, there is varying degree of performance improvement, but not very much (which is to be expected as the 512×342 screen with 1-bit colors does not require a lot of the CPU in regards of pixel format expansion and SDL surface updates).

With these changes, we got CPU usage of 100% to drop to around 7-10% on a Mid-2010 Mac Pro, and 9-12% on 2015 MacBook Pro of the currently bundled applications. Of course, some test applications we are experimenting with still require further work to gain these benefits, but this should help at least keep those fans more quiet on MacBook computers 🙂

We have also updated the test application bundles with these new optimizations in the downloads section on this blog.

Hierarhical menus and PopUpMenuSelect

In addition to performance optimizations, we also finally added the hierarhical menu and popup menu support which was for a long time waited to make especially Railroad Tycoon way more playable.

There was a bit of tweaking, bug fixing, polishing, more bug fixing and tweaking to get those two features to work, but now they appear to work exactly like on a real Mac (timing, positioning, mouse tracking behaviour, screen buffer saving, etc). Sadly we don’t have yet any bundled test applications to demonstrate this, but at least the applications we are using internally all work perfectly.

These are again another major features which gets us closer to the “general classic replacement” release, although there is still lot of work needed to get us there. The progress is however very promising at current pace.

Full list of changes since last post

2019-09-05 01:47:24 +0300 • Add dirty rectangle visualization to debug build 
2019-09-05 00:35:24 +0300 • Update runtime version in cmake script 
2019-09-04 12:44:03 +0300 • Fix offset of minimum-intersection dirty-rectangle 
2019-09-04 03:38:02 +0300 • Finally fix that darn StretchBits… 
2019-09-04 03:14:55 +0300 • Improved, faster dirty rectangles with bit-o-magic 
2019-09-03 22:35:25 +0300 • Implement GetItemIcon and SetItemIcon traps 
2019-09-03 22:21:14 +0300 • Add pixel-double support also to windowed mode 
2019-09-03 22:20:11 +0300 • Fix uninitialized mouse loc in PopUpMenuSelect 
2019-09-03 01:27:55 +0300 • Implement prototype dirty-rectangle video renderer 
2019-09-02 22:42:44 +0300 • Add EnvRunBackgroundTasksReleaseCPULockWithYield 
2019-09-02 20:43:19 +0300 • Fix hit-testing of hierarchical menus in MBDF 
2019-08-30 22:31:02 +0300 • Fix mouse button check in relative mouse mode 
2019-08-30 03:13:25 +0300 • Update trap list XLS/CSV 
2019-08-30 03:06:21 +0300 • Implement basic non-styled TEPaste 
2019-08-30 02:56:58 +0300 • Fix two masking bugs in CopyMask 
2019-08-30 02:07:25 +0300 • Implement pic opcodes 0x1A & 0x1B (RGBColor fg/bg) 
2019-08-30 02:04:08 +0300 • Fix popup menu crash 
2019-08-30 01:13:42 +0300 • Initial fullscreen (and pixel-double) support 
2019-08-28 22:25:51 +0300 • Enable delay hack in Brickles test app 
2019-08-28 22:25:34 +0300 • Add execution yield for 1ms to SystemTask 
2019-08-28 21:54:36 +0300 • Fix the earlier video update limiter adjustment 
2019-08-28 13:31:18 +0300 • Experimental change to lower CPU usage, has issues 
2019-08-27 23:54:36 +0300 • usleep logic to m68k helper 
2019-08-27 05:04:18 +0300 • First proto of PopupMenuSelect, has bugs 
2019-08-27 00:32:21 +0300 • Merge branch 'master' of  
2019-08-27 00:32:17 +0300 • Also handle shortcut keys for submenus in _MenuKey 
2019-08-26 21:46:33 +0300 • Merge branch 'master' of  
2019-08-26 21:45:28 +0300 • revert unwanted modification 
2019-08-26 17:02:42 +0300 • Fix left-hand side hierarchical menu handling 
2019-08-26 03:13:36 +0300 • Fix icon alignment in default MDEF 0 item drawing 
2019-08-26 02:30:48 +0300 • Merge branch 'master' of  
2019-08-26 02:30:42 +0300 • Add "bermuda triangle" to hierarchical menus 
2019-08-25 20:36:24 +0300 • move pc increment away from instructions + some optimizations 
2019-08-24 23:38:22 +0300 • Update instructions.c 
2019-08-24 23:17:34 +0300 • some optimizations and ea macros 
2019-08-24 15:33:05 +0300 • Improve hierarchical menu tracking 
2019-08-24 01:12:35 +0300 • Fix GetItemMark/Cmd/Style result byteswap bug 
2019-08-23 19:41:36 +0300 • Fix drawing of hierarchical menu items 
2019-08-23 04:36:38 +0300 • Merge branch 'master' of  
2019-08-23 04:35:57 +0300 • First rough proto of hierarhical menu support 
2019-08-22 21:51:52 +0300 • fix cmpm instruction with FAST_ADDRESSING 
2019-08-21 00:30:28 +0300 • Add cmake config for Brickles, which was missing 
2019-08-20 02:31:00 +0300 • Use rb fallback in NFSOpenFileHook is r+b fails 

A LOT of fixes, improvements and new test applications

It’s now approximately one month since last update, and a lot of “under the hood” type progress has been made.

M68K Tester and 68K bug fixes

One major task we had on roadmap was attempting to see how we could integrate Pukka’s emulator core with the amazing M68K Emulator Testsuite (by Gwenolé Beauchesne and Ray Arachelian, archived at this URL: http://web.archive.org/web/20090417202939/http://gwenole.beauchesne.info/en/projects/68ktester ). After holidays, Pukka was able to get the integration done, and we found a bunch of hard-to-find CPU bugs thanks to it:

Even though a lot of bugs were fixed, there are still test cases which do not pass, and will be fixed soon. However, with the fixes already done to the bugs that were found thanks to the test suite, Dark Castle and Beyond Dark Castle appear to be stable in rooms that used to crash before:

New relative mouse mode

A long-time task that has been waiting implementation has been the relative mouse mode, in which M.A.C.E. captures mouse cursor entirely, and instead of using the absolute position given by the host operation system, calculates mouse motion itself using the mouse delta vectors. This both prevents stray clicks outside emulated desktop window, and allows long delta swipes such as required to control helicopter in Apache Strike. As a result of this, Apache Strike can now be played, as previously it would immediately crash into the wall – and Pirates! no longer hangs in the swordfight at beginning – both which depended on certain low-level mouse low memory globals being emulated correctly:

Another fun feature which we added to complement the relative mouse mode as experimenting with game controller input API of SDL, which took about 30 minutes to do and works quite OK as seen on this video:

Apache Strike controlled using USB gamepad

Another issue that was fixed was missed clicks problem with “Tap to Click” MacBooks, which now should generate mouseDown events correctly.

Currently the cursor itself is still displayed using SDL “hardware” cursor API, but with the recent changes adding support for software cursor should not be a big problem – One that might be required to have decent cursor is certain games, where the cursor visibility depends on the cursor having ability to “invert” depending on underlaying graphics (such as the crosshair on black background in lemmings), which the SDL hardware cursor does not support.

Keyboard text input rewritten with proper character mapping

In first iteration of keyboard handler, we only had scancode support with minimal MacRoman mapping (without modifiers), which although was working for the simple use cases, did not allow flexibility required for example to enter upper-case letters.

For second iteration, we attempted to utilize host operating system keyboard layout mapping by using SDL text input API, and although it worked nicely for TextEdit text input, broke command keys and other things (such as Dark Castle & Lode Runner controls) horribly.

For the latest, third iteration, we went back to scancode based input, but instead are simulating the exactly same type KCHR scancode to MacRoman mapping like on real Macs. This allows every key to work properly in text editing, game input and menu shortcut etc, but one downside is that for international layouts we will need to have separate KCHR resources, currently only having U.S. as default layout. However, as “dead keys” are also working, typing special characters such as accents, umlauts, etc is working nicely even on this layout.

ResEdit bug “treasure chest”

ResEdit proved to be a real “treasure chest” of bugs and improvements, as it used literally every possible List Manager routine, and it also surfaced some unexpected bugs in other places such as TextEdit, Dialog Manager, QuickDraw, and other already heavily used toolbox traps.

Some major improvements found through testing ResEdit:

  • Most of List Manager calls were implemented, including resizing lists, deleting rows/columns, searching data. Also some bugs were fixed, such as corruption of data offsets in some cases, etc.
  • Dialog manager edit field handling had a number of bugs, which surfaced in the DLOG editor, and there also were bugs in item hiding/showing which were also manifesting in Civilization.
  • TextEdit had some bugs in the MixedMode handling of special case calling conventions, which caused crash in the hex editor. After fixing them, not only the hex editor works, but also MacPascal editing bugs were fixed, as it also used custom TEDoText hooks.
  • The PACK 1 (“BitEdit”) package in ResEdit surface a couple QuickDraw bugs, such as random SeedFill memory corruption (using the paint bucket tool) and StretchBits memory corruption. These also make MacPaint more stable, which was also suffering from the SeedFill bug.

Some of these fixes had quite universal effects, such as improving the previously buggy behaviour of Lists and TextEdit in HyperCard 1.x:

Other improvements

  • Text scaling and measuring had some bugs, which were fixed so that MacPaint and Macwrite can display text of all sizes, faces and styles.
  • MacWrite now works, at least kind of…all other edition options are working (styling, page breaks, font sizes, fonts, rulers, etc), except adding newline using enter/return causes text to corrupt.
  • As the region corruption bug in polygon rendering was fixed, PT-109 appears now the be completely stable.
  • With “dummy” SetPalette trap implemented, Railroad Tycoon is now able to proceed past first “End of Fiscal Period” report
  • Scrollbar (CDEF 1) implementation (and related Control Manager TrackControl/DragControl stuff) were finally finished, so the indicator thumb dragging works now.
  • Prototype of font fractional width support added, only test case for this is curently Photoshop, which enables them. Right now only FOND width tables are supported, but extended width tables could be easily added as soon as some application needs them.

New test applications added

Also, we have now finally two new test app bundles that should be complete enough for trying out. These are ZeroGravity by Duane Blehm, and Blob Manager Demo by Paul DuBois. They are now available for trying out in the Downloads section.

Additionally, the old test application bundles have been updated with the new M.A.C.E. runtime, which includes all the bug fixes and improvements done since their last update. For example, Stunt Copter uses now the new relative mouse mode, and all apps have been updated to support the “Tap to Click” trackpad mode on MacBooks.

Full list of changes since last post

There are too many changes to detail in a single post, so as a new feature below is the version control log of all changes made since last update:

2019-08-19 22:54:49 +0300 • Add SetDialogDefaultItem/CancelItem/TracksCursor 
2019-08-19 22:47:32 +0300 • Tweak TrackControl and LClick delay to 8 ticks 
2019-08-19 18:15:47 +0300 • Save alert default item also in the DialogRecord 
2019-08-19 16:23:03 +0300 • Partial support for default dialog StdFilter proc 
2019-08-19 07:03:12 +0300 • Bumb the M.A.C.E. version number & use current date 
2019-08-19 07:02:52 +0300 • Replace printfs with log, and add a few DEBUG #ifs 
2019-08-19 07:00:17 +0300 • Fix the SDL window title for now by using EnvStartupFile 
2019-08-19 06:59:57 +0300 • Tweak Blob Mgr Demo and ZeroGravity for release 
2019-08-19 06:00:30 +0300 • Work on FOND widthtable and fractional size support 
2019-08-17 03:17:27 +0300 • Work a little on resource map "shadowing" handling 
2019-08-17 03:04:32 +0300 • Fix WaitMouseUp to not return 1 extra "down" state 
2019-08-17 02:23:40 +0300 • Fix SeedFill random memory corruption case 
2019-08-15 23:36:01 +0300 • Merge branch 'master' of  
2019-08-15 23:35:56 +0300 • Fix TextEdit cursor right-side clamping 
2019-08-15 22:55:05 +0300 • add xcode 6 project 
2019-08-15 22:43:06 +0300 • Fix incorrect register usage in TEDoText overrides 
2019-08-15 21:21:29 +0300 • Fix StretchBits stack buffer overrun 
2019-08-14 23:46:30 +0300 • Fix scrollbar redraw bug in List Manager 
2019-08-14 23:25:48 +0300 • SetPalette classic-QD dummy trap 
2019-08-14 21:44:36 +0300 • Fix LSetSelect highlight bugs 
2019-08-14 19:28:32 +0300 • Fix lNoNilHilite flag handling in LSetSelect 
2019-08-13 03:14:24 +0300 • Fix value clamping in Control Manager 
2019-08-13 03:11:51 +0300 • Tweak CloseWD stub to not crash 
2019-08-13 03:11:28 +0300 • Implement LClrCell and proto of LDelRow/LDelColumn 
2019-08-12 21:10:26 +0300 • Fix a couple byteswap bugs in lists (selection) 
2019-08-12 17:39:11 +0300 • Fix one-cell delta calculation in list pageup/down 
2019-08-12 15:03:44 +0300 • Fix maxIndex bug in LNew 
2019-08-12 02:34:18 +0300 • List Manager LSize implementation 
2019-08-11 05:12:39 +0300 • Finished LAutoScroll, another easy one for ResEdit 
2019-08-11 04:56:40 +0300 • Implemented C_LFind, was simpler than I thought… 
2019-08-11 04:46:28 +0300 • Finish LNextCell trap, work on LClick multiselect 
2019-08-11 02:18:20 +0300 • Implement C_LClick scroll indicator handling 
2019-08-11 02:06:55 +0300 • Fix dividend overflow in scrollbar CDEF 1 rounding 
2019-08-11 01:43:52 +0300 • Implement DragControl, finish TrackControl & CDEF1 
2019-08-10 03:41:37 +0300 • Fix caps lock usage in SDL key event handler 
2019-08-10 02:01:37 +0300 • SDL gamecontroller support "hack" to Apache Strike 
2019-08-09 20:45:44 +0300 • Add FSpOpenDF, FSpCreate, FSpDelete, FSpGetInfo, FSpRename & (dummy) _Rename 
2019-08-09 18:08:02 +0300 • Hacked procinfo for UIMagIDString call in LSearch 
2019-08-09 03:28:00 +0300 • Fix buffer overflow in GetVolParms 
2019-08-09 02:42:54 +0300 • Added dummy GetVolParms selector to _FSDispatch 
2019-08-09 02:00:31 +0300 • Add Jigsaw test app 
2019-08-09 02:00:10 +0300 • Implement LSearch, and (dummy stub) IUMagIDString 
2019-08-09 01:58:42 +0300 • Implemented 1-bit NewGWorld in QDExtensions 
2019-08-08 17:10:34 +0300 • Fix allocation of key state buffer 
2019-08-08 02:45:45 +0300 • C_KeyTranslate Deadkey support + fix mac scancodes 
2019-08-07 18:30:41 +0300 • Fix dialog edit field handling 
2019-08-07 12:13:26 +0300 • Fix picture recording of port fgColor/bkColor 
2019-08-07 04:49:10 +0300 • Add git log generator, for fun 🙂 
2019-08-07 04:21:32 +0300 • Tweak type of ScrapSize in low memory 
2019-08-07 04:21:04 +0300 • Fix fromMode handling in Picture Line recording 
2019-08-07 04:19:58 +0300 • Fix picture line recording Point saving 
2019-08-07 04:17:59 +0300 • Fix Hide/Show dialog item bounds calculation 
2019-08-07 04:15:36 +0300 • Prevent editfield citation replacements in dialogs 
2019-08-06 19:43:28 +0300 • Fix Scrap Manager bugs 
2019-08-06 19:41:31 +0300 • Clean up Mouse module a bit 
2019-08-06 19:07:03 +0300 • Fix bug in DefaultShowCursor 
2019-08-06 16:38:47 +0300 • Un-obscure cursor before _MDrawCursor in CrsrTask 
2019-08-06 01:42:37 +0300 • Make phase-shift adjustment configurable 
2019-08-06 00:20:31 +0300 • Move audio phase-shift from RAM to ClassicSound 
2019-08-06 00:17:50 +0300 • Disable VM_MAP and reverse-page mapping 
2019-08-05 20:44:39 +0300 • Disable a couple of mouse debug log calls 
2019-08-05 00:10:21 +0300 • Updated some test apps with relative mouse mode 
2019-08-05 00:09:22 +0300 • Refactor cursor to support relative motion mode 
2019-08-04 00:30:38 +0300 • fix abcd, partially fix sbcd instruction 
2019-08-03 23:20:09 +0300 • fix zero status for subx.b/w/l instruction 
2019-08-03 00:40:07 +0300 • fix add.b, addx.b/w/l and negx.b/w/l status bits 
2019-08-03 00:40:01 +0300 • Merge branch 'master' of  
2019-08-02 00:05:57 +0300 • Fix TEIdle blinker 
2019-08-01 23:10:07 +0300 • Implement LLastClick and tweak trap glue generator 
2019-08-01 16:03:22 +0300 • Disable the extra log debug trace of heap compact 
2019-08-01 16:02:28 +0300 • Reduce logging in MixedMode calls 
2019-08-01 16:01:44 +0300 • Fix size bug in WriteResource 
2019-08-01 16:00:47 +0300 • Fix resProtected flag check in ChangedResource 
2019-08-01 15:58:51 +0300 • Fix dialog edit field disabled check + others 
2019-08-01 05:19:49 +0300 • Add missing lowmem changes for CurActivate bug fix 
2019-08-01 05:19:04 +0300 • Work on KeyTranslate KCHR mapping (fix keyboard) 
2019-07-31 01:21:08 +0300 • Add MacConcentration test app 
2019-07-31 01:20:39 +0300 • Implement FADDD & fix major 64-bit float SANE bug 
2019-07-31 01:08:21 +0300 • fix lsl and lsr instruction for shift > 31 
2019-07-30 22:53:46 +0300 • fixes for m68k tester 
2019-07-30 21:50:19 +0300 • move x68k xcode6 project to x68k root 
2019-07-30 21:48:42 +0300 • add m68k tester 
2019-07-30 21:46:25 +0300 • make kemu lib c++ compatible 
2019-07-29 15:51:21 +0300 • Tweak GunShy config to use version 1.3 
2019-07-28 04:54:27 +0300 • Fix window activation handling 
2019-07-27 04:36:27 +0300 • Fix GetFontInfo for scaled fonts 
2019-07-27 04:22:21 +0300 • Fix bug in fast-case of text blitter 
2019-07-27 02:13:56 +0300 • Set Lo3Bytes lowmem global properly 
2019-07-27 02:12:01 +0300 • Implemented ReadDateTime and (dummy) SetDateTime 
2019-07-27 01:26:28 +0300 • Fix LActivate 
2019-07-27 00:46:46 +0300 • Fix cursor leak 
2019-07-26 18:29:56 +0300 • Tweak InitResources to be (again) re-callable 
2019-07-26 17:32:20 +0300 • Fix UPP ProcInfo of DABeeper callback 
2019-07-26 15:46:55 +0300 • Implemented LActivate selector in List Manager 
2019-07-26 05:45:33 +0300 • Fix bug in resource manager name remove code 
2019-07-26 05:44:44 +0300 • Add Glider4 test app 
2019-07-26 05:44:24 +0300 • Fix possible bug in SetVol NFS handler 
2019-07-26 05:43:07 +0300 • Setup CurrentA5 during boot stage 2 for INITs 
2019-07-26 02:55:49 +0300 • Re-fix PurgeSpace to also return correct D0 
2019-07-26 01:52:59 +0300 • Optimize FramePolygon case 
2019-07-26 01:43:57 +0300 • Actually update the existing ADF header 
2019-07-26 01:36:18 +0300 • Finish SetFileinfo handler in NativeFS 
2019-07-26 01:35:28 +0300 • Fix region corruption bug in polygon drawing 
2019-07-25 23:16:56 +0300 • Fix MapRgn duplicate inversion point bug 
2019-07-25 23:07:21 +0300 • Fix MapPt negativity check 
2019-07-25 22:08:06 +0300 • Fix size of ScriptUtil selector 
2019-07-25 22:02:10 +0300 • Fix edge case bug in UnpackBits 
2019-07-25 21:02:00 +0300 • Fix ADF file create bug, mixed up rsrc and data 
2019-07-25 19:27:32 +0300 • Also reserve space for mapping in the stack after packBuffer 

Update on resource fork writing, ResEdit, and resource ‘dcmp’ decompressors

Resource write support status

So, the plan since last update was to do some work on resource write support, and there was indeed some progress. The nature of how resource forks are handled on disk compared to memory is quite interesting; After the resource map is read into memory, the resource fork on disk only needs to preserve resource data, and thus the resource manager has quite free hands on how it can move data around in the resource fork:

  • When a new resource is added using AddResource, it gets both a resource entry in the in-memory resource map, AND the resource fork gets expanded (to reserve space on disk), and on-disk location for the resource is appended after the current data area on the disk
  • When ChangedResource is called on existing resource, its size is checked to see if it grows or shrinks. Shrinking is more simple, as its data length just gets shorter, and does not need space allocation. However, if a resource grows, it needs more space allocated to it, so while the resource file is open, the resource manager simply allocates new space at end of resource fork, like it does in AddResource, and leaves the old location as “gap” in the file. This feature actually is why apple warns in the Inside Macintosh documentation, that calling ChangedResource often without calling UpdateResFile may lead to large amount of disk space used, as each new allocation expands the file size by the new allocation length!
  • When WriteResource is called, a resource with resChanged flag will get contents of its handle written to the location specified by resource entry in the resource map
  • UpdateResFile does the real “house-keeping” of resource fork, as it will iterate through all resources, removing any gaps in the resource fork, moving data to the proper places (a process called “compating” of resource fork). After this, the resource map is written at end of the resource fork (after the data section), and file size is truncated to minimum required, releasing the extra disk space.

Currently, most of write support is in place, except that resource fork compacting still needs finalization to make it work properly

Extended resources and ‘dcmp’ decompression

Of course a natural way to stress-test resource manager is to try ResEdit, which I think uses literally almost all of resource manager features. However, an interesting surprise was that ResEdit seems to utilize resource compression on a large number of its own resource, which led to the following MacTech article:

http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html

Which had nice, although brief, insight on ‘dcmp’ resources and their role in resource decompression. Adding the parsing of extended resource headers was not that difficult, and luckily ResEdit contains its own ‘dcmp’ decompressors, so they could be immediately be tested, resulting in following type of logs:

Debug log of ResEdit’s ‘PACK’ resource ID=1 decompression.

This snippet of debug log shows ResEdit’s ‘PACK’ resource ID=1 decompression procedure, when Get1IndResource was called. This shows how on-disk compressed data of 19568 bytes gets decompressed into 27200 bytes in memory, using a ‘dcmp’ ID=0 decompressor which is contained within ResEdit as 68k code.

ResEdit

And of course, with the decompression working, and with a lot of fixes to file manager, list manager and various other places, ResEdit magically appears to be starting to work, one bit at a time 🙂 In this test case, we made a copy of ResEdit itself as “Test File” file, which was hard-coded for now as reply to CustomGetFile call to get it open. And this is what we got so far:

Naturally there’s still a lot to fix, but surprisingly many features seem to work out-of-the box. But at least the following problems were immediately noticeable:

  • The actual resource writing does not yet work properly
  • List Manager still has some calls which are not properly implemented which leads to a number of assertions, and most likely causes the missing icons in ‘ICN#’ editor.
  • Font scaling has issues which is why dialog texts appear weird in the preview of ‘ALRT’ editor
  • A couple IconDispatch/ScriptUtil calls which need to implemented for resource file info dialog (mostly to get Finder labels, and format the file dates)
  • Proper multi-select support for List Manager
  • A bunch of other minor issues, such as text missing from a couple places, etc.
  • Probably other issues in features not yet tested

It’s still a bit too early to start testing the actual resource write support, but it won’t be far away from now after ResEdit is made more stable first.

Package Manager update

One small, but important thing which was fixed in this update is an actual, proper ‘PACK’ resource support. This is because ResEdit uses Apple’s ‘PACK’ ID=1 to implement its “BitEdit” graphic editor (used at least in ‘ICN#’ resource editor). Until this point, all packages could be implemented as regular dispatcher calls, but for the ResEdit to work we actually had to load the ‘PACK’ resource in InitPack, and register it as one of the AppPacks in low memory globals. A quite small change, but one which ResEdit seems to be happy with, for now 🙂

Ps. As another interesting side effect, with bug fixes done to toolbox HyperCard can now both close files properly (allowing us to navigate back to Home stack), and also HyperTalk editing seems to work well enough that changes to the script source actually affect behaviour of the stack 🙂

Second mid-summer update: Post-roadtrip, TextEdit progress, Clipboard…

Okay, so both members of development team have been busy being on vacation trips for the past few weeks, but we’re again back (although still on “kind of” summer break). To celebrate this we decided to each post a picture of what we’ve been up to, but to make this appropriate to topic of this blog, we do this by showing the photos using M.A.C.E running Photoshop 🙂

Road trip vacations

First, Pukka went on a road trip to Eastern Finland with his good old trusty Volvo station wagon for one week:

Pukka went on a road trip to Eastern Finland with his good old trusty Volvo station wagon for one week

Toni went on a two-week road trip to Northern Germany, visiting 12 cities in 14 days, including taking this amazing train from Hamburg to Kiel:

Toni went on a two-week road trip to Northern Germany, visiting 12 cities in 14 days, including taking this amazing train from Hamburg to Kiel

TextEdit progress (and Scrap Manager)

During this two weeks of visiting a lot of places, Toni had a couple evenings time to do some work on (non-styled) Text Edit, which now is nearly complete, and is able to run TeachText with almost all text editing features working:

TeachText works now on M.A.C.E.!

These are the notable features which work now in TextEdit:

  • Multi-line text display, with scrolling support (viewRect/destRect offsetting)
  • TEKey text input (still a bit hacky but functional)
  • TEClick with autoscrolling support, double-click word selection (using TEFindWord), and fExtend support for “shift-key”-type extending of selection, and recognition of left/right side of character for proper caret positioning
  • TextEdit hook support for nearly all TE Hooks (TEDoText, TERecalc, TEWordBreak, ClikLoop, TEFindWord, TEFindLine, TETrimMeasure, EOLHook, HighHook, etc…) which can be customized by 68K apps
  • Caret support with TEIdle blinking, TEKey cursor key movement (including multi-line support), and multi-line selection range support
  • Nearly all “quirks” of regular System 7.x TextEdit implemented as they work on real Mac; including the weird way caret gets positioned when moving across line boundaries using left/right arrow keys if previously clicked on right-hand side of a character, etc…
  • Dialog Manager edit field support, with System 7-type multi-line text support (i.e. edit fields with one line get extended horizontally 2x size, and scroll both horizontally and vertically, while multi-line edit fields only scroll vertically), “tabbing” between edit fields, and complete TEKey/TEClick support
  • Maybe some others I’ve forgotten to mention

Additionally, as TeachText wanted to use also Scrap Manager for edit functions, in parallel to implementing TECut/TECopy which use private TextEdit scrap, the implementation of generic Scrap Manager was finalized so that Clipboard file is fully supported (previously only in-memory state of scrap was implemented).

However, TEPaste still needs to be added for a 100% completion of “Edit” menu features for TeachText – and also, styled TextEdit work has not yet been started, which will be required in future to support more complex apps such as SimpleText and HyperCard 2.x.

Next up, on Toolbox API side, work is being done on Resource Manager’s resource fork write support, mostly just because after Scrap Manager was implemented, SuperPaint was able to progress a bit further in it’s startup, but wants to create “SuperPaint Prefs” file using Resource Manager’s functions, so might as well add that next. As of writing this, there’s already support for AddResource, ChangedResource and WriteResource, but we need to add resource fork compacting support to UpdateResFile to actually write the file properly. But we’ll get back to that later.

Bonus pic – for fun 🙂

Meanwhile, here’s a funny pic of what it looks when we run 28 M.A.C.E. instances at once 🙂 All quite responsive even though they’re all in full debug mode, although same can’t be said about the old Mac Pro when it was running them… 😀

28 individual M.A.C.E. applications (in debug mode) running at once 🙂

Mid-summer update: TextEdit progress

We’re still on summer “break”, but there has been a few evenings time to work on some TODO items, lately especially on the TextEdit implementation. It’s still far from complete (the basic, single-style TextEdit – Styled TextEdit is another undertaking on its own!), but there has been some progress that has already some visible effects.

For example, one of the games we’ve been testing emulation with, Scarab of RA, relies EXTREMELY lot on TextEdit, including quest log, inventory (which includes selecting items with text selection highlight included), help/hint, about, and score dialogs. Some of these work in very interesting ways, for example quest log and inventory assemble text by concatenating text from individual segments – for example, “16 ounces of food” line in inventory is assembled one by one from strings “16”, ” “, “ounce”, “s”, ” of food”, and carriage return. The game also appears to use the internal line start offsets to intercept clicks in inventory, highlighting individual lines using text selection by itself.

The game is already approaching playable state, but there are still a few issues – most notably, recalibration of line start offsets does not update last line offsets correctly when removing, causing removed lines to sometimes leave “trashed” text at end of text edit boxes. Here’s some screenshots of how it looks like now:

Scarab of RA gameplay screen, with inventory selection highlighted
Multi-line text highlighting of recent message in quest log

Additionally, a few other games (Lemmings, Simcity) can now proceed past copy protection screens, as we can finally use the VERY hacky TEKey implementation to enter correct answers to the protection questions (SimCity also required a bunch of new selectors to be added to the FP68K SANE dispatcher):

Lemmings can now proceed to gameplay, which appears to work perfectly, including sound & music output using Sound Driver
SimCity (black & white version) can also proceed to game, although sound does not yet work as it still has the weirdly mixed up sound completion routine method signature which crashes the device manager if sound is not disabled

EDIT: Here’s a video of first 15 minutes of Mac Lemmings gameplay:

Lemmings running on M.A.C.E.

There is also some effect on productivity apps:

Text object can now be created in MacDraw
Hypercard at userlevel 5

As the “message” window in HyperCard 1.x is now able to accept commands, we can use “set userlevel to 5” command to actually modify the included stacks! Still a lot of things broken there, but yet a lot new functionality unlocked… We haven’t though had time to play with them, but it should soon be much more usable!

EDIT: After minor tweaking with SDL TextInput API, MacPascal can also now accept text input, compile and run Pascal code:

Some test code entered, compiled and run in MacPascal

In general, TextEdit still has a lot of important features unfinished or to be implemented:

  • No caret display yet (teCaret in TEDoText)
  • TEClick not yet implemented, thus using mouse for selecting text does not work yet
  • TEKey is VERY hacky, needs for example arrow key control, backspace support, etc
  • Dialog Manager keyboard input does not yet handle default items/clipboard commands etc
  • The keyboard driver is still passing key events through scancodes without real mapping to actual MacRoman characters, preventing modifiers/host operating system keyboard layout from having effect
  • Line start recalibration has still sometimes issue when removing lines from middle of text
  • Selections have still some visual glitches, especially when typing text in dialogs
  • Styled TextEdit support still completely missing…for now
  • Probably other things too which I can’t remember…

And now back to the summer break…

Some “Music” to entertain in the quiet period

The past few weeks have been a bit quiet for us, as we’ve been both busy doing important work and personal life stuff, so we haven’t had as much time recently to spend on this project. We will continue progress soon when each of us gets more spare time, but meanwhile here’s a bit of entertainment for meanwhile:

MS-BASIC 2.0 running “Music” example program in M.A.C.E.

As mentioned a few weeks ago, the audio output – which previously was not working in MS-BASIC – suddenly started working as Pukka fixed some 68k CPU bugs. At that time, I recorded a short video of the “Music” sample program running with audio included – and as there’s no other news to share at this time, we hope this “filler” video will keep you entertained for the time being!

Experimenting with INIT/cdev support

This previous weekend, while helping Pukka to fix a mysterious coordinate calculation bug in Fool’s Errand intro, I decided to experiment with one weird idea: How about adding support for System Extensions and Control Panels? Therefore, yesterday I added the After Dark ‘cdev’ control panel to MacPaint app’s build settings to be included in the System Folder, and took some time to look what it would take to make it work.

Basically, INITs and cdevs are loaded during the startup by Mac OS by checking a few bits (such as file type, visibility, etc), opening the resource map of each file, and executing the first ‘INIT’ code resource found. At this experimental stage, we don’t yet have full INIT/cdev scanning, but instead have just a hard-coded filename to load INIT code from which works ok enough for this test case.

Our boot-time heap wasn’t quite properly set up yet, as the boot stack was too high, not leaving enough space between MemTop and BufPtr, which on the first attempt of running the INIT caused After Dark’s InitZone call to actually end up at top of the stack, causing all sorts of havoc. Besides that, there were a ton of bugs/improvements:

  • 68k-to-68k trap calls did not work correctly (i.e. calling a trap patched with 68k from 68k code)
  • InitZone parameter block handling was completely broken from the last year’s trap refactor, so it was fixed
  • Memory Manager’s Handle validation had not taken into account the case where handle’s “relative handle” field would point to invalid memory location in some “fake” handles (as one in After Dark), for which validation was added
  • Empty stubs were added for Notification Manager calls to allow After Dark to call them during activation/deactivation of screen saver
  • Locking of 68k mutex was not done early enough, so After Dark’s VBL handler was making the initial call Launch trap to have invalid register values
  • After Dark uses grafProcs to call QuickDraw, so one missing case used by it was added (ovalProc for ovals)

After that bunch of minor tweaks, we have now at least After Dark working nicely!

After Dark icon displayed during loading of MacPaint

Although the INIT code itself works, there is not yet any UI for displaying cdevs (Control Panels), so modules cannot be configured yet, and thus only the default “Starry Night” built-in module is displayed.

“Starry Night” (default module) running in After Dark inside MacPaint application bundle

There’s also a short video to demonstrate After Dark running:

Although not very usable feature at the moment, this will definitely allow interesting possibilities in the future, as theoretically we should be able to run applications requiring certain application-specific extensions to be present in the System Folder. This experimenting with After Dark has anyway allowed us to find a number of important bugs (as previously mentioned), and also works as a very good “compatibility” test to see how well our Toolbox API and 68k implementation can handle one very quirky control panel, as it patches a ton of toolbox traps and does a lot of low-level stuff which all appear to be compatible with our emulation.