Sunday, April 3, 2016

SCART to JP21 Adapter - Solderless DIY

Most of my consoles are setup for RGB output via SCART connection, and SCART cables and converters are much easier to find than ones wired for JP21. However, my XRGB-Mini Framemeister  only takes JP21, so I've searched all over for a low-cost SCART-to-JP21 adapter to no avail. There's one from a store in the UK that costs like $40+international shipping, which is just too expensive, IMO, and a SCART-to-Framemeister-mini-DIN from retro_console_accessories on eBay (the source for the best-quality retro gaming cables anywhere) for ~$30, which is a bit more reasonable but would cause wear-and-tear on the Framemeister's already-finicky mini-DIN port. Neither of these options seemed very attractive to me, so I figured I'd try the DIY route using a cheap Chinese SCART-passthru-breakout-whatever available for <$4 shipped on eBay:
I'm not altogether sure what the intended purpose of these things is but whatever...
They're sealed shut pretty well but the most effective strategy for cracking them open seems to be applying steady pressure on the sides toward the female end (the curved/scalloped part next to the male end is stronger). I used a C-clamp and slowly tightened it until the clamshell halves started buckling and then popped apart. A bench vise would work well, too.

Once you pop it open, you can cut the wires from the breakout jacks, which will clean out some of the rat's nest inside. In my case, all of the wires were black, which... isn't great:
The input/output switch isn't glued in, so you can remove it and save it for another project.
The female side is soldered in and attached pretty tightly, but the male end is actually pretty easy to work with. Each of the spade-type prongs is secured with a little popout leg in the middle. You can squeeze that in with needle-nose pliers and then press the whole prong inward and pull it through from the inside:
Here you can see one of the removed spade-prongs, with the popout locking mechanism.
Once they're all pulled through, you can rearrange them to match the JP21 pinout and push them back through, as per this key (not my pic, but I'm re-hosting it here because the original Tinypic hosting could vanish at any time):
You'll notice the SCART side has one less ground and one more white, unlabeled prong than the JP21 side. I just left that prong out entirely (#12) and everything still seems to work fine.
Once the prongs are all reordered, you can use a small pokey thing to engage the center-locks, no soldering required. I used a cheap dental tool I had laying around to do the job, but those little pointy electrician's tools (the ones that look like dentists' tools with screwdriver handles) should work fine, too. After that, just glue the casing back together and you should be all set.

Wednesday, March 9, 2016

Using Rollback to Hide Latency in Emulators

This is something I thought about a pretty long time ago but recently decided to flowchart out in case anyone wanted to implement it.

The way rollback-based netplay (e.g., GGPO and RetroArch's current netplay implementation) works with emulators is that the two players exchange input states (which are small enough to pass back and forth within the time of a single frame of emulation) and then when they diverge, you roll back to a previous known-good time where the input still agreed and then emulate the intervening frames with the corrected input data to catch up (in RetroArch, these are called 'lag frames').

I had an idea for hiding a frame of latency that would work similar to that concept but in a single-player situation (click to embiggen):

The way it works is whenever the player's input changes, you roll back one frame and apply the new inputs retroactively and then emulate two frames to catch back up. This makes your inputs go into effect one frame before you actually pressed the button(s). This wouldn't result in a rollback loop because, even though we feel like we press a lot of buttons all the time when we play a game, most of the time (particularly from the emulator's point of view) we're really just holding a button or two.

You would want the audio to run one frame behind the video, so your audio buffer wouldn't be constantly emptying and skipping around every time you roll back. Instead, it would fill back up with the next frame's audio during catch-up. Despite running a frame behind the video, our brains would unconsciously sync the audio with the video, as they are very forgiving about that sort of thing (this is a known effect).

One drawback is that you would frequently lose a single frame of animation but thankfully our brains are quite good at papering over that sort of thing, as well. The other drawback to this method is that it would require emulating two full frames in the space of a single frame, so the CPU requirements for any emulator using it would be doubled.

Again, this method would hide a single frame of latency. While most PC setups have significantly more than one frame of latency to contend with, every little bit helps.

UPDATE (4/12/2016): I was talking to letoram, author of ArcanFE, the other day and he mentioned that he had implemented something just like this recently and that the lagging audio made everything seem *more* latent, presumably due to the everything-syncs-to-the-slowest effect mentioned in the above link. I didn't ask him whether muting/disabling audio helped with this or not.

Friday, March 4, 2016

New and Updated Shaders

It's been awhile since I've done any shader posts, so I figured I'd cover some updates that have happened recently. Same format as usual, the only difference this time around is that instead of zooming into screenshots using photoshop, I made them straight from RetroArch using the 'zoom' from my image adjustment shader. This gives sharper, cleaner detail shots. Anyway...

No Shaders (Nearest Neighbor)

Here is a shot with no shaders for comparison.


ScaleFx

Sp00ky Fox had been working on cleaning up artifacts from the Scalenx shaders and, in the process, worked up a similar algorithm of his own, known as ScaleFx. By cranking it up to 9x, he managed to do some pretty impressive smoothing:

Some notable things here: check out the circle-c copyright symbol, which is very hard to deal with in this sort of shader, as well as the straight lines throughout the logo and on the shallow slope below the dragon coin.

He also made a variant, known as scalefx-hybrid-9x, that uses reverse-antialiasing and creates some interesting depth and shading effects:

It does introduce some haloing, though, and is pretty heavy, performance-wise.

xBR-lv2-accuracy-multipass

For the past year or so, Hyllian has been working to improve his now-famous xBR upscaling algorithm, mostly by pairing it with other algorithms and working to improve handling of problematic edge-cases. Very recently, though, he changed the way corners are detected and added a new color diff algorithm, which fixes some weird artifacts that could occur when bright red and bright blue pixels were next to each other (artifacts not pictured):

Here is an older version of the same shader for comparison:

Again, the copyright symbol is a good example of the improvements, along with the circles inside the number 9s. The updated version also works quite well with Playstation-era antialiased images, while ScaleFx works best on bold, cartoony graphics, like Super Mario World and Shantae.

CRT-Lottes Updated

Since we first ported Timothy Lottes' scaling pixel art shadertoy, he did a couple of iterations to add bloom and a few more variations on his awesome shadow mask code. r5 incorporated those updates into our port and threw in some runtime parameters (including the aforementioned mask variations):

This is the default compressed TV-style shadow mask^^.

This is the Trinitron-style aperture grille^^.

This is the stretched VGA-style shadow mask that was used in the original shader^^.

And this is another VGA-style mask with larger phosphors^^.

CRT-Royale-Kurozumi Preset

Shmups user Kurozumi posted some really nice settings for TroggleMonkey's CRT-Royale shader that makes it look very much like a high-res broadcast monitor (e.g., Sony's BVM line or the 800-line PVMs):

I put these settings into a cgp preset, located in the 'cgp' subdirectory of the main common-shaders repo.

Monday, January 25, 2016

Bash Scripts for RetroArch

I wrote a couple of bash scripts that could be useful for people using RetroArch and figured I'd post them here.

First off is a script for 'dumb' scanning and playlist generation. That is, it doesn't check against any databases, so it catches things like ROMhacks and translations that don't appear in the No-Intro database that we use for the built-in content scanning:
#!/bin/bash
COUNTER=1
echo "Enter the absolute path to your core"
read core
echo "Enter the absolute path to the directory you want to scan"
echo "(exclude trailing slashes)"
read content
echo "Enter the name of the playlist"
read name
for i in $content/*; do
echo "$i" >> $name
echo "$i" | sed 's=.*/==' >> $name
echo "$core" >> $name
echo "$core" | sed 's=.*/==' >> $name
echo "$COUNTER|crc" >> $name
echo "" >> $name
done
When you run the script from a terminal/CLI, it will ask you to enter the path to the core library used to launch the content you're scanning, the path to the directory to be scanned and a name for the playlist, then it'll scan through the specified directory for any files and add them to a new playlist.

Askot, a user from the RetroArch forums, made a variation in which you pass the variables directly at launch time instead of answering questions:
#!/bin/bash
COUNTER=1
for i in $2/*; do
echo "$i" >> $3
echo $i | sed 's/\.[^.]*$//' | sed 's/.*\///' >> $3
echo "$1" >> $3
echo "$1" | sed 's/\.[^.]*$//' | sed 's/.*\///' >> $3
echo "$COUNTER|crc" >> $3
echo "" >> $3
COUNTER=$[$COUNTER +1]
done
It also removes the file extension from the playlist entries, which some people may prefer (e.g., Tetris instead of Tetris.nes). You would use it like this:
./dumbscan /path/to/core.so /path/to/content /path/to/playlist.lpl

The next script generates basic cue sheets for disc image files that don't have them already, like iso and img. This will make (most of) them usable with the Mednafen/Beetle-PSX core.
#!/bin/bash
longname=`echo "$@" | sed 's=.*/=='`
name=`echo "$@" | sed 's/\.[^.]*$//' | sed 's/.*\///'`
if [ -e "$name".cue ]
then
echo "Cue sheet \"$name.cue\" already exists. Aborting."
else
echo "FILE \"$longname\" BINARY" >> "$name".cue
echo "TRACK 01 MODE1/2352"       >> "$name".cue
echo "INDEX 01 00:00:00"                >> "$name".cue
fi
You would use it like this:
./cuemaker whatever.iso

Thursday, October 22, 2015

Bash Scripts for Non-Repeating Random Playback

I have an HTPC with a huge amount of files to choose from, which can lead to analysis paralysis / the paradox of choice: when there's so much to choose from, I end up consuming the same handful of media over and over instead of consuming a proper variety. Many media players have "shuffle" functions that will pick a random item from a list but that only works for traditional media and doesn't help with, for example, emulated video games. So, I decided to play around with some bash scripting to see if I could come up with anything more universal.

The first thing I wanted to do was to create a playlist based on a recursive scan of a directory. For example, I have my TV shows separated by show and then subdivided by season/series, and I want all of these files included in a playlist, which I accomplished with this script:
#!/bin/bash
if [ -e playlist]
then
rm playlist
fi
for f in **/*
do
echo $f >> playlist
done
This script will check for any existing playlist and delete it if it finds one already, then it search recursively and adds any files (but not directories, importantly) from any subdirectories to the playlist. If I add more files (e.g., I get new episodes of a show), I can just run the playlist-maker script again and it will delete the old playlist and make a new one with the new files added.

So that's great. However, these files are all in order and I want them to be randomly sorted instead (in case my desired launch program doesn't have a built-in shuffle function), which I can do like this:
#!/bin/bash
if [ -e random-playlist ]
then
rm random-playlist
fi
for f in **/*
do
echo $f >> nonrandom-playlist
shuf --output=random-playlist nonrandom-playlist
done
rm nonrandom-playlist
This script does all of the same steps as before but it also uses the shuf command, which is a common UNIX utility that will randomize the entries in the non-randomized playlist and then output a randomized playlist. The script then deletes the non-randomized playlist.

Alright. Looking good. If my playback program has a built-in shuffle function, I can just pass this playlist to it and it will shuffle among those files. However, as anyone who uses shuffle frequently will tell you, it often feels like a small minority of songs/videos gets played more often than others (whether this is actually true or not is up for some debate but it can be annoying all the same), so I want to remove entries from the playlist once I've consumed them so there can't be any repeats until every file has been played at least once:
#!/bin/bash
line=$(head -1 random-playlist)
echo "Next up: $line"
mpv --fs "$line"
echo "I have a lot of great memories from that episode..."
sed -i -e "1d" random-playlist
echo "And now they're gone!"
This script uses the UNIX utility head, which lists any number of lines from a file, starting with the first line. In this case, I just ask for the first line, echo it (for diagnostic purposes, and it will be useful later) and then pass it to my player (in this case, mpv). After playback is finished, it echoes again and then uses the UNIX utility sed, which is a super-powerful text/stream editor, to delete the first line ("1d") from the random playlist. It echoes one more time to complete the Bender quote and let me know that the entry was successfully deleted.

Ok, this is great but I don't want to have to always run my script to consume the next file. That is, I want to just get things rolling and have a marathon of random selections without continued interaction from me. To do this, we just wrap the whole thing in a 'for' loop, like this:
#!/bin/bash
for i in {1..100}; do
line=$(head -1 random-playlist)
echo "Next up: $line"
mpv --fs "$line"
echo "I have a lot of great memories from that episode..."
sed -i -e "1d" random-playlist
echo "And now they're gone!"
done
This will go through playing and deleting 100 times (you could make this 1,000 or 62 or whatever), which is great. However, I don't usually want to watch 100 episodes of a show, and if I launched this script from a desktop environment (i.e., by double-clicking), I have no way of stopping it! That is, each time I close my playback program, either because the episode is over or I clicked on the 'close' button, the next episode is queued up and starts automatically. If this were running in a terminal window, I could just close that window and it would stop the script in its tracks but this leads to a dilemma: it's not convenient to have to launch the script from a command line each time instead of double-clicking, and if I tell Nautilus (my file manager) to launch scripts in a terminal when I double-click them, it tries to run the scripts from my home directory instead of their actual location >.<

Thankfully, there's a solution. I can write a helper script that doesn't care if it runs from the home directory, and that script can launch my marathon script:
#!/bin/bash
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
x-terminal-emulator -e ./marathon-script
The first line of the script is a handy one-liner that navigates to the directory where the script lives. The second line tells it to launch whichever terminal application is registered to your alternatives system as the 'terminal emulator,' in my case it's gnome-terminal, and the -e switch tells it to run the marathon script inside that terminal, where we can easily view our diagnostic echo messages and close it when we're done consuming media.

As I mentioned, these scripts are easily modified to suit any kind of file, so I have some set up to launch a random NES game from my collection via RetroArch. I plan to use this setup to play through the entire NES library over a period of a few weeks/months.

Monday, September 7, 2015

SNES to NES Controller Adapter

Historically, the SNES was originally intended to be backward compatible with the NES, with a CPU from the same 6502 lineage as the NES and a similar controller signal protocol over the same 5 wires: data clock, data latch, serial data, +5v and ground. The SNES signal has more data clock pulses after the data latch pulse but the protocols are the same, otherwise. This means that if you make a pin-to-pin adapter that sends SNES controller signals to an NES port, everything works just fine and the NES will just ignore the extraneous clock pulses.

Now, original NES controllers are getting fairly expensive these days--particularly the "dog bone" controllers that are round instead of pointy--and the third party reproductions from Tomee et al. are pretty universally reviled for being flimsy, unresponsive and altogether crummy. SNES controllers, on the other hand, are more ergonomic, originals are generally more plentiful and some of the repros (like this one) are actually pretty great. I also already have a bunch of SNES controllers, including an ASCII Pad, which is possibly the finest third party controller ever produced...

So, with all of that in mind, I cobbled together an SNES to NES controller adapter (there are a number of guides online, see also: here and here, among others) using a couple of cheap extension cables, specifically Tomee NES extensions (I found a set of 2 from an eBay seller for $9) and "Gen" 2-pack SNES extensions. I'll post which wires I connected, but it seems the factories that produce these extension cables just pick a handful of randomly colored wires when they put them together, so don't assume they'll apply to your own cables in any logical way:
NES side   |   SNES side 
  Yellow  <->  Black 
  Orange  <->  Green 
   Black  <->  Red 
    Blue  <->  Yellow 
   Green  <->  White
Ideally, you would figure out which wire carries which signal using a multimeter but mine is on the fritz, so I just trial-and-errored my way through it, which was a pain in the ass, but whatever. It worked in the end. Interestingly and unexpectedly, NES' B-button becomes the SNES' Y-button, and NES A becomes SNES B, which is really fantastic, since the angle of the SNES B and A buttons would make them hard to press at the same time had the naming/mapping been consistent.

Thursday, August 27, 2015

Analogue Nt HDMI - First Look

I finally got my Analogue Nt with HDMI upgrade in the mail today and while the base Nt has already gotten plenty of coverage, the HDMI-upgraded model hasn't been covered at all (AFAIK) at the time of this writing.

I got the default silver model with the "limited edition" white and gold controller ports:
As covered elsewhere, the construction is incredibly high quality (the power/reset button being a button from an NES controller is a particularly interesting touch, I thought):
There has been much ado about the aluminum cart slots scratching up cart shells, which hasn't been a problem for me, but I suppose if you're really worried about it, you could open the flaps with a finger while inserting the cart. /shrug

Below the HDMI port is a standard 15-pin d-sub connector, which I've seen called a "VGA port" elsewhere, though this is not the case. That is to say: you can't hook it up to a VGA-compatible display and get any video out of it. For that matter, it's apparently not RGBHV-over-d-sub, either, as it didn't produce any picture when hooked up to my multisync XM29+'s d-sub port, even though that monitor will accept "240p" non-interlaced video over that connection. I'll have to catch up with the guys at Analogue and ask about the pinout of the port. Hopefully, it's something that won't require too much effort to get going on my own. I'm planning to hack together a SCART cable and a working RGBHV-over-d-sub cable (if possible). EDIT: I just talked to the Analogue guys and it seems the HDMI and RGB/component functionalities are mutually exclusive. It seems they will swap between the two at no charge, though:
"When you upgrade your Analogue Nt to HDMI it outputs HDMI only, there are two different systems that are not compatible with each other. Though, whats cool is you can switch between RGB and HDMI if you wanted either version (we'd install either daughterboard for you for free). Also, any other cool mods that the NES community may make in the future would be compatible with the Nt too - which is super cool!"
That in mind, I may decide to switch to RGB at some point in the future, as much of my equipment is tooled for RGB/SCART/JP21, but for now HDMI is convenient.

I won't rehash any more of the information that's readily available elsewhere, so instead I'll spend most of this post talking about the HDMI upgrade. While originally advertised as an external upscaler, they eventually nixed that idea (and rightfully so, since it's not likely they'd be able to get anything worthwhile for the ~$60 cost of the upgrade) and opted instead for kevtris' brand new HDMI board, which AFAIK isn't available anywhere else just yet.

I'm still doing testing to get a feel for what the HDMI board is capable of, but out of the gate: it's pretty incredible. The image is exactly what you would expect from an emulator, warts and all. That is, the upscaled pixels are as sharp as you can get; much sharper than the output of an XRGB-Mini Framemeister (which I also have, so I can give them the Pepsi Challenge):
The colors in those shots are washed out, but that's the fault of my camera trying to adjust to the brightness of my TV's backlight. The colors are actually bright and vibrant, and the HDMI connection has none of the "shimmering" on large swaths of color that have been reported elsewhere with RGB connections.

Getting into the HDMI onscreen menu is pretty cool/crazy in that it's a button combination on the gamepad (specifically, dpad-down+select by default). Once there, it's got a ton of cool options for tweaking the video, including overscan cropping, horizontal scaling, scanline settings, scaling filters and resolution selection driven by your display's EDID:
At 1080p, the scanlines are uneven thickness (alternating thinner and thicker), just like you would get on an emulator at 4.5x scale:
1080p's uneven scanlines
I would have liked to see an option to force integer scaling to avoid this issue, but I didn't see anything like that (I'll update this post if I find otherwise; such a thing may also be added in a future firmware update UPDATE: 9/26/2015 looks like kevtris is on it). At 720p, the scanlines are even thickness, since it's an even 3x integer scale already, but the pixels aren't as sharp due to my TV doing the rest of the scaling:
720p's even scanlines
Scanline settings. NES is locked to the resolution, while the others are pre-set scale factors (i.e., they don't always line up with the pixels, which I can't stand). There's also an intensity slider that controls how dark the lines are.
Likewise, at 1080p, the vertical size of pixels isn't consistent, just as you would expect from nearest neighbor scaling at a non-integer scale factor. This is also the case with the horizontal scaling, but there is an option screen specifically for tweaking it, from 3x (for 1:1 PAR at 720p) up to stretched 16:9 (barf). The default setting is 4:3 on 16:9 displays, which has faintly warped pixels but is the natural aspect ratio for NTSC:
1:1 PAR (i.e., square pixels) using 3x horizontal scale at 720p; notice the even checkerboard pattern in the road.
4:3 aspect ratio; notice the uneven pattern on the road.
There's also a preset for 4:3 on 16:10 displays, which is handy if you're hooking up to a monitor instead of a TV, as well as an option labeled "interpolation," but I can't really see any difference between it being enabled/disabled.

There are also scaling filters, including the ever-popular hq2x/hq3x/hq4x (no filtering vs hq4x):
No scaling filter
HQ4x scaling filter
The HDMI board also includes a lot of audio settings, with individual enable/disable toggles and volume/panning for main audio and each specific add-on audio chip for FDS audio and any in-cart audio add-ons, like the ones in Gimmick and Akumajou Densetsu:
Independent volume control for each audio device
There are a ton of other options I haven't gotten to play with yet, including hotkeys for over/under-clocking (presumably to get around some incompatibility with the Everdrive N8, which I haven't gotten to test yet), soft-reset and more.

UPDATE (8/30/2015): Looking into the Everdrive issue, it seems the N8 is indeed incompatible with kevtris' HDMI board with N8 firmware >= 4.0, while the 3.0 firmware may be okay. The Analogue folks uploaded a copy of the older firmware to their Dropbox account. Testing is easy, just copy and paste the files onto your SD card and replace the newer versions. Leave me a comment if you get to try it. I hope to borrow a friend's N8 soon, so hopefully I can provide some input/testing.

Update (9/25/2015): KRIKzz got his hands on an HDMI board and has been investigating the issue. Hopefully, a fix will be forthcoming. Update (9/28/2015): Oh sweet: looks like he's got it working! Anyone with the HDMI board should upgrade their N8s to firmware v13. Update (10/3/2015): d'oh, not working for me :(. I get a garbled mess if I boot my new N8 with no SD card and a green screen with firmware v13.

In addition to the N8 issues, Kevtris' HDMI board has been having issues with some games and mappers, the most popular of which is Castlevania 3. Apparently, holding reset while powering on the console alleviates some of the issues but this isn't an option on the Nt. Kevtris is working on it, though, and it should be addressed in the upcoming update. Update (10/30/2015): Kevtris' update is out and it apparently fixes the Everdrive green screen, FDS and MMC5. Anyone still having the Everdrive green screen issue can still apply the update from their Everdrive by powering on with the HDMI cable unplugged, wait a few seconds for everything to initialize and then plug the HDMI cable back in.

I plan to update this post as I learn more but I wanted to get this info out there, both for people waiting on their Nts and for those eager to learn more about kevtris' awesome HDMI board (for reference, it seems my board's serial number is 30, so I would assume it's one of the first to make it into an end-user's hands).

Anyway, here are a couple more glamour-shots:

Analytics Tracking Footer