I've taken the liberty of proposing this in the same way as a newgroup would be created. I'd like to have this proposal dealt with by the same process; the reasons will become clear as you read this.

Also, please try to keep all discussions of this public; not that I won't reply to e-mail, but I want everyone concerned to be able to keep track of *every* facet of the discussion, so that people won't think that something was "slipped in while they wer en't looking".

Thanx! :-)

Bob Blechinger

UPL - The END of SPAM!!


# The Problem #

There's no question that SPAM is one of the most pressing problems facing Usenet right now. The massive amounts of bandwidth consumed by superfluous posting of this worthless crap has totally bogged Usenet down, and threatens to destroy it completely. A lready, many Net veterans have abandoned their favorite newsgroups, and predictions of the "Death of Usenet" abound.

To put it bluntly, the situation sucks!

Several anti-spam measures have been proposed, including killfiles, cancelbots, notifying ISPs, mailbombing the spammers, the "Usenet Death Penalty", the BI, etc.; to date, unfortunately, none of these have proven to be entirely effective. The underlyin g reason, unfortunately, is that all of these solutions are *reactive*; in other words, they can only function *after* spam has *already* propagated throughout Usenet.

Which is, of course, too late; kinda like closing the barn door *after* the cows have all gotten out.

The necessity, then, is to find a *proactive* solution, one that *prevents* spam from being propagated to begin with; that solution is what you're reading now!

# UPL - What is it? #

UPL stands for "Usenet Posting License"; before you go into some kind of conniptions, let me explain what I mean by the term...

UPL is *not* a "license" in the sense of being issued by a regulatory agency, i.e., your state DMV. Rather, it's a user verification protocol to insure that every Usenet poster has at least a nominal understanding of proper procedures, Usenet traditions , etc., *before* they start posting.

In the past, before the Internet's population explosion, net.veterans used to dread September, since that was the time of year that the new college freshmen would cascade onto Usenet; for about a month or so, newsgroups would be full of newbies who were desperately in need of a clue. The majority of them eventually got their bearings, and became Netizens in good standing, but there were always a few...

Within the past few years, though, the massive influx of new users has been too much for the core of net.veterans to handle. Worse yet, the willingness to learn that was shown by the college newbies hasn't been mirrored by the new masses. AOL is a good (but by no means exclusive) example; when they came onto the Internet in 1994, quite a few AOLers had the attitude of, "I *paid* for this, I can do *whatever I want*!".

Naturally, this wound up endearing AOL to everyone else on the Net forever and ever...

The net result (no pun intended) is that the time-honored traditions of Usenet have pretty much fallen by the wayside.

Much worse, though, are the spammers. The most problematic of the lot are *not* regular Usenet readers; rather, they're the Bastard Hellspawn of Canter & Siegal! For some reason, these sub-humans have gotten the idea into their malformed brains that if they post hundreds of off-topic ads to thousands of newsgroups every day, regular Usenetters, instead of being prodigiously pissed, will be entranced and enticed to do business with them.

Talk about wishful thinking!

Which brings us back to the Usenet Posting License, albeit in a rather circumlocutionary manner...

# UPL - How does it work? #

As I said before, the Usenet Posting License is a user verification protocol; simply put, the concept is to insure that new posters have at least some familiarity with Usenet rules, conventions, and traditions *before* they start posting.

The UPL protocol operates on the user's news server, so that users are verified *before* the articles are propagated throughout Usenet; this is the most logical way to do it, since it stops unverified articles from going beyond the news server to begin with, rather than waiting until they've been sent to every server on the Net.

When I first came up with the idea, I had considered the possibilities of setting a flag in either the .newsrc or in the article header; that didn't last too long, since it offered too many possibilities for abuse of the process. Thus, I eventually came up with what you're now reading.

My objectives in creating the UPL protocol were to make it both easy to code and implement; thus, modifying existing daemons, etc., would be more practical than starting from scratch for some of the UPL protocol.

In other words, an "elegant kludge".

There are a number of parts to the UPL protocol, as follows:

1) UPL News Server Patch - this adds code for verifying users into the news server source; it will require recompiling, and can eventually be integrated into the news server code base.

The patched news server will check the identity of posters against the upldata file (#2; see below) before accepting an article to be posted and propagated; a verified user's article will be passed, and the ID # will be added to the upltemp file.

The server will also verify articles coming in from upstream servers; it invokes upld (#3; see below), which sends a verification request (VRQ) packet to the sending machine. If the sending machine returns a positive ack, the article is accepted by the server; a negative ack will reject the article. If the sending machine doesn't respond within a reasonable time, upld will send a VRQ to the *originating* machine to check verification.

2) UPL Data Files - the UPL protocol creates 3 data files. The first is upldata, which is directly accessed by both the news server and the UPL Daemon (see below); the second is uplmaster, an encrypted file that is regularly compared to upldata via a cr on job. A third, upltemp, keeps track of ID #s of verified articles.

The reason for having 2 data files is simply to prevent unauthorized changes from affecting the integrity of the UPL for any length of time; for example, if an unverified user hacks the upldata file in order to post forged articles, the next time the cr on job runs, the discrepancy will be noticed and rectified.

The upldata file simply contains basic information on users and their verification status; the uplmaster file can also contain anything else that the newsadmin feels is relevant information.

The upltemp file is a constantly updated list of the articles that have been approved by the server; having both userlists *and* article verification provides another safeguard against forgery.

3) UPL Daemon - upld is basically a verifier which contains a modified pingd; the main difference between upl and ping is that upld only transmits a *single* ping-style packet, which contains the ID # of the article, and a flag set to accept or reject the article, based on whether the user is verified or not.

To do this, upld accesses the upldata and upltemp files, verifies the user/article, and sends an ack to the VRQ back to the sending machine.

4) UPL Admin Utility - upladmin is a simple utility that reads and edits the uplmaster and upldata files; fairly self-explanatory.

5) UPL Test - this is a CGI script, Java applet, or shell application that tests users on their knowledge of Usenet procedures, conventions, traditions, etc.; it operates on a pass/fail basis, with passing scores allowing a user to be added to the uplda ta and uplmaster files, either automatically or by the newsadmin.

6) UPL Study Guide - this is simply a text file that contains all the information about Usenet that's covered on the UPL Test.

As an example, John Doe is a verified user on machine A; Doe posts an article to A's news server, which checks its upldata file. Finding Doe in the userlist, it accepts the article, and sends it to machine B. B's news server receives the article, then s ends a VRQ back to A; since Doe is a legit user, A sends back a positive ack, and the article is posted to server B, and sent to server C. C does the same, accepts the article, and send it to D. This keeps going until it reaches server K; for some reason, server J didn't send an ack to K's VRQ, so after a while, K sends a VRQ back to server A, which verifies and returns the ack.

In the meantime, Joe Spammer hasn't bothered to get verified; he tries to post to server A, but the server doesn't find him in the upldata, so his article is rejected. Since the article wasn't accepted, it doesn't get sent to machine B at all; thus, it doesn't get propagated, and doesn't waste bandwidth.

Since the article doesn't get sent in the first place, none of the reactive measures (which in and of themselves take up bandwidth) are necessary; cool, huh? :)

Along with getting rid of most spam, this should also take care of stuff like bogus addresses, etc., and we get it for *free*!

# Adopting UPL #

As I stated in the note at the top of this article, I'd like to see this dealt with in the same way that a newgroup is handled; while it's not a newgroup per se, it *does* represent a technical standard that will be a benefit to Usenet as a whole, and a s such, I'd like to see it adopted through a democratic process.

Besides which, when the spammers (who don't actually *read* most of the newsgroups that they post their spew to) start complaining about their posts not getting out, we can simply tell them, "Hey, it was done the way we do everything on Usenet; you had your chance to comment and vote on it, so if you didn't, don't come bitching to us about it!".

Sneaky little bastard, ain't I? ;D

This can also be useful in dealing with those who want to regulate the Internet (i.e., Exon); adoption of technical standards such as this can potentially be used as an alternative to unenforceable laws that don't take the dynamics of Usenet and the Int ernet into account.

Please feel free to interject any input that you can think of wrt the UPL protocol; I didn't go deeply into technical detail, simply because I want to make sure that nothing was forgotten, and other points of view help immensely toward that end.

Keep in mind, too, that I'm not presenting this as a 100% bulletproof, hack-proof solution. I'm figuring that this should eliminate about 95% of the spam; I get the feeling that to get that last 5% would require much more time and effort, and the object ive here is to get something that can be quickly and easily implemented ASAP.

Let's get rid of SPAM once and for all!!!

Bob Blechinger
26 April 1997

Back to Electric Skippyland