What bugs me here is not so much that stuff gets broken now and then, but this nonchalant attitude to breaking things, and lack of acknowledgement for others that have to wipe up the mess.

> And... people just have to fix it once.

> Given that is not a mature language or a stable framework yet, I would disagree that breaking changes, per se, should be avoided as a matter of course.

Also, I have a strong aversion to CAPTCHAs, partly because they're non-free Google SaSS, but also just because they are just plain annoying. Why would anyone want to inflict that upon themselves? There's already plenty of ways to deal with spammers in News.

I think it's possible to experiment with things while still considering stability and backwards compatibility. I recently reworked the password hashing function, but I took care to not break the logins of every user, when doing that.

I must say I'm a bit upset by you being so apparently casual and careless about creating several breakages[0][1][2] in News that I've had to fix, and then suggesting that I should just get off the master branch if I don't like your "experimentation".

I will. I'm done with this.

Experimentation is not the same as carelessness. Seriously, WTF is this?:

    (abs-link site-url* "anarki" "http://github.com/arclanguage/anarki"))

What you want are local macros (Common Lisp's `macrolet`, Racket's `let-syntax`). Suppose we name Arc's equivalent `w/mac`.

Then (w/html ...) could expand to this:

  (w/mac html tag-html
         head tag-head
         body tag-body
         ...
    ...)

This would be something of a breaking change to Anarki because it means Anarki will now allow a local variable to shadow a global macro. That was the problem you encountered with (body) in the first place; some calls to local variables were expanding as macro calls now because the macro lookup ignored local variables altogether.

There might be some code out there of the form

  (let foo 1
    (bar foo))

For some time I've thought it would be worth it for the local macro support, but I'm too afraid to make the breaking changes.

Nevertheless, a while back, there was a consensus here that the way things worked already was a bug. See "Global macro names take precedence over local lexical variables" at [1]. I don't remember if this bug was ever fixed in Anarki; it doesn't seem to be fixed now. (Maybe there was another consensus to reverse it? It could happen.) It's the very same bug you encountered with (body) in the first place.

Maybe there's a difference somewhere in what we mean by "API." There's a distinction to make, but I wouldn't make that one.

I'm saying APIs arise naturally at boundaries people already feel aren't worth the effort to cross for various plausible reasons. Recognizing an API boundary is a positive thing: Then we can talk about it and help people understand the ways in which it's worth it for them to cross the boundary. But talking about it also makes it something people form as a concept in their mind and form expectations around, so it can become a more imposing boundary that way if we're not careful. We should seek a sweet spot.

The important distinction is that sometimes the reasons people don't cross boundaries are good ones, and sometimes they're baloney.

I think what you and I want to avoid for Anarki is documentation that gives people baloney reasons to think it's not worth it for them to edit the Anarki code.

While we can try to minimize the good reasons that exist, and we can encourage people to try it before they knock it, there's no point in denying that the good reasons sometimes exist.

If you ask me, the reason we have to settle for a "sweet spot" is totally a consequence of the fact that Anarki is made of text. Text loves bureaucracies. Someone sees text, they ask for the Cliff's Notes, and then someone has to maintain the Cliff's Notes.... So you've gotta get rid of the text, or you'll have an API boundary.

Programming without bureaucracy would be like having a conversation without writing it down. I believe in this. I think conversational programming is exactly what we can do to simplify the situation of programming in the world, and Era's my name for that project, but Anarki is a more traditional language, and it takes a more traditional trajectory into APIs.

Our documentation can try to convey "This is all subject to change! Edit it yourself, open an issue, or drop by on Arc Forum for some help! We count on contributions like yours to make Anarki a good language. Since others are welcome to do the same thing, watch out for changes in this space." We could put an "unstable, just like everything else here" tag on the corner of every single documentation entry.

Anarki is far from the only unstable codebase people have ever seen. Being a Node.js user, my code has regularly been broken by upgrades, and Node.js isn't even a version 0.0.x project. I think only a certain number of people would mind if Anarki promised stability and then couldn't deliver on it, let alone mind if Anarki made it clear in the API documentation that there was nothing stable to find there in the first place.

---

"guaranteeing that anybody calling them will be able to upgrade without modifying their code"

You know me well. I probably said that pretty insistently in the past. It's something obsess over and a major requirement I have for code I write (and hence all its dependencies), even though things like Node.js show me it's not a current norm in software.

It's the 100-year language ideal. Spend a lot of time to get the axioms just right, and then the language is stable and nobody ever feels the need to change it since there's nothing to improve on! In 100 years, Arc is supposed to be the language that's too perfect to be anything but the stablest language around.

I still believe in that ideal, but not in quite the same way. People will come up with a reason to promote their alternative language, even if it's just to say they're different than their parents. But I do believe languages can be designed to be more compatible with their forks, even to the point where it's difficult to say where one language ends and another begins. The resulting unified "ecosystem of all languages" could have a much simpler design than the "ecosystem of all languages" we currently have.

That's also part of my approach in Era. I make decisions in those codebases with the goal of simplifying the ecosystem of all languages.

I'm used to more verbose languages and the redundancy of heavier syntax. Sometimes Arc reads to me like a diagram sketched on a napkin.

>It could be that whtml.arc's time for porting has arrived. :-p

That documentation is for Anarki Stable, the branch meant for users who aren't thrilled with Anarki's changes and just want to use official Arc, but who would rather not have to fix official Arc's known bugs on their own. On that branch, html.arc probably is the only HTML-generation library we can or should document.

Despite not being written as documentation for the Anarki master branch, it's nevertheless some of the most accessible documentation for that purpose. I see that's pretty frustrating. It's good that you're mentioning it or no one would work on it.

Yesterday, since this was on my mind thanks to these threads, I wrote some code for generating a simple HTML page based on Anarki's (help foo) documentation. Once I get this to automatically deploy with each new Anarki commit, it should give us a much more up-to-date documentation resource to link to.

---

"any newcomer is going to see it there and think that's what they should use when they want to generate html"

It pretty much is! Even if it's not as good as you can imagine it being, it's still the go-to HTML generation library for Anarki.

When I implied it wasn't "a self-contained library for public use," what I meant was that it was primarily made to get the webapps to work. It's quite a non-minimalistic approach to HTML generation if you ask me -- I'd rather treat every attribute uniformly rather than having a big table of special cases -- but in other places it does only what it needs to do for the purposes of those applications.

Paul Graham pursued the web server code in order to put pressure on the "core language" in arc.arc, to make sure he was focusing on operators that were actually good for something. He released the language to the public so he could put a wider variety of pressure on arc.arc. The operators in arc.arc were never finished, so html.arc was probably a long way from being finished itself.

---

"If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?"

If someone makes a new library for HTML generation, we can potentially embrace that as the new html.arc. Anarki is unstable enough that this would be pretty normal (although I think people have had a polite tendency not to delete something in case someone else was still attached to it).

---

"[paraphrased] What is /lib for?"

Personally, I'm reluctant to call it a "standard library" without a standards document, but it does "come standard" with Anarki, so...

Of the 14 files in the release of arc0.tar, the libs.arc file contained this:

  (map load '("strings.arc"
              "pprint.arc"
              "code.arc"
              "html.arc"
              "srv.arc"
              "app.arc"
              "prompt.arc"))

Looks like there was always a lib/ directory in Anarki, even as far back as the first Git commit. At this point it's the obvious place to put files that some people might be interested to (load ...) and some people wouldn't care about. If someone has a single .arc file to share, dumping it there is easy regardless of what state it's in, and it usually fits right in.

---

"I really like it... but wow am I not capable of even reading that code ._."

Aw, I bet you could! :) Do you want to? Maybe if you've got a specific piece of code that perplexes you, we can help out.

For this:

  (cons 1 (cons (car x) (cdr y)))

  (1 (/ car x) / cdr y)

  `(1 ,(car x) . ,(cdr y))

  (quasiquote (1 (unquote (car x)) . (unquote (cdr y))))

If you write a slightly different macro where the syntax for (unquote ___) is (/ . ___), then you could use it like this, which matches the syntax of your example exactly:

  (t-expr (1 (/ car x) / cdr y))

It sounds like you're looking at this as a data encoding? A lot of data encodings take pains not to be Turing-complete. Then again, for inexpressive languages like HTML, there are a lot of Turing-complete templating languages designed to dynamically generate them.

HTML has other languages embedded in it like JavaScript, and it sounds like that's relevant to what you're going for here. You're going for a language which can begin running even though not all the syntax is computed yet. That's pretty interesting. (Actually, maybe I'm projecting, 'cause I've been dreaming of something like that too: A language which multiple people collaborate on, which can begin running before all of the code has been written.)

You can of course experiment with that in most Lisp dialects using a quasiquoted term where all the unquotes have functions in them:

  (my-interpreter `(1 ,(lambda () (car x)) . ,(lambda (cdr y))))

Certainly a lot of things could be tested more; this would help.

But also, we might want to set Github to require the tests to pass on a PR before letting it be merged. I could be wrong, but I think a PR can be merged even if the tests fail.

The thing is, it is, whether it's meant to be or not. Arc's documentation describes it under the heading "html generation" (in general.) It's in what appears to be Arc's "standard library" /lib, so any newcomer is going to see it there and think that's what they should use when they want to generate html. And it has the rather canonical-seeming name of "html.arc."

If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?

I think discussions about APIs and interfaces are missing the confusion (which, admittedly, may just be mine) over what exactly /lib is supposed to be for?

>I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

The main idea is pretty much just 'S-expressions have dots; what if that dotted part could be not just an atom but a full structured expression like a Lisp form'. (or a Prolog term, if you are familiar with Prolog).

This very small idea opens up a number of new possibilities in creating very compressed notation, I think. It's mostly useful for people interested in building Domain Specific Languages, where you want a minimum amount of friction from the underlying syntax, and no limit on the kinds of expressions that can be represented.

With a particular emphasis on something that can be embedded easily into raw text as a markup language, without needing to remap a lot of characters like in HTML, but that also allows the full syntactic power of an arbitrary programming or data-representation language (eg Lisp or Prolog) if required. Also something that can be used to represent very low-level concerns, like circular references and read-time macro insertion and other tricky syntax-level things that Lisps tend to use reader macros to do.

Again, this is purely at the syntax level. Translating an actual language to work smoothly on top of this syntax is a much bigger project. But this syntax stands midway between raw S-expressions and 'heavier' syntaxes that have a notion of 'typed expressions' which S-expressions don't natively provide.

SXML is a great example of how XML can be translated into pure S-expressions. But I think using T-expressions we could translate, losslessly, all of XML/HTML, CSS, Javascript, Lisp, RDF/N3, etc, into something like Lisp 'forms' or Prolog 'terms' on top of a very thin layer of lists-plus-type-markers.

This would be fine. Whether we consider something an API has nothing to do with how we divide code into repositories or sections. Perhaps the key is to know who is calling us. I'd be happy to provide a guarantee to anyone who "registers" their repository with Anarki: if it's easy for me to run your codebase, I'll check it everytime I make a change to Anarki, and ensure that it continues to pass all its tests. If I break an interface, I'll fix all its callers and either push my changes or send the author a PR.

Exactly! It's a huge problem in the programmer world today that we are indisciplined about this distinction. There's a major difference between making a boundary just to make edits easy and "APIs". APIs are basically signalling to most people that they shouldn't be crossing them, and guaranteeing that anybody calling them will be able to upgrade without modifying their code. That is worlds away from just not being able to see everything on a single screen.

Since we use words like "abstraction" and "interface" for both situations, it is easy to cross this line without any awareness that something big and irreversible has changed.

Currently, we break up the API boundaries in Anarki by function and by file. We document the function APIs with docstrings, and we write verified documentation for them with (examples ...). We verify the file APIs with "foo.arc.t" unit test files.

If we put more effort into documenting or verifying those boundaries, it would hardly be a big change in how we use the language. We could perhaps add a bunch of "foo.arc.doc" files or something which could compile into introductory guides to what foo.arc has to offer (whereas the docstrings still act as API documentation). And we could perhaps add an alternative to Anarki's (require ...) that would load a library in a mode that faster faster, abiding by an interface declared in a "foo.arc.i" file.

It seems like the most drastic change in philosophy would happen if we broke Anarki apart into sections where if someone's working on one section and discovers they're breaking something in another section, they don't feel like they can fix it themselves.

I suppose that happens whenever people put Arc libraries in their own separate GitHub repos. There are actually a number of those repos out there, including my own (Lathe). Even aw, who was a strong believer in avoiding unnecessary black-box interface boundaries, would still put things in various repositories, and the hackinator tool would collect them into one file tree.

It also happens naturally with poor knowledge sharing. Seems like despite my attempts at explaining ns.arc, I'm still the only one with a very clear idea of how it works and what it's good for, so other people can't necessarily fix it when it breaks.

(Come to think of it, that makes ns.arc a great test case for pulling out an Anarki library into its own repo. That'd save other Anarki maintainers the trouble of having to fix it to keep Anarki's tests running.)

What you're saying would make sense if html.arc were a self-contained library for public use. Why does it have shortcuts for some HTML tags but not others? Someone in the public might want to use those others!

I think I'd approach this differently: Every client of html.arc can already say (tag foo ...) or (tag (foo ...) ...) to write an HTML tag. Verbosity is sometimes valuable; it can let us grep the codebase for something like "tag" to find all the places HTML tags are generated. So if the shortcuts of html.arc are causing distress over their inconsistency, I would remove all of them... from html.arc, at least.

Sometimes brevity is valuable too. It helps us fit a bunch of code on one page, for instance. So those things I would remove from html.arc could still belong in another file. Or maybe a few other files; sometimes utilities are self-consistent on their own but are inconsistent when viewed as part of a group.

I think your vision of how these shortcuts should work is self-consistent, and it would even work well as a Racket library. In Racket, we can use `(require (only-in ...))` or `(require (except-in ...))` to require some identifiers from a library without getting identifiers that clobber the ones we're using for other purposes. And we can use `(require (prefix-in foo: ...))` to require a library's entire set of exports under a prefix.

Anarki doesn't have Racket's `only-in`, `except-in`, or `prefix-in` abilities; it only has the ability to load everything in a file. But Anarki users can achieve something similar: The library itself can use more verbose names like `body>` or something, and users can write `(= body body>)` if they want to get rid of the verbosity.

---

I think I've actually seen that `body>` naming convention somewhere, but I can't find it. While I was looking for it, I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

https://github.com/arclanguage/anarki/blob/arc2.master/whtml...

  (w/html
    ('head ('title (prn "Page")))
    ('(body style "font-size: 200%")
      (prn "HELLO!")))

Did you perhaps have a similar opinion as well? I certainly recall other people asking for a package manager.

Are you sure? After all, these issues are being reported as problematic already [1,2].

[1] http://arclanguage.org/item?id=20490 [2] http://arclanguage.org/item?id=20524 "(cough (body) cough)"

> If and when Arc grows 10x or something...

Can you give examples? If you give concrete examples we can try to improve matters.

Many libraries have a long header comment up top, like rocketnia's ns.arc.

Anarki comes with online help. Try saying `(help deftem)` at the commandline. Admittedly online help is patchy inside lib/ but make it a habit to check it if you aren't already.

Lastly, it's worth looking inside the CHANGES/ directory for major features.

> ..or to know how stable the code is, because no one seems to use most of it.

If by 'stable' you mean "we shouldn't change it", then nothing is stable, inside or out of the lib/ directory.

If by 'stable' you mean "working as intended", then tests are our proxy for intent. There's tons of tests under lib/ and I run the tests fairly often. If a library doesn't have tests then I have a lot less to say about it.

So "no one seems to use most of it" feels like an exaggeration. Most code under lib/ is constantly being monitored for bitrot.

> The dependencies for news should be libraries, but they still seem to bake in too many assumptions and interdependencies to be useful as standalone libraries.

I think here you're cargo-culting[1] notions of "good practice" without considering their costs. Not everything that other people use is a good idea for Arc. We have a tiny number of users, which makes it better to keep the community in a well-integrated codebase rather than fragment it across several "lines". Interfaces have a tendency to balkanize users on one side or other of them. We don't want that; our tiny community doesn't really benefit from the scaling benefits that interfaces provide.

If and when Arc grows 10x or something, and we have three different sets of html generation libraries, we can think about drawing better lines. Until then I suggest a different "best practice": YAGNI[2]

[1] https://en.wikipedia.org/wiki/Cargo_cult

Keep in mind that if you use ns.arc to load an Arc library in a namespace, you'll usually have to work around problems with unhygienic macros on a case-by-case basis. I don't think this would typically be any easier than working around whatever problems were preventing you from loading it in the main namespace.