Well, I think I have been misunderstood. The problem with bringing Gentoo into the conversation, is that people immediately believe (with good reason) that I am envisoning a harrowingly close-to-the-metal experience for the home user. Nothing could be further from the truth -- I am focussing on one and only exactly one aspect of Gentoo, that is it's installation system. What happens between the time you type "emerge wordprocessor" and the time it finishes it's compilation/installation. In this one aspect, Gentoo shines as a very easy-to-use system. Put a GUI front-end around that process, and it becomes trivially easy. Imagine -- the user runs "software installer" which automatically looks about for software to install (checking the CD, floppy, and internet without needing any direction to do so). It finds software and an organized list begins to populate itself. Some have price tags attached, most are free.

The user just clicks on a few, perhaps one $15 game and five free utilities or trinkets. Noting the game, it pops up a (completely standardized) enter-all-your-info screen for credit card number, comments and so on. The download begins, and a time estimate pops up. With the time estimate, the user also gets an abbreviated list of just the software that was selected. The user realizes that the game is on the botton of the list (last), so he/she drags it to the top. The currently software gets shifted under it and the game starts to download/install. It does so quickly, and the user plays that white the rest download/install. That's the user experience. What I've been talking about above in the thread is just the "how", as a developer.

In the background, there is the equivilent of an "emerge" process. Unlike Gentoo, this system would never show dependancies which are not also leaf-nodes. For example, if there's no reason someone would download a graphics library unless they also ran something which needed it, that graphics library wouldn't end up on any list (but would be included in the download, compilation, and time-estimated-to-completion of the first software that needed it).

As for splitting the compiler in half -- you may wish to explore the Oberon/Juice project, which did exactly that -- Oberon code is compiled into an intermediate language which is saved out. Then you could download that (as it happens, into a web browser) where the compilation is completed and machine code for the local CPU is produced. This system is good -- but the TenDRA compiler is new. I'd much rather go with developed technology -- GCC. Now if someone can figure out how to split GCC in half (it is already split into a front-end and back-end design-wise -- just no way to capture the intermediate output) then we'd really have a winner.

Hi,

One of us must have a wrong idea of what TenDRA is. I haven't had the time to read much of the TenDRA site, but I think the system works on a lower level.

TenDRA is platform-independent, but not OS-independent. It is not like .NET where also higher-level APIs are abstracted (edit - well, actually, I think MS just provide the classes for all supported platforms).

From what I understand, TenDRA is simply a compiler cut in half (as Fabio explained in this thread). After the first step of compilation, translation of C/C++ code to the intermediate language, a "binary" is generated. This intermediate language is abstracted well enough from any processor architecture, it doesn't care about the width of RAM and busses for example, it could look alot like C (I assume).

The "binary" can be further compiled by the other half of the compiler, which is processor-specific, translating the intermediate language to opcodes specific to the processor. Thus this second half of the compiler is processor-specific.

As in languages as C/C++ there basically is no knowledge of the OS APIs -- you just call functions from libraries -- the TenDRA intermediate language also only just stores calls to these libraries. (edit - probably the same in .NET, just that MS provide classes for other platforms - I should read up on this :)

So, I'm afraid it's not a solution for the incompatible AmigaOS APIs :/ That, or I'm completely wrong on this :)

http://www.ten15.org/ seems to have some more documentation on TenDRA.

to Fabio:
You said: "The compiler is able to perform most of the optimizations on the intermediate language, which is still platform-independent, then it does some more (mostly instruction reordering and peephole optimizations) after it translates this intermediate language in the target's CPU language."

But I now read on the TenDRA site: "ANDF [the intermediary language] focuses on abstracting high level languages instead of assembler languages, as is common with most compilers. This makes TenDRA a powerful tool in code verification and checking."

I assume traditional compilers do optimize alot in their intermediate language, but that is a lower-level language. I expect most optimisations to be done with the "second" compile step, as it's compiling from a high-level language.

Mmm, this isn't really a big problem though, except that the "second" compilers will need to be optimized (one per platform), which is just more work.

by paolone on 2004/4/24 6:46:23


Quote:

With nautilus, it more or less can do that for you, even a tar.gz file.

Dammy

A very interesting topic. I thought I would add my 2 cents as both a user and a support tech. Also, just some of my own thoughts. I like the TenDRA idea myself and if you don't mind I will explain why.

Lets just pretend for a moment that Aros has taken this approch and being open source, versions of, whatever it might be called, are avaliable for all Amiga like platforms. Even though all the Amiga like OSes share a great deal of their API any company thinking of writing/porting software would have to look at each OS as alomost a sperate thing. Support for one reason, not all the API's are exactly the same so some code will have to be different between each platform. This means seperate devlopement trees to keep track of. The Amiga devlopment communtiy is not very big and cannot afford the overhead assoicated with that. Deployment would also be differnt. If we had a sysem like TenDRA then they could support all the Amiga like OSes with the same code and "binary", I know that binary is not he correct word but I cannot think of the correct one.

Also, instead of looking at each Amgia like OS as having a seperate userbase, the Amgia community can be looked as a whole, thus making us a much larger group. No matter what you say, at this time, each OS is looked at speratly and no one platform has a large userbase. But together we are larger and can grow a quicker.

I do agree that native apps are faster and there is no preventing that but there are some things that can be done to improve performance. I use C#, no I do not want .net on Amiga, but I can set a flag that tells .net to "compile this file at install time" so that pieces that will most benifit from being compliled natively can be. I do not have to compile each version myself and do not have to worry about it. Exact same code, exact same file. Not all pieces can be done this way but it is a nice compromise. TenDRA being OSS and not just an open standard allows optimizations for each platform. I do not think that something like TenDRA or .net is good for all types of applications but it does cover a rather broad base.

Right now the 68k emulation is the only way to do this type of thing on Amiga like platforms right now. We all know of the limitaions of this but it does allow for a larger base of apps. By doing something like this we can still have common apps, better performace that 68k emulation etc. Also, as time goes on each platform is bound to diverge making the API less and less driectly compatiable. Something of this nature could keep some common ground between us all and still allow for native apps to take advantage of that platforms unique strenghts. Still making the Amiga "niche" community a little larger overall.

I did have some questions about somethig like TenDRA though. Right now all the Amgia based OSes share a lot of common ground. In .net a whole new set of libaries had to be made. Would TenDRA be able to make use of the common APIs already there, decreasing the size or would all new libraies have to be writen even using the same API?

Also this would be the R in AROS. If something like this can be done in Aros and shown to work and work well. I am sure the rest of the Amiga community would pick up on it.

Quote:

That's exactly what I mean when I say that Windows is blocky and fat, but Linux isn't the right solution for desktop users. Linux completely lacks the "final user"-friendly approach. It assumes there is a root with power and - above all - KNOWLEDGE, which is exactly what a newbie can't have. It's not only a matter of architecture. It's matter of philosophy. Try to propose a real user-friendly linux, and you simply won't have linux. You will have a sort of kind-different-windows with user acting always as root. Every linux advanced user should understand what a tragedy should be a computer with a newbie acting as root.

AmigaOS is a sort of intermediate OS which can be friendly with developers and suitable for users. That's what Linux has completely failed to be. And this is my main hope for AROS: a fresh alternative which doesn't require a master in IT to be used. But PLEASE avoid source-only distribution of software for AROS.

In the fine old linux days, I learnt how to ./configure, make & make install the application I needed, rigorously taken in gz tarballs. But the best way to install an application is clicking twice on a icon. It's not so difficult to fuse those two worlds togeteher. We need only a simple executable script, and a user friendly interface during the compilation procedure. What I mean, is that FROM THE USER POINT OF VIEW, there won't be difference between installing a binary program, and compiling & installing an application starting from the source code. Click twice, and you get it.

In order to do this, however, we need common interfaces for compilers, and the needed files installed in the right position immediately, starting from AROS-itself installation.

Quote:

Okay... I'll address each point. Keep in mind the whole time that, #1: I don't think that the idea of a virtual machine is bad, I just prefer source compilation for the reasons I've gone over and will go over here. And, #2: I'm always comparing to a virtual machine binary distribution, not to a one-architecture native binary distribution.

Quote:

It depends on the build system. Under Gentoo, of the over 500 packages I've built, 4 have failed (temporarily) to build (of those built from stable). That's a success rate of over 99% This is a better success rate than I had under the old AmigaOS on an original Amiga, using binary only software. It's also comparable to the success rate for installation that I've had for Windows, and much, much higher than for RedHat. This is on a chaotic, ever changing system. If a binary installation fails, due to whatever variance in the users target system (hardware or software) -- assuming that they are non-geeky users -- they are at a complete loss on how to proceed just like they would have been with a source build. Except to contact the original publisher and ask for help. In the unstable builds, I've had a lot more than 4 of course, but it's unfair to include them. Of those two which had source problems, patches were almost immediately available, autmatically applied next time I initiated the install process.

Expert users, however, can usually recover even without that. Of the four failures I had, one was hardware, (machine overheated), one was network (which recovered), one was a problem with the source, which I could not fix, and the last was a source problem which I could and did. If the source were not available, then my *final* failure rate would have been one higher. I realise it means nothing to a non-geeky user -- but they are not worse off than they would be with binary-only packages.

Quote:

This is true. Perhaps, however, AROS has at least some immunity -- it's software tends to be small. A lot of functionality is in reusable libraries. This is a real problem, however, it does make the situation worse than binary installation. I would say that the total benefits markedly outweight this disadvantage, however. It's also possible to add pre-built native (not virtual-machine) binaries for popular architectures, to speed up installation (although personally I'd oppose that approach, more variables to keep track of). Remember, games don't usually have that much code to compile -- it's mostly multimedia -- which does not change from architecture to architecture (usually) and does not need to be compiled (although sometimes scaled down for older/slower hardware, but that's usually done at run-time)

Quote:

True -- but I'm comparing to a virtual machine environment, which also takes up disk space. And memory. And CPU time. The build environment takes up disk space all the time, but only takes memory and CPU time during a build. After which, this approach saves CPU time. Admittedly, it takes up more disk space than a virtual machine approach would. Choosing between sacrificing disk space, and CPU time for me is easy -- especially on the generally very disk-light AROS.

Quote:

This I must firmly disagree with.

AROS would hardly be the first system which: #1, ran on multiple platforms, #2, built everything from source, and #3, had a thriving proprietary market.

Another group of systems which fall into that category are several embedded system platforms. There are two common solutions, both good:

#1: Distribute the source -- but not as free software. It's still proprietary. If you pirate it, you are pirating it exactly as if you pirated a binary, with no legal or moral difference. Just because you have the source does not mean the software is Free as in speech (one effect of the work of the free software community, probably unintentional, is that people have come to believe that if you have the source, the software is Free. But, historically and legally , this idea is nonsense), or even free as in beer. I've worked a lot with software distributed as proprietary source before -- it's quite common. As for being easy to pirate -- no more (or less) so than a binary. The difference is that you might be able to read it and modify it (legally or illegally) or your competitors might be able to reverse engineer it more easily.

A better approach, which solves these problems --

#2: Obfuscate, then distribute the source, but not as free software.

Obfuscated code is code which has had all the white space and comments removed, and symbol names randomized (and usually shortened). Good code obfuscators (they are automated programs which require no human input) can also reorganize code and rewrite code in ways which are semantically meaningless (so it still runs) but tend to eliminate any possible information about what might have gone on in the mind of the programmer when it was written. It's even possible to optimize source code (which makes it less readable) in addition to or instead of obfuscation, which would make your software run faster or use less space! It's "source", but it's almost completely unreadable (to a human) -- just as unreadable as a binary run through a dissassembler (or just as readable, you might say). Your would-be pirate or corporate thief is no better off (or worse off) with such code than they would be with the machine-independant binaries. Piracy is no easier (or harder), and interpreting the source (in the case of obfuscated code) is no easier or harder than with an intelligent dissasembler. In fact, you'd probably be better off, if you wanted to reverse engineer well-obfuscated code, to compile it, then run it through a good dissasembler. Far more readable. Code-beautifiers won't help you here -- they can't repair the munged symbol names, the reorganized instructions, or the stripped comments, only the white space -- and even there they can do so only algorythmically. White space is usually informationless anyway (in the sense that it should always be possible to determine where and what kind of white-space to use), or should be, and if it were informative, obfuscating it would eliminate that information. At least the good dissasemblers try to generate reasonable symbol names.

Quote:

As it's getting late, this will be in telegram-style, likely full of spelling errors...

compiling from source:
* lot's of errors can occur. Errors that mean nothing to the less-geeky user
* large programs require long compile times
* the build-environment takes up disk-space
* not an option for commercial software
remember that games make an OS popular, popular games are commercial products

I think software companies are afraid of developing software for Linux because of the general open source/free mentality that surrounds it.

For the PC hobbyist, it's nice to be able to download and install free (optionally open source) software. However, companies like to have professional support from the software developer, that comes at a price.

Let's not force developers to go open source or to bugger off instead. Lets leave the choice to them.

One thing you seem to forget is that one option does not make the others disappear. Sources will still be available to people who want to compile from them.

It is a good idea to design a system that can handle the compilation of a "package" transparently for the user though. That's a great option for the tech-heads.

I assume you haven't read Fabio's reply on the subject, and were also not aware of TenDRA. This is surely a much more attractive option than a VM, don't you think?

Goodnight :)

I recommend that we, instead of implementing a virtual machine for AROS, focus on improving an existing feature that can give us the same benefits, but do a better job of it than any virtual machine can.

Well, I have to say, after some thought I've decided against the virtual machine approach to cross-platform-ability. Basically, I use a working cross-platform OS right now, as my daily desktop OS. It works just fine, with no virtual machine. In fact, it's faster than a virtual machine would have been, had it worked on that principle. For Gentoo, the one I use, or any source based distribution for Linux or BSD, you just select the software that you want, and it grabs the source (common for all architectures) and builds it right on your box. Therefore, it runs nativelly, fast, on the hardware. In fact, sometimes it's a little faster than even an ordinary OS, because useful variations between your specific CPU (in my case a Nehemiah M10000) and the generic, runs-everything, least common denominator CPU (in this case probably a 486) are taken advantage of for speed. So rather than tending to cost performance, and other resources (basically CPU, memory, and storage resources) this approach tends to minimize resource consumption (CPU especially) and perform better, overall, than even a comparable generic approach (RedHat, Mandrake on one side, Gentoo, ArkLinux and so on on the other).

Usability problems can be solved, there's nothing inherent in the idea of source distribution which makes things more difficult to install -- it's just that it tends to show up on distributions which are generally geared towards advanced users. Most of my love of the Amiga goes back to the fact that, even originally, the Amiga was kind to both beginners and experts. Unlike Linux, which "is picky about who it's friends are", and unlike (traditionally) the Mac, which tends to pursue "keep it simple" types.

Basically, as developers, we are already using a "build-from-source" approach. I think keeping that feature and adding it to AROS's installer is a good idea.

I believe that most of the desire to build a virtual machine layer comes from the need or percieved need to run proprietary binaries on multiple platforms. I believe that effort spent in this is wasted, perhaps even as far as inappropriate considering the rest of the project.

Well, I'll reveal my biases and give an explaination.

AROS is open source.

Amiga people have always been the sharing type -- through both illegal means (piracy clubs) and legal means ("public domain" licenses). In many ways Amiga developers predicted the open source movement. I'm opposed to copyright violations, but want to protect the desire of Amiga'ites to share.

My approach to running old Amiga binaries, is this: Either don't, or if you really want to, buy Amiga Forever, or copy your original Amiga ROMS's to a file, and run UAE under AROS.

I bought (every version of) Amiga Forever and have also copied the ROMS from my (defunct) Amiga 1200.

What would be good for AROS I believe would be to continue the tradition of sharing among its users (we already do that), and to keep the "powerful but simple" elegance of the Amiga's OS, as well as the "this is a desktop OS, made for users, albeit creative and smart users" philosophy. In that regard, I'd like to see Installer to be improved so that it can automatically install a "package" (which would be the source to a software) right there on the spot for you, with a pretty progress bar, cancel button, restartability, uninstallability and so on. Best that it has the ability to recognize which hardware you are on for you and make all appropriate adjustments. It's precisely because I don't want another Gentoo that I'm proposing this idea. The power of Gentoo, but the famous simplicity of AmigaOS.

Forget about binary distributions, like old Amiga software and new AROS software distributed as binaries. Who cares? Some think we need that ability to be "successful", but I don't believe so, and can point to numerous, extraordinarily successful projects which did not need that. Linux people care very little about binary compability, yet Linux is doing wonderfully, far better than the Amiga ever did, for example. Numerous games of higher quality than the Amiga ever had are being produced, in open source, right now. Powerful productivity suites far better than what was availiable for the Amiga at any time exist now, as open source. Powerful multimedia software exists too, I just used Audacity yesterday and am struck on how much it reminds me of old Amiga software. Simple such that I picked it up right away, but immense power, I could basically just do whatever crossed my mind to do with the sound -- unlike SoundEdit for the Mac.

Anyway, I guess I just wanted to rant and suggest this: That we actually make use of the "open source" feature of AROS. It's not just a philosophy, or a cost (dollars) or even a freedom -- it's a feature which can be put to technical advantage. There's no need to invent or implment yet another feature (virtual machine technology) which gives not quite the same features, and does a much worse job of it in general.

So to recap:

Pros and cons of virtual machine:
Pros: Runs new AROS binaries and allows developers to distribute binaries for AROS which will run on multiple platforms.
Cons: Slower than native, takes slightly more memory and harddrive space, does not exist and needs to be developed for AROS, or coopted (from java?)

Pros and cons of source distribution:
Pros: Consistent with existing OS philosophy, technology partially implemented already, as fast or faster than typical native processing, takes extra memory only during build.
Cons: Requires build environment to be installed as standard on every installation, only works on source-available projects (which at the moment anyway, is all of them).

Quote:

Ah, I knew someone in the open-source community must've been busy executing such an idea :)
I suppose it wouldn't be too difficult to get TenDRA applications to run on AROS.
But hypothetically speaking, how much (roughly) of AROS could be ported to such a platform-independent code?
(Understand I don't want to force AROS to go totally platform independent, this is all just out of interest and curiosity :)

Actually, I was also thinking of using a higher level language instead of a virtual CPU, but then on the OS level. That's probably a bit too high. Good thing there are smarter people out there that do know how to do it ;)

Regards

Quote:

Well, 3.1 is the foundation AROS is built upon, but it has its own APIs, and will have many more in the future, and nothing stops you from using AROS' own APIs.

Regardless, I think darklite was talking about simplicity, speed and charm from the user point of view, rather than the programmer's.

Quote:

Well, you always write for a theoretical virtual machine, unless you write stuff in ASM directly. The virtual machine is the one the compiler uses internally (all compilers use an intermediate language before outputting the asm code for the target CPU), but this virtual machine is invisible to you, who write in the high level language of choice.

Quote:

You need to recompile, if things are handled as they should, or you need to remove assumptions about types sizes, if things have not been handled as they should have.

A virtual machine is no different, in this regard: either you compile your SW for a virtual machine whose word has the highest possible number of bits (64 bits should be enough for everybody in the next 20 years), or you need to have as many virtual machines as are the different word's sizes of the target CPU's you address, and compile your program as many times. This is certainly better than compiling your program for all the different CPU's, because many CPU's share the same word's size.

Quote:

Not necessarily it has to be an emulator. As said, every compiler already has a VM "built in": the compiler first translates the high level language in a language suited for this internal VM, then it translates this other language in the one for the target CPU. The compiler is able to perform most of the optimizations on the intermediate language, which is still platform-independent, then it does some more (mostly instruction reordering and peephole optimizations) after it translates this intermediate language in the target's CPU language.

Thus, you could distribute binaries compiled for the compiler's internal VM, and then have another compiler, much simpler than the "real" one, on the target machine, which would translate on the fly, and do the rest of the optimizations, this intermediate code in the target CPU' language, all being transparent to the user, who'd not notice any speed penalty when loading the program, because the translation process would happen while loading the binary, and because the speed of the modern CPU's is quite high, and also because the translation process itself wouldn't be that expensive, in terms of CPU power.

There already exists a compiler that allows you to distribute such binaries: its name is TenDRA, its intermediate language is called ANDF, and it also has a quite friendly license (BSD).