Hello Guys,

As shared with GuitarBob at https://forums.clamwin.com/viewtopic.php?p=20376#20376 , I'm starting developing a new UI to ClamWin, based on htmlPy. As it is a HTML interface, it will allow us update the UI without messing with ClamWin logical, and we will can to build ClamWin with newer Python versions (actually, ClamWin is stuck at Python 2.3 because there is some legacy dependencies). Most AV vendors use a HTML UI, you can see some samples at: https://sciter.com/



As you can see at screenshot above, it is not exactly beautiful right now, XD It happens because I didn't adapt the HTML code to htmlPy load the CSS code as it should. But as you can see, it is a standalone HTML application developed into Python, =)

The source code can be found at: https://github.com/coldscientist/clamwin-htmlpy-ui/tree/master

I know the UI sucks right now and it's not fully functional (my intention is finish the UI first, then link ClamWin actual back-end logical into it), but if you access the page of the project into the link above you'll see all the instructions to start building the interface, and it's a mess because it requires building ClamAV sources (and I'm not exactly a C/C++ dev, I lost hours trying to build it!), among others.

I want to post some mock-ups/prototypes of the interface for us to share ideas about how it should be. But, in resume, the intentions with the new UI would be:

* Integrate Quarantine into main program.
* Add type of virus at quarantine.
* Allow create an exception before releasing items from quarantine.
* Add option to take actions after scanning.
* Add friendly reports (and keep the full report for those who like it! =)).
* Allow send manual scanning/updates to background.
* Show total progress of scanning in percentage.
* Friendly download updates (and keep the full update output for those who likes it =)).

As shared with GuitarBob at the post above, I pretend to port ClamSentinel logical to make it built-in into ClamWin (providing a basic on-access scanner and heuristics to ClamWin), port the installer to a MSI one and allow ClamWin be centralized managed through ClamWin WebAdmin (https://github.com/coldscientist/clamwin-wa). But all of it'll take a little months to stay ready, as I'm working on this into my free time.

But it has a price: compatibility with Windows versions before Windows 2000 will be broken, because htmlPy requires Python 2.6, that is incompatible with olders versions of Windows. =( In fact, no one should use Windows 9x/3.11 into a daily base. Exist better alternatives to legacy systems, like Puppy Linux, and Wine is mature enough to support the majority legacy applications. There's the ReactOS project as a open source Windows alternative (https://www.reactos.org/), that's promising (actually, it's binary compatible with Windows Server 2003). It's cool seeing ClamWin as an alternative to protect legacy systems, but it makes harder to fix bugs into these versions. Maybe we can still provide the legacy UI with ClamWin installer for these versions, or let them go. Even the UI being incompatible, sherpya builds of ClamAV (clamscan.exe, freshclam.exe) will still be compatible with these platforms, so these users can run ClamWin through Command Prompt, and (in theory) they can replace the ClamWin executables with builds from sherpya's ClamAV (oss.netfarm.it/clamav/) and keep the executables from ClamWin actual UI untouched. And sherpya provide a basic UI into his site to update and scan through ClamAV engine too. Better than nothing. =)

The size of ClamWin installer will grow up exponentially (from the actual 8MB to ~50MB). Maybe we can tweak the py2exe build script to make it smaller.

Why I say "we"? Because it's a work that I want that evolve all of us:

For HTML/CSS devs, I'll try to create a gulp environment to build a HTML ready to htmlPy for facilitate the UI development.

For Python/C devs, there is instructions on how to build ClamAV, pyc module and the UI into the repository above.

And for those who loves ClamWin but are not devs (I'm not exactly a dev too!), please share your ideas, suggestions, call me names because compatibility break with legacy Windows versions (I tried the best that I could guys, I even tried installing older versions from dependencies and alternative builds of Python for Windows 9x, but newer modules are incompatible with older versions of Python).

I need the help of all of you! It's not my project, it's a project for the ClamWin Community!

I suggest that you keep it as simple as possible. A few attempts at improving ClamWin in the past have been started but quickly left to die. I think this is due to a couple of things: burnout from lack of help/ideas and too much work that is not compensated. I hope this doesn't happen to you.

I think it is a good idea to drop support for older versions of the Windows OS. Windows 98 was 20 years ago! XP support has been dropped by lots of AVs.

I also think that you should not be completely dependent upon Clam AV. There is always a chance that Cisco will drop support for it. The Clam Sentinel heuristics will help some, but they need to be updated. I can help with this--I already have a set of new heuristics and a better scoring system. In fact, there is a way to incorporate machine learning (in Python) into Clam Sentinel. All this is in the future, however. You are right to develop a good UI first.

Are you using the free versin of Sciter? It looks interesting. It is good of them to release the free version.

Regards,

Hello GuitarBob,

Yes, Windows XP is being abandoned by most AVs. Avast is the only one that I know that still supports XP. Maybe some majors AV commercial vendors, like Kaspersky, still supports this platform. htmlPy (in theory) supports Windows 2000 and up. I`m almost sure it can be successfully build for Windows XP and up at least. =)

When the UI be fully , I`m excited to start working into ClamSentinel port and your help will be more than welcome to work into a new heuristic engine as I have no specific knowledge into this area. A machine learning engine would be amazing, but I didn`t even know how to start. I read here on the forums that one student pretends to work into a machine learning engine for ClamWin, but he disappears after a while =(

It`s developed into a engine similar to sciter. It`s being developed into htmlPy (an opensource alternative), but the intention is the same: develop HTML UI applications. htmlPy is Python specific, so I think it`s even better working with it. =) Sadly, there isn`t much documentation and code examples.

When I sent an e-mail to xqrzd (Hazard Wizard developer) asking him if he still has ClamWin on-access scanner source or if he can help us into implement Hazard Wizard Real-time protection engine (or teach us the steps needed to build https://github.com/xqrzd/HazardShield from sources - I`m not a C dev, but maybe we could tweak the code to work) into ClamWin project (he still didn`t replied the e-mail - I`m not sure if he will), I wrote him into the mail that working into OpenSource projects is, sometimes, a sad job: most of people just complain about bugs and want quick fixes without gratitude from the work so far (as if we have obligation to fix it fast or as if we were a paid product), we gain little to non-budget through donations, there is little to non devs to help (but I have hopes to make ClamWin project gain more visibility through a new UI, and other features - so maybe we call attention of community and newer devs. Until there, I know that it`ll probably an alone job)... I really don`t want to give up ClamWin project. As I`m a businessman and we have no condition to buy a commercial AV solution to use into our environment right now, ClamWin is a priority to me. And as I worked into Brazilian government, I think how much ClamWin can help the needy people at my country and facilitates IT managers managing ClamWin centrally through ClamWin WebAdmin (I think it would be a killer feature for ClamWin, as none free AV offers a Console Management to update, clean, and setup antivirus into endpoints). =)

Yes - it requires a lot of work to develop a good Open Source application, and it never stops. Users accept it but do not appreciate/support it enough.

There was an attempt to get funding to hire a ClamWin development programmer (via revenue from Ask toolbar installations). A little money was received, but a programmer was never found.

It will be great if you can develop the UI. have machine learning documentation/tools and heuristics/scoring system if you ever get to that point.

Working on Clam Sentinel, we found that it would have taken $200,000 (US) to pay a developer to do what Andrea Russo did. He told me not to tell his wife!

Regards,

It says all.

Yes, I read about hiring a program here on the forums. I would not like to work into this mode (installing PUA into computers - I think that the ClamAV engine itself would remove it latter, as it supports detection of PUAs XD), but I know that a simply icon to send donations aren`t enough. I like Wikipedia/Wayback Machine approach: they make a lifting for the year and ask for donations with a big header on the top of the site. Would be amazing set objectives (like Self-defense, Real-time protection, etc) and preview the budget of investment of them. showing how much we get by now and how much we still need, and them ask for donations to catch that objective. There is crowdfunding sites that may work too, and after making transparent the money invested - I would like to see where and how much my money were invested if I donate to a project, because I can feel me stolen if I see no result from what I donate - even social projects out of computer ones. I believe that even seeing donation as a objective, the people may donate more and higher values to help us catch the objective. I want to put warnings into the UI about missing features that would be amazing for us to do and make a link available for donations or with instructions for devs helping in implement missing features - like self-defense, real-time protection, etc).

I read about that too, and were impressed by that value! Hope his wife would never known! XD It makes me admire the efforts of everyone involved in ClamSentinel (including your work) even more. Someone paid with a higher price than money - time - to provide to millions people into the world a program to provide them protection - it`s not a simple software, it doesn`t matter if it`s heuristics are outdated nowadays: someone donated time, efforts, to make it happen, into the best way that they can. I believe that the efforts on ClamWin as a whole (including ClamAV engine port, etc) is not much different than this!

I have sent the ClamWin developers an email about your intentions and that you need some direction. This is the second time I have done so.

It might be easier/better to use the Clam AV Windows port from the Clam AV web site. It will have all the functionality from Clam AV. I think that the port used by ClamWin does not have all the functionality. For instance, when I was sigmaking at Clam AV, with ClamWin version .95, I lost the ability to normalize files so that I could get the code from which to prepare good virus signatures.--especially on HTML files. I have not been able to get an HTML signature since then.

Regards,

Hello GuitarBob,

I sent you a mail to your personal mail because I couldn't submit a post here on the forums because an error message appeared every time I tried, and you replied from there, but I will try to reply here on the forums again so the people can go along with us in the discuss and maybe someone appears with some suggestion or offering help.

Did you already test the new ClamAV 0.100? I'm not sure it will bring a big novelty, I read into ClamAV blog that it's more like a version number adjustment from the 0.99.x series (https://blog.clamav.net/2018/01/clamav-version-number-adjustment.html).

It's amazing that you knew Alch and Sherpya personally, and that Alch was already married! It explains a lot of things. If Sherpya is from Italy too, maybe he new Andrea Russo (ClamSentinel developer) personally too - at least, if I do live there, I would try to know. I think that Alch would love to known him, at least, to share some ideas about the Clam project. One thing I know: into the new ClamWin UI and into the config file, I'll honor Andrea Russo work calling the on-access scanner settings as "Sentinel", at least, on the English User Interface version. It's not a thing that will be debatable with the community, I would love to talk to Sherpya, know his motivations with ClamAV port, I'm not sure he does the port exclusively for ClamWin (if he does, it's great!), among other things. Maybe we could find things in common about the ClamAV project that we could build together. I really want to join the team and help to keep the ClamWin community alive.

"Since Clam AV was/is primarily an email scanner, there was never a need for a fast AV that does anything more than scan email attachments. [...] Most email services have several other AVs also, so there never has been a need for Clam AV to be a great AV." You said it all.

When I start working in ClamSentinel port to Python, I would appreciate your help in developing further heuristics to ClamWin. One that I would love to develop is some kind of Ransomware protection (that is almost nonexistent into Free AV products - even on paid ones [Sophos sells Ransomware protection apart]) - but I'm not sure how to implement it (not technically speaking, as ClamSentinel already has a code that, if adapted, can easily detect possible ransomware actions because it already monitors file creation process, per example). Maybe detecting if 1000 files were encrypted and deleted in a short period of time? I'm not sure. When I develop the UI, I pretend to allow changing it to developer mode, with extended logs and making possible to enable heuristics only through the ClamWin UI, making the debug more easily.

I'll try to not burnout into ClamWin project (I pretend to work on it at least 1-2 hours into working days, so maybe took a little time to have a functional UI and others stuff, but I don't pretend to give up - in my country, is common seeing ClamWin protecting governmental computers, public schools, Windows Servers - not everyone has sufficient budget to buy a commercial solution, and providing a "Basic signatures/heuristics will detect most viruses/malware [...] common sense heuristics, and some behavior/registry checking", as you said into the mail, for free, for everyone, would be amazing!), and I know it can be a walk that maybe only me and you will do together into the beginning (I really appreciate if you be the #2 beta tester from the new UI!). You said into the e-mail that maybe the 0.100.x ClamAV series may be your last into the project. I hope to have some UI to show you before June or July, to you to show to Sherpya as the project is going. Maybe it can motivate him (and you!) even more into the project!

I have not tested Clam AV 0.100. I am a Windows user only, and I have to wait until ClamWin has a similar version ready. Even when I was a sigmaker with Clam AV, I still used Windows. This version will never be released--it is only for test purposes.

Andrea Russo and Sherpya seem to have different philosophies regarding code development, and I do not believe they know each other.

I do not know how Sherpya gets involved with the different ports.

I am sure that Andrea Russo would appreciate an acknowledgment of "Sentinel."

ClamWin could certainly use another team member. What it desperately needs now is some direction. I do not know how many users ClamWin currently has, but I think it must be much less than 100,000 (one hundred thousand) now. At one time, it was used by several hundred thousand people. I did not know that it is used a lot in Brazil.

Many ransomware detection modules seem to trigger on encryption (under certain conditions). The philosophy behind the Clam Sentinel heuristics was to use common sense and keep detection as simple as possible. There are certain things that a malware does that is hard to hide. If it does some very important things or if it does enough things, it will often describe itself as malware. I think the existing Clam Sentinel heuristics are capable of detecting lots of current malware if the heuristics and the heuristic scoring system are updated. No AV is capable of detecting every malware. The most important thing an AV can do is to detect a large amount of the malware that its users are likely to encounter. Pareto's Law says that if you can get 80% of maximum, that is good enough, as the effort to get that additional 20% is too great. I do not believe that AVs can get 99.XX% or even 95% of all possible malware like they seem to do on some tests. The undetectable malware is not on those tests!

Regards,