Hey just wanted to post my experience with audio programming on android. I developed an app recently and went thru hell and back to get it working properly with respect to some of the known issues with Android and the audio subsystem.
I programmed a OpenSL Implementation using ANDROIDSIMPLEBUFFERQUEUE and used all the properly real-time audio constructs such as a circular-buffer, no allocations of memory during sound playback and buffer "queue" etc etc etc.
However I found that while multitasking with a release build on my android phone sometimes the audio would pause or skip. Well what ended up giving me acceptable results (finally) was to encapsulate the audio layer in a "Service", specifically a foreground priority service. There are some excellent doc's covering this on the official android developer documentation.
So I recommend:
Implement a Service separate from your main activity and ensure Android recognizes it as Foreground
Use a circular buffer, statically allocated - boost has an excellent implementation - leverage that
Don't do any real-time allocation of memory while playing audio or filling in the next buffer queue
Compile you application using the arm7 abi, ensuring that optimizations are in place
Use lockfree constructs and don't block the thread.
Useful tips
Carandiru said:
Hey just wanted to post my experience with audio programming on android. I developed an app recently and went thru hell and back to get it working properly with respect to some of the known issues with Android and the audio subsystem.
I programmed a OpenSL Implementation using ANDROIDSIMPLEBUFFERQUEUE and used all the properly real-time audio constructs such as a circular-buffer, no allocations of memory during sound playback and buffer "queue" etc etc etc.
However I found that while multitasking with a release build on my android phone sometimes the audio would pause or skip. Well what ended up giving me acceptable results (finally) was to encapsulate the audio layer in a "Service", specifically a foreground priority service. There are some excellent doc's covering this on the official android developer documentation.
So I recommend:
Implement a Service separate from your main activity and ensure Android recognizes it as Foreground
Use a circular buffer, statically allocated - boost has an excellent implementation - leverage that
Don't do any real-time allocation of memory while playing audio or filling in the next buffer queue
Compile you application using the arm7 abi, ensuring that optimizations are in place
Use lockfree constructs and don't block the thread.
Click to expand...
Click to collapse
Good jobs guy, thanks to your tips. I know we can process audio with native SLES APIs more powerful.
I implemented reverb effect using opensl in android but unable to identify any effect in the output..so pls let me know what is the problem..I added permissions in manifest file..
I used OpenSL to play motor sounds in my racing game. But that library is not officially supported by Google. Android 4 for example removed ability to change playing speed and I had to start using SoundPool from java.
Greetings, dear sound modifications developers for Android OS. We want to ask you a few questions about feasibility and performance of some components inside your audio mods. We are very interested in features and their structure, resources & components. We are not great experts in the .nix systems. Simply, there are some situations what uncertainty is obvious even to a child. Let's start in order:
For the convenience of questions will be numbered
#1. The first thing we would like to know, what is the point from the availability of binaries and libraries of application-players in system/lib? Even if it is working, why would you add a library from Onkyo HF? This media application is an exclusive for their headphones & amplifiers, as well as their player app. It does not have unique properties such as hi-res audio, and other "goodies". And we didn’t find any circuits and connections, including those libs. Neither of the players does not become system directories, and if treated - what for? For the purpose of hardware decoding built by resampler at the level of the system to replace a core staff of the mixer? After all, the application has its own lib, furthermore - it was is not able to originally bypass ALL system. Only FX chain. This problem seems very paradoxical.
#2.Beats Audio. The first thing you need to consider - is deep binding of Beats api in flinger and srsfx library. If we open the original file from the M7 - we see that the effect starts with reference to sub_16A44 (substream?), and then goes on distributing multi-channels. And that's just for the first observations. In any case, even just to "rip" these lines is not possible (the only one who did something on porting it was @sun_dream). Well, we can replace the HTC M8 flinger with beats one entirety – it will work (probably the same trick will work with other 801qualc’s). But this trick works only under 5.0 OS – why did you say it is confirmed for MM? Besides, if we can set a modification to smartphone that already has beats by default the same configs replace themselves and this trick is not fully presented in the updater scripts. Furthermore – we are embarrassed with presence of "dummy-binaries" / bin / beatsbass, / bin / beatsnormal, / bin / spkamp, / bin / snd… etc. System will work safely without them, and their presence is not giving any help to set effects (beats, sony, etc) working.
#3.SRS. From small analyze dozen all-different versions, we have not seen a competent implementation of this technology. As well as previously mentioned beats, srs is proc-dependent (and any other libs) – so this will never work on Exynos or Mediatek as most of the technologies are deeply tied the same libaudioflinger. For samopisny or donorskiy. Even if it is working properly there are very interesting thing going in configs, for example Project Yume (probably all other mods by @PDesire) the eg compressor value is 15000. Why?? While the maximum is only 1! We consider these (and most of the libs) just "hanging" in the system and wasting its space. After al, speaking of SRSl, we should not forget that the SRS prefers Little Endian, while some fashion mods exhibit Float_64 (sic!) with unrealistic resampling.
#4.Sony DSEE-HX, Sony LDAC and other software technologies by Sony. Why it is presented for universal modifications, as its various types are unique for devices based on qualcomm? Or have you forced to work DSEE-HX, wired directly into the hardware libraries sony? If secret - how? There are no deep links with the system, or any other implementation. The only truly working tech port srom sony is – but even with this thing you should be careful. LDAC is hw implementation only.
#5. Alsa. As the developers, you know that Mediatek and Exynos does not have full alsa (the one you stick in your mods). In the low-level code at the kernel level they work within tinyalsa. While “usr” folder in your mods is designed to work in Qualcomm
environment. So there will never be any sampling above those, that MTK or Exynos got by default!
№ 6.Stegefright-decoders are not spelled out in media_codecs.Simple presence in system will not give an effect.
№ 7. 24/192 is impossible without touching a "true-low-level".Alsa isnt working on 90% devices,we was talking about this in previous number
We don't want to discredit anyone - maybe we missed something. We really wonder how mod authors have coped with porting many libraries. However, at this moment it looks more like a fraud and exorbitant increase in the system partition with a bunch of garbage files.
Sorry for our bad English.
With regards,
Android Modders from the Other Side.If you have questions-ask me.
@ahrion @Ben Feutrill @guitardedhero @PDesire @androidexpert35 @DeadRod @anandmore @A.R.I.S.E. Sound Systems @TheRoyalSeeker @mrchezco1995 @Hani K. and others.
Hello,
I'm trying to write a mediaplayer application for the MTCB. An issue that I'm not able to solve now is to stop the music from the other stock applications when my application start.
Does anybody has an idea about how to do that ? Seems that's not a possibility to use audio focus ...
Any help will be much appreciated.
mumu02 said:
Hello,
I'm trying to write a mediaplayer application for the MTCB. An issue that I'm not able to solve now is to stop the music from the other stock applications when my application start.
Does anybody has an idea about how to do that ? Seems that's not a possibility to use audio focus ...
Any help will be much appreciated.
Click to expand...
Click to collapse
Decompile stock apps and look inside.
mumu02 said:
Hello,
I'm trying to write a mediaplayer application for the MTCB. An issue that I'm not able to solve now is to stop the music from the other stock applications when my application start.
Does anybody has an idea about how to do that ? Seems that's not a possibility to use audio focus ...
Any help will be much appreciated.
Click to expand...
Click to collapse
You should try and PM Booroondook. Below is a Google-translation of his post on this matter:
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
The developers originally wanted to make a device completely replaces the car's radio. Moreover, up to the most complete simulation interface to motorists who allegedly used to. In addition, radio conceived such that it immediately ( "out of the box") would perform all the functions inherent in the car radio. Therefore it developed a set of so-called "factory" of applications that simulate the function of "advanced" regular radio - "Radio", "a DVD", "Music", "Video", "Photos", "the iPod," "Handsfree Bluetooth". By this set has also been added to the navigation functionality - as Application- "starter" selected by the user of the navigation program.
More precisely, mutual exclusion - integration system has been created for these applications. You work, "Radio", you run the "Music" - "Radio" is turned off. Launched the "Video" - disconnected "Music." And so on. It has even invented a special function (called by the MODE key), switching these applications in a circle - by analogy with the head unit.
These applications (according to developers) would have to cover all the needs of the motorist. And the Android operating system was the basis of fashion or marketing reasons, or both.
All would be good, but the developers have not considered (or ignored) the fact that the Android OS allows you to install additional applications. These additional user-installed applications do not "fit in" with the original logic of the functional simulation regular radio.
We take your specific example. You factory application "Music" chose a third-party, the establishment of additional "Google Music" app. It is clear that it is your choice to which you are entitled. But the developer is not able to assume that you install this application - and therefore your "Google Music" is not involved in "switching system" applications and "dampens" the other application.
What's in this case? Either put up with such behavior (ie, do not forget to manually turn off the "Radio" before turning on the "Google-Music"), or you can use some tricks to automate this business - good Android OS lets you do things with the help of supporting applications and system modules.
Members of our branches also have repeatedly faced with problems similar to yours. Force enthusiasts have developed tools that optimize the behavior of third-party applications such as media players.
Do not be lazy, look at the cap, there is a lot of links. Note the module Car Audio Service, as well as a set of tools from a reputable MVG-V70.
.............................................
Link to original post
.............................................
Unfortunately he is not that active on XDA
If someone is interested, finally it's very simple
private AudioManager am = null;
this.am = (AudioManager)context.getSystemService(Context.AUDIO_SERVICE);
// Cut sound from stock audio applications
this.am.setParameters("av_channel_exit=dtv");
// Enable sound from stock audio applications
this.am.setParameters("av_channel_enter=dtv");
mumu02 said:
If someone is interested, finally it's very simple
private AudioManager am = null;
this.am = (AudioManager)context.getSystemService(Context.AUDIO_SERVICE);
// Cut sound from stock audio applications
this.am.setParameters("av_channel_exit=dtv");
// Enable sound from stock audio applications
this.am.setParameters("av_channel_enter=dtv");
Click to expand...
Click to collapse
Could you please elaborate? I'm having the same issue, however I want to apply this behaviour to already existent apps, eg. Google Play Music (to pause everything else when Google Music starts)... Does your solution apply to my case and how should I procceed? Thank you so much in advance!
sarruma said:
Could you please elaborate? I'm having the same issue, however I want to apply this behaviour to already existent apps, eg. Google Play Music (to pause everything else when Google Music starts)... Does your solution apply to my case and how should I procceed? Thank you so much in advance!
Click to expand...
Click to collapse
I think one solution is to write a very simple application with just the mentionend code Inside. Then use tasker, and when you launch Google Play Music ask him to launch the small program before.
I can build the small app for you if you want to try ?
For my media player it's not working 100%. I'm able to cut off the sound when I launch my app but when I want to launch again the stock radio , it dosen't actually stop my mediaplayer since I can't get any notification Inside my app from the stock app.
mumu02 said:
I think one solution is to write a very simple application with just the mentionend code Inside. Then use tasker, and when you launch Google Play Music ask him to launch the small program before.
I can build the small app for you if you want to try ?
For my media player it's not working 100%. I'm able to cut off the sound when I launch my app but when I want to launch again the stock radio , it dosen't actually stop my mediaplayer since I can't get any notification Inside my app from the stock app.
Click to expand...
Click to collapse
What do you mean that you cannot get any notification?
What I've been doing till now is create a custom app using tasker (with a plugin called App factory, since my knowledge in programming doesn't go so far) and in fact use the created app to force close the radio app before launching, in my case, Google Play Music. But then, when I launch the radio app (or even the Bluetooth app)again, Google Play Music stops... So every shortcut of my preferred music app or even the mode.ini I use for the Xposed module I need to cycle through hu modes using my swc is associated not with the actual app, but with one custom made to suit my needs. Of course is kind of a mess, but so far haven't found anything better and working... Could you suggest anything better? Do you believe that there's an easier way for my case? Thank you so much for your help..!?
sarruma said:
What do you mean that you cannot get any notification?
What I've been doing till now is create a custom app using tasker (with a plugin called App factory, since my knowledge in programming doesn't go so far) and in fact use the created app to force close the radio app before launching, in my case, Google Play Music. But then, when I launch the radio app (or even the Bluetooth app)again, Google Play Music stops... So every shortcut of my preferred music app or even the mode.ini I use for the Xposed module I need to cycle through hu modes using my swc is associated not with the actual app, but with one custom made to suit my needs. Of course is kind of a mess, but so far haven't found anything better and working... Could you suggest anything better? Do you believe that there's an easier way for my case? Thank you so much for your help..!
Click to expand...
Click to collapse
What I mean, there is no communication between stock app and let's say a regular android app. I mean, when you launch a stock app, there is no way (at least I didn't find it) for another regular app to know that you launch the stock one.
In fact what I want to do is that when I launch my media player it stoppes the Stock app (This goal is reached) but then when I launch again a stock app, it should stop my mediaplyer app (Actually it just put the sound done. To sumarize what I want is that each time I bring an app in fordeground it should stopp all other (Music player). In fact it's the case between the stock media player and the stock radio.
So I think so far we are both at the same place
mumu02 said:
What I mean, there is no communication between stock app and let's say a regular android app. I mean, when you launch a stock app, there is no way (at least I didn't find it) for another regular app to know that you launch the stock one.
In fact what I want to do is that when I launch my media player it stoppes the Stock app (This goal is reached) but then when I launch again a stock app, it should stop my mediaplyer app (Actually it just put the sound done. To sumarize what I want is that each time I bring an app in fordeground it should stopp all other (Music player). In fact it's the case between the stock media player and the stock radio.
So I think so far we are both at the same place
Click to expand...
Click to collapse
Yeah, no matter how much I have searched, I haven't found any better solution..! But I'll keep trying, this solution doesn't satisfy me and isn't at all instinctive... Any way, I'll let you know if I find anything else! Cheers... ???
I thought all this was sorted about 3 years ago when @agentdr8 developed his XposedMTC app ?
typos1 said:
I thought all this was sorted about 3 years ago when @agentdr8 developed his XposedMTC app ?
Click to expand...
Click to collapse
My work on XMTC ceased a while back, but what was posted above was correct at the time. The audio output was controlled by the MCU, and triggered via different AudioManager object parameters.
The problem is, all Android app-based audio is done via 1 MCU output (av_channel=sys IIRC). So anyone designing an app would have to determine what the current av_channel parameter is set to. The stock MTC apps do that sort of (they just assume the newly-launched app is taking over audio control, and switches the av_channel).
agentdr8 said:
My work on XMTC ceased a while back, but what was posted above was correct at the time. The audio output was controlled by the MCU, and triggered via different AudioManager object parameters.
The problem is, all Android app-based audio is done via 1 MCU output (av_channel=sys IIRC). So anyone designing an app would have to determine what the current av_channel parameter is set to. The stock MTC apps do that sort of (they just assume the newly-launched app is taking over audio control, and switches the av_channel).
Click to expand...
Click to collapse
That was pre @7floor audio mod which bypasses the MCU and gives Android control of the audio IC, I wonder how XposedMTC works with that ? Not heard any one say that I remember.
mumu02 said:
If someone is interested, finally it's very simple
private AudioManager am = null;
this.am = (AudioManager)context.getSystemService(Context.AUDIO_SERVICE);
// Cut sound from stock audio applications
this.am.setParameters("av_channel_exit=dtv");
// Enable sound from stock audio applications
this.am.setParameters("av_channel_enter=dtv");
Click to expand...
Click to collapse
Does this code work as it is?
It is a java code isnt it?
You said it worked for you to cut the audio from stock apps, but inside there is the option to enable it as well? I don't understand?
I would like to stop dvd using this code in tasker. I assume I have to change dvr with dvd and remove this part : // Enable sound from stock audio applications
this.am.setParameters("av_channel_enter=dtv");
I am sorry but I dont unsterstand java.
A time ago I creaTed Modeswitch https://forum.xda-developers.com/an...t/app-modeswitch-spotify-swc-control-t3310966
An application to help with audio focus on our devices. I am now trying to optimize it to speed it up, and now allow it change the audio focus even if the mode circle didn't run ( that was a limitation of the releases until now)
I would appreciate your help
p_mike83 said:
Does this code work as it is?
It is a java code isnt it?
You said it worked for you to cut the audio from stock apps, but inside there is the option to enable it as well? I don't understand?
I would like to stop dvd using this code in tasker. I assume I have to change dvr with dvd and remove this part : // Enable sound from stock audio applications
this.am.setParameters("av_channel_enter=dtv");
I am sorry but I dont unsterstand java.
A time ago I creaTed Modeswitch https://forum.xda-developers.com/an...t/app-modeswitch-spotify-swc-control-t3310966
An application to help with audio focus on our devices. I am now trying to optimize it to speed it up, and now allow it change the audio focus even if the mode circle didn't run ( that was a limitation of the releases until now)
I would appreciate your help
Click to expand...
Click to collapse
Wouldnt it be easier to go into factory settings and uncheck "DVD" ? That would remove the DVD from the circle entirely.
typos1 said:
Wouldnt it be easier to go into factory settings and uncheck "DVD" ? That would remove the DVD from the circle entirely.
Click to expand...
Click to collapse
I am aware of this. Actually my app has already a setting to exclude dvd from mode circle. But its not what I am trying to do. If you are listening to a track on cd/dvd disk and you open spotify (or other music app) the audio from the track continues to play, and you have audio from 2 sources !?
p_mike83 said:
I am aware of this. Actually my app has already a setting to exclude dvd from mode circle. But its not what I am trying to do. If you are listening to a track on cd/dvd disk and you open spotify (or other music app) the audio from the track continues to play, and you have audio from 2 sources !?
Click to expand...
Click to collapse
Right, I thought this was sorted by @agentdr8 3 years ago with MTCXposed.
typos1 said:
Right, I thought this was sorted by @agentdr8 3 years ago with MTCXposed.
Click to expand...
Click to collapse
p_mike83 said:
I am aware of this. Actually my app has already a setting to exclude dvd from mode circle. But its not what I am trying to do. If you are listening to a track on cd/dvd disk and you open spotify (or other music app) the audio from the track continues to play, and you have audio from 2 sources !
Click to expand...
Click to collapse
It was something I tried to address with XMTC and had moderate success with. Unfortunately that code has aged beyond the current ROM releases, and I didn't have time to keep it up to date.
In order to catch all launches of audio-outputting apps, you'd need an Xposed module to hook into either the current Launcher process to know when apps are opened, or into the system ActivityManager. From there, you could interrogate the AudioManager instance and get/set parameters to flip audio sources. It was something I was looking into for XMTC, but never made any progress.
EDIT: Not sure if you've tried adding in this to switch away from the dvd source and back to the Android source:
Java:
am.setParameters("av_channel_exit=dvd");
am.setParameters("av_channel_enter=sys");
Also, I don't think any of this would matter if the @7floor mod was done. Everything should be piped into the standard Android audio streams at that point.
agentdr8 said:
It was something I tried to address with XMTC and had moderate success with. Unfortunately that code has aged beyond the current ROM releases, and I didn't have time to keep it up to date.
In order to catch all launches of audio-outputting apps, you'd need an Xposed module to hook into either the current Launcher process to know when apps are opened, or into the system ActivityManager. From there, you could interrogate the AudioManager instance and get/set parameters to flip audio sources. It was something I was looking into for XMTC, but never made any progress.
EDIT: Not sure if you've tried adding in this to switch away from the dvd source and back to the Android source:
Java:
am.setParameters("av_channel_exit=dvd");
am.setParameters("av_channel_enter=sys");
Also, I don't think any of this would matter if the @7floor mod was done. Everything should be piped into the standard Android audio streams at that point.
Click to expand...
Click to collapse
Oh, I thought that XposedMTC was still THE way to stop 2 sources from playing at the same time and to add extra sources into the src button (I use the built in apps myself so dont use XMTC for that)>
typos1 said:
Oh, I thought that XposedMTC was still THE way to stop 2 sources from playing at the same time and to add extra sources into the src button (I use the built in apps myself so dont used XMTC for that)>
Click to expand...
Click to collapse
Thanks! I came along this as I was trying to find the answer. (You have to remove "this" from the code). But anyway I already found an other work around ?. So time for testing my app. The first tests were successful!?
This is sort of a research thread and I hope someone here is willing to weigh in with their knowledge.
I'm a Ruby / Java / Python / JS / PHP developer, who did a little bit of Android game development during my studies back in 2012. I assume things have changed since then.
I'm working on a commercial project where we need a network controllable video player for LED TV's and/or video projectors. Currently, we are using a Raspberry Pi 3-based design with the OMX Player, but this board is somewhat weak and the player is cumbersome to interact with and has limitations. Especially when it comes to rendering multiple layers with transparency. I would like to work on a platform where I have a rich multimedia API for rendering sound and video with an object-oriented API.
I have obtained myself an Asus Tinker, which has an official Android distribution. This runs rather smooth and from what I can tell, the API's for Android appear rich and flexible. So my questions are:
1) Is it possible to develop a launcher / kiosk app, that will allow me to boot into a "blank" screen and allow the app to place video surfaces, image surfaces and text layers? I should also be able to interact with the sound card and playback PCM audio. I would like an API that supports audio mixing, amplification, etc... There is no direct user input on the device, so I will need a solution that does not present any status bars, google account wizards, wifi wizards, update prompts, notifications or anything. In fact, when the Tinker is powered on, there should ideally not be anything indicating that it's Android.
I guess what I'm asking for is kind of a console video game engine / SDK, minus game controller support.
2) What kind of libraries or API's would I need to dive into and understand? Where should I start?
3) How complex is it? What is the scope of it? How much development time? Days? Weeks? Months? Years? Would I need more developers with specific skills?
4) Is there any developer here who's interested in participating in such a project as a paid freelance developer?
5) Is there any alternative software/OS platforms I should look into? I want to be able to boot into a custom passive user interface that is remotely controlled over REST by another device. I would like to avoid dealing with low level implementation of video decoding and rendering, but at the same time I would prefer to have control over screen resolution, refresh rate, color depth and I would like to run a ssh server on the client, so it can be serviced. Ideally, the platform should be able to both stream from the internet, but also accept commands to download to local storage and play from there.
6) Is there any alternative hardware platform I should look into?
7) Anything else I should consider? Problems that I'll need to address / prepare for?